Vigilia.
The warnings
Extinction riskFrontier labMember of Technical Staff, Anthropic4 entries on the record

Follow

Holden Karnofsky

Co-founder of GiveWell and Open Philanthropy, author of the “Most Important Century” series and since 2025 a member of Anthropic’s technical staff, who warns that racing to build transformative AI as fast as possible risks the permanent disempowerment of humanity.

Roles

  1. 20032007Analyst, Bridgewater AssociatesSource
  2. 2007presentCo-founder, GiveWellSource
  3. 20142023Co-founder, CEO and later co-CEO, Open PhilanthropyTook leave in 2023, then moved to a Director of AI Strategy role.Source
  4. 20172021Board member, OpenAISource
  5. 2024Jan 2025Visiting Scholar, Carnegie Endowment for International PeaceWorked on international security risks from AI, including if-then commitments.Source
  6. Jan 2025presentMember of Technical Staff, AnthropicWorks on the Responsible Scaling Policy and AI safety planning.Source

On the record

  1. 30 Oct 2025 · Podcast · 80,000 Hours Podcast

    We're just racing to do this as fast as we possibly can. That seems like a really, really bad way to handle this technology.

    Member of Technical Staff, Anthropic at the time

    Recorded 25 and 28 July 2025, nine months into his Anthropic role.

    Holden Karnofsky on dozens of opportunities to make AI safer lying on the table — and all his AGI takes

  2. 13 Sept 2024 · Paper · Carnegie Endowment for International Peace

    AI could pose a variety of catastrophic risks to international security in several domains, including the proliferation and acceleration of cyberoffense capabilities

    Visiting Scholar, Carnegie Endowment for International Peace at the time

    If-Then Commitments for AI Risk Reduction

  3. 9 Jun 2022 · Post · Cold Takes

    the kind of AI I've discussed could defeat all of humanity combined, if (for whatever reason) it were pointed toward that goal.

    Co-founder, CEO and later co-CEO, Open Philanthropy at the time

    AI Could Defeat All Of Us Combined

  4. 13 Jul 2021 · Post · Cold Takes

    this century could determine the entire future of the galaxy for tens of billions of years, or more.

    Board member, OpenAI at the time

    Opening post of the "Most Important Century" series.

    All Possible Views About Humanity's Future Are Wild