Vigilia.
← Les avertissements
Risque d’extinctionSociété civileBoard member, Safety and Security Committee, OpenAI Foundation4 entrées au registre

Suivre

Paul Christiano

Chercheur en alignement qui a dirigé l’équipe d’alignement des modèles de langage d’OpenAI, fondé l’Alignment Research Center et dirigé la sécurité de l’IA au US AI Safety Institute à partir de 2024 ; il situe la probabilité d’une prise de contrôle par l’IA à quelques dizaines de pour cent.

Fonctions

  1. 2017 – 2021Researcher; led the language model alignment team, OpenAICo-authored foundational work on reinforcement learning from human feedback.Source →
  2. avr. 2021 – aujourd’huiFounder and Executive Director, Alignment Research CenterCurrently listed as executive director on his own site, after returning from government.Source →
  3. sept. 2023 – aujourd’huiAdvisory board member, UK Frontier AI TaskforceSource →
  4. 16 avr. 2024 – 2026Head of AI Safety, US AI Safety Institute, NIST (renamed Center for AI Standards and Innovation, June 2025)Reported in August 2026 to have stepped back to a part-time advisor role in July 2026; the reporting (X) could not be fetched, but his site now lists him as technical advisor.Source →
  5. 2026 – aujourd’huiTechnical advisor, Center for AI Standards and Innovation, NISTSource →
  6. 9 sept. 2026 – aujourd’huiBoard member, Safety and Security Committee, OpenAI FoundationAppointed to the OpenAI Foundation Board and its Safety and Security Committee, and a non-voting observer on the OpenAI Group PBC Board, while remaining a Senior Technical Advisor at CAISI recused from OpenAI matters; his own statement of the same day (paulfchristiano.substack.com) calls it "the OpenAI nonprofit board".Source →

Au registre

  1. 9 sept. 2026 · Publication · Substack

    Based on the recent trajectory of capabilities and the continued difficulty of alignment, I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term. I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.

    Board member, Safety and Security Committee, OpenAI Foundation à l’époque

    Written on joining the OpenAI Foundation board and its Safety and Security Committee; footnote 1 puts his all-things-considered risk of catastrophic loss of control at 4% over the next year and 15% over the next three.

    Personal statement on joining the OpenAI board →

  2. 24 oct. 2023 · Publication · LessWrong

    I think AI developers are not prepared to work with very powerful AI systems. They don't have the scientific understanding to deploy superhuman AI systems without considerable risk

    Advisory board member, UK Frontier AI Taskforce à l’époque

    Argues that a good responsible scaling policy must name the conditions under which development would be paused.

    Thoughts on responsible scaling policies and regulation →

  3. 27 avr. 2023 · Publication · LessWrong

    Probability of an AI takeover: 22% … Probability that humanity has somehow irreversibly messed up our future within 10 years of building powerful AI: 46%

    Founder and Executive Director, Alignment Research Center à l’époque

    Also gives a 20% probability that most humans die within 10 years of building powerful AI.

    My views on "doom" →

  4. 19 juin 2022 · Publication · LessWrong

    Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much more likely failure mode than humanity killing ourselves with destructive physical technologies.

    Founder and Executive Director, Alignment Research Center à l’époque

    First of his listed points of agreement with Yudkowsky.

    Where I agree and disagree with Eliezer →