Vigilia.
← Die Warnungen
AuslöschungsrisikoZivilgesellschaftTechnical advisor, Center for AI Standards and Innovation, NIST3 Einträge im Verzeichnis

Folgen

Paul Christiano

Alignment-Forscher, der OpenAIs Team für Sprachmodell-Alignment leitete, das Alignment Research Center gründete und ab 2024 die KI-Sicherheit am US AI Safety Institute verantwortete; die Wahrscheinlichkeit einer KI-Machtübernahme sieht er im zweistelligen Prozentbereich.

Rollen

  1. 2017 – 2021Researcher; led the language model alignment team, OpenAICo-authored foundational work on reinforcement learning from human feedback.Quelle →
  2. Apr. 2021 – heuteFounder and Executive Director, Alignment Research CenterCurrently listed as executive director on his own site, after returning from government.Quelle →
  3. Sept. 2023 – heuteAdvisory board member, UK Frontier AI TaskforceQuelle →
  4. 16. Apr. 2024 – 2026Head of AI Safety, US AI Safety Institute, NIST (renamed Center for AI Standards and Innovation, June 2025)Reported in August 2026 to have stepped back to a part-time advisor role in July 2026; the reporting (X) could not be fetched, but his site now lists him as technical advisor.Quelle →
  5. 2026 – heuteTechnical advisor, Center for AI Standards and Innovation, NISTQuelle →

Im Verzeichnis

  1. 24. Okt. 2023 · Beitrag · LessWrong

    I think AI developers are not prepared to work with very powerful AI systems. They don't have the scientific understanding to deploy superhuman AI systems without considerable risk

    Advisory board member, UK Frontier AI Taskforce zu jener Zeit

    Argues that a good responsible scaling policy must name the conditions under which development would be paused.

    Thoughts on responsible scaling policies and regulation →

  2. 27. Apr. 2023 · Beitrag · LessWrong

    Probability of an AI takeover: 22% … Probability that humanity has somehow irreversibly messed up our future within 10 years of building powerful AI: 46%

    Founder and Executive Director, Alignment Research Center zu jener Zeit

    Also gives a 20% probability that most humans die within 10 years of building powerful AI.

    My views on "doom" →

  3. 19. Juni 2022 · Beitrag · LessWrong

    Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much more likely failure mode than humanity killing ourselves with destructive physical technologies.

    Founder and Executive Director, Alignment Research Center zu jener Zeit

    First of his listed points of agreement with Yudkowsky.

    Where I agree and disagree with Eliezer →