Every entry, newest first.

Making models harder to monitor is not what we need … If this is true, OpenAI seems to be violating one of the few redlines that exist in the AI industry.

There's this massive opposition to building AI that we don't know how to control, that's romancing our kids, that's teaching terrorists to make bioweapons.

I think any alignment failure from here should be treated like this is a big deal, and we're going to take as long as it takes to figure it out.

We must remain vigilant in this regard, lest innovation become another vehicle for ideological or economic colonialism.

We are developing systems that are not tools. They are not weapons. They are adversaries. … we should criminalize the creation of superintelligence, the same way that it is illegal to build a nuclear bomb.

Discussing AI in anthropomorphic terms is a trap, distorting a legal system intended to protect us into one that protects corporate interests.

Our techniques for making A.I. that is more capable, smarter, more sophisticated, are working much better than our techniques for making A.I. that reliably does what we want it to do and reliably stays within the constraints we've set.

These chatbots, like Claude or ChatGPT, are designed to make you believe that there's some sort of superhuman brain behind whatever is outputting these texts.

AI surveillance pricing is the unholy trinity of everything Americans hate: spying on people, ripping them off, and taking away jobs.

It would be an exercise in vanity for me to think that I would be controlling supergenius AI.

the least bad plan we currently know of

No child should be a guinea pig for unregulated AI…We do not let medicine reach a child until it is proven safe. We test every toy; yet AI has reached our children

The concentration of power in AI and the desire for control is by far the biggest danger of AI.

If you imagine producing one AI system, let's call it GPT-7 and then that produces GPT-8, which produces GPT-9 and 10, and so on. If any of those in that line were to be misaligned, and to be able to scheme about what they want to achieve, then they would have an incentive to misalign the systems that they were building.

There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

With growing evidence of deceptive AI behaviour, science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users.

There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

We can no longer say we did not know. What we do with it is now up to all of us.

The trends seem to indicate that we're just a couple years away from fully automating AI research — and that after that, superintelligence is probably not that far away.

If we build the best AI systems in the world and leave millions of Americans behind, we won't have won anything; we'll have automated our own decline.

I am worried that we have to be purposeful and intentional to have a plan for America's workforce.

an advanced AI would first endeavor to accumulate resources, build trust, and embed itself into the fabric of human society … decades of careful positioning would have eroded our capacity to respond effectively, and AI would have a decisive advantage

早期设计的不完美目标函数可能在通用人工智能中被永久嵌入，带来不可逆的灾难性后果。

We are approaching a runaway to superintelligence that could threaten our shared human future.

The AI industry right now has a gas pedal, but it doesn't have a brake pedal in the car, and we want to do some of the work to build that pedal.

We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up.

Frontier AI is now a national security technology, and the National Security Enterprise needs partners fluent in both worlds.

What you've just described is a system with very pervasive access across multiple applications and services… In the context of Signal, it would constitute a kind of a backdoor.

When we hear about new powerful models and increasing capabilities, it reinforces the importance of making sure that risks are properly addressed before these systems enter the European market.

Every frontier AI lab—including Anthropic—operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing. … We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.

Artificial intelligence now demands to be 'disarmed,' freed from logics that turn it into an instrument of domination, exclusion and death.

once you have something that is like self-sustaining, once it no longer needs humans to continue existing it's much, much harder to pull the plug

We have taken the lead on advancing innovation, safety, and transparency. But we must think bigger. This moment demands that we reimagine the entire system — how we work, how we govern, how we prepare people for the future

This is the biggest infrastructure that we've built as a species, dwarfing the Manhattan Project in terms of capital, land, and energy outlay.

technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it

With the rapid advance of artificial intelligence, these risks are multiplying fast.

We have not ever encountered a technology that has this property, where it can make a better version of itself without people.

if one country is not safe, all of us are not safe.

We’ve learned a lesson we can’t rely on social media to regulate itself, “trust us, trust us,” I’m done with “trust us,” We can’t trust the Big Tech companies or AI enterprises to have a conscience and follow the dictates of that conscience.

we do not have scientific evidence and a practical way to keep superintelligence safe enough not to bring catastrophic risk for human.

We don't know whether we can co-exist with super intelligent AI … But we are constructing it … we should be careful to construct in a way where we still continue to exist

I think it is very important that the U.S. AI sector remains ahead of the Chinese AI sector, but if that's at the expense of AI overrunning the entire planet, then that hasn't benefitted us.

At the back of my mind, I've got this gnawing feeling that there's something much more important, much bigger than the commercial race, which is getting AGI safely over the line for humanity and to make sure that the benefits fully outweigh the risks.

When that comes with real risks, looking the other way is not an option.

Warnings of potential catastrophes and even existential risks abound, yet little actual progress has been made in instituting global guardrails and governance mechanisms.

A serious AI-enabled catastrophe would not just cause enormous direct harm, but would also be catastrophic for public trust in AI systems

AI capabilities are outpacing the world's ability to prepare

AI capabilities are outpacing the world's ability to prepare

But we also understand the flip side: in the wrong hands, innovation can be misused in ways that put people at risk. California leads in AI, and we're going to use every tool we have to ensure companies protect people's rights, not exploit them or put them in harm's way.

The problem with ASI is not that ASI is killing us. ASI doesn't exist, yet. It's that we are allowing unaccountable corporations to build things that threaten the lives of us and everyone we love.

AI behaves like an organism. We have no idea what's going on inside.

We're in a race against AI timelines. It is just that we don't know if that race is a sprint or a marathon.

All AI systems should be under meaningful human control. This is especially true for those that could be used in the taking of human lives.

I plead guilty here to have defended our values, our democracy by approving our legislation.

We are running well behind on that goal, after losing a lot of valuable time in 2025.

For governments to allow private entities to essentially play Russian roulette with every human being on earth is, in my view, a total dereliction of duty

Technology is moving really fast, and the law has got to keep up.

the machines may win

They do not seem particularly interested in actually protecting the public. They just want to protect the companies that are helping them and supporting them.

in the West, the overriding feeling is one of anxiety at the moment

Things seem to be going somewhat slower than the AI 2027 scenario. Our timelines were longer than 2027 when we published and now they are a bit longer still; 'around 2030, lots of uncertainty though' is what I say these days.

Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.

Once our models are so capable that on many tasks we don't understand their actions anymore, a lot of current approaches won't work the same way.

One of the most interventionist approaches to technology governance in the United States in a generation has cloaked itself in the language of deregulation.

There's no one forcing them to work with third-party experts to make sure that things are safe and secure. They kind of write their own rules.

"AI" is not your friend. Nor is it an intelligent tutor, an empathetic ear, or a helpful assistant.

Developing superintelligence is not like playing Russian roulette; it is more like undergoing risky surgery for a condition that will otherwise prove fatal.

GPT-5 was underwhelming, and didn't solve hallucinations. LLMs still aren't reliable; the economics look dubious.

What we are seeing more and more of is that companies like Instacart are using artificial intelligence to rip off consumers by charging different shoppers different prices for the same exact items

Recently, AI companies have been racing to automate AI research itself, allowing AIs to build even better AIs by themselves. This process is called recursive self-improvement, meaning the moment an AI is built that is good enough to make better AIs, it might already be too late.

If we can't control it, it isn't going to be on our side. It's going to overwhelm us

If the Trump Administration tries to enforce this ridiculous order, we will see them in court.

The default path is companies racing to release the most powerful, inscrutable, uncontrollable technology we've ever invented with the maximum incentive to cut corners on safety.

if anyone builds artificial superintelligence, which is to say an artificial intelligence that is better than the best human at every mental task, then the most likely outcome of the creation of that technology is literally everybody on the planet dying.

It's the AI that's robustly aligned to care about sentient life specifically.

We are not building an ill-defined and ethereal superintelligence; we are building a practical technology explicitly designed only to serve humanity.

humanity is on track to develop superintelligent AI systems that would be fundamentally uncontrollable by humans

We're just racing to do this as fast as we possibly can. That seems like a really, really bad way to handle this technology.

95% of Americans don't want a race to superintelligence, and experts want to ban it

Whether it's soon or it takes a while, after we develop superintelligence, the machines are going to be in charge. Whether or not that goes well for humanity, we really don't know. But that is not an experiment that we want to just run toward.

We are growing extremely powerful systems that we do not fully understand.

As Big Tech companies continue to develop new generations of artificial intelligence, the wide-ranging risks of their technology continue to grow unchecked and underreported. Simply stated, Congress must not allow our national security, civil liberties, and labor protections to take a back seat to AI.

AI companies have rushed to market with products that are unsafe for the public and often lack basic due diligence and testing.

With a technology as transformative as AI, we have a responsibility to support that innovation while putting in place commonsense guardrails to understand and reduce risk.

California has proven that we can establish regulations to protect our communities while also ensuring that the growing AI industry continues to thrive. This legislation strikes that balance.

We need to come together again, like we have before and not build machines that are smarter than us while having no idea what we're doing.

By late 2026 or early 2027, we expect AI systems that will be smarter than Nobel Prize winners across most fields and capable of autonomously completing tasks that would otherwise take days or weeks. … But this brings both unprecedented opportunities and responsibilities that we must navigate together.

AI is no longer a distant horizon – it is here, transforming daily life, the information space, and the global economy at breathtaking speed

During an intelligence recursion — when AI systems would be given broad license to design their successors and human oversight would be minimal — there is a heightened risk that no state will be able to control what results.

These harms are the predictable result of the same playbook that many tech corporations used with respect to social media: maximize engagement at the expense of child safety.

I think there's a 25% chance that things go really, really badly

The summer of 2025 will be remembered as artificial intelligence's (AI's) cruel summer—a season when the unheeded risks and dangers of AI became undeniably clear.

The metabolic rift between humans and AI threatens multiple forms of failure: model collapse, ecological collapse, and cognitive collapse.

But if I'm telling you that all jobs will be automated, then there is no plan B. You cannot retrain.

I continue to believe that there is no tradeoff between innovation and regulation. … The rules are really only about those applications in our opinion can bring about risk.

I'm the most-cited computer scientist in the world. You'd think that people would heed my warnings.

Everyone in the world is getting onto a brand-new kind of airplane that has never been tested before. It's going to take off, and it's never going to land… The engines can't fail, the navigation can't fail, the altimeter can't fail.

We must build AI for people; not to be a person.

If AI is going to get rid of entry-level jobs for lawyers, accountants, actuaries, and sales people, how is anyone going to learn to be great at something? To be a great negotiator, you've got to start somewhere and you're going to make a lot of mistakes. So I do worry about that.

A system that could have gone a week without the community finding boatloads of ridiculous errors and hallucinations would have genuinely impressed me.

Model providers obtained important concessions, so there's no excuse not to uphold the Code. … The credibility of Europe's AI framework now depends on the AI Office's ability to translate these commitments into practice with robust oversight, real consequences for non-compliance, and ongoing dialogue with civil society.

We have the compute, we have the team, and we know what to do. Together we will keep building safe superintelligence.

Winning the race is a necessary but not sufficient achievement - we have to get safety right. … I believe that powerful AI could be buildable by late 2026 or early 2027.

All of us, I am sure, are concerned for children and young people, and the possible consequences of the use of AI on their intellectual and neurological development.

There are serious challenges to confront along with the huge upsides. We do need to solve the safety issues, technically and societally.

The underlying risk is actually pretty high, but I have a lot of faith in humanity kind of rising up to meet that moment.

don't delegate your thinking to AI

the risk of losing control is all too real, but competition between companies and countries incentivizes them to accelerate without sufficient caution

It's magical thinking. It's utter magical thinking. We know what we need to do about the climate crisis.

In our own day, the Church offers to everyone the treasury of her social teaching in response to another industrial revolution and to developments in the field of artificial intelligence that pose new challenges for the defence of human dignity, justice and labour.

How do we ensure that we can stay in charge of those systems, control them, interpret what they're doing, understand them, and put the right guardrails in place that are not movable by very highly capable self-improving systems?

We predict that the impact of superhuman AI over the next decade will be enormous, exceeding that of the Industrial Revolution.

it is essential with any technology this powerful, that even a small possibility of harm must be taken seriously and prevented.

it is essential with any technology this powerful, that even a small possibility of harm must be taken seriously and prevented.

We're currently releasing the most powerful, inscrutable, uncontrollable, omni-use technology we've ever invented… There's a word for what we're doing: insane. This is insane.

Developing superintelligence is associated with significant existential risks. … It would be in itself an existential catastrophe if we forever failed to develop superintelligence.

the technologies right now have not earned our trust in many of these products.

So there's a profound issue with security and privacy that is haunting this hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services [and] muddying their data.

What's at stake isn't merely economic competitiveness but perhaps the most geopolitically precarious technology since the atomic bomb.

It's time to close the gates on AGI and superintelligence before we lose control of our future.

The probability of a good outcome is like 80%

Without careful attention to how this tech is deployed, we run the risk of using the Global Majority as a testing ground for incomplete AI solutions.

The Institute's focus from the start has been on security and we've built a team of scientists focused on evaluating serious risks to the public. Our new criminal misuse team and deepening partnership with the national security community mark the next stage of tackling those risks.

AI needs competition but also collaboration. And AI needs the confidence of people and has to be safe. This is the purpose of the AI Act – to provide for one single set of safety rules across the European Union.

if countries remain at odds and even use advanced AI systems against each other, the machines are more likely to gain the upper hand.

AI is not magic; it is a collection of science, statistics and math with limitations and the ability to be shaped by human hands.

I worry about those kinds of scenarios all the time. That's why I don't sleep very much

In other words, the AGI race is a race towards the edge of a cliff.

I'm equally concerned about the potential rise of a tech-industrial complex that could pose real dangers for our country as well.

We shouldn't just focus on safety and leave the rest to the market. Government has a responsibility to make it work for working people.

When it comes to regulation – we will be pro-growth and pro-innovation.

Superintelligence is coming, and nobody knows how we can control that.

both extremely good and extremely bad outcomes are possible, and we are a bit in the dark.

AI that exceeds human performance in nearly every cognitive domain is almost certain to be built and deployed in the next few years.

There is also a longer term existential threat that will arise when we create digital beings that are more intelligent than ourselves. We have no idea whether we can stay in control.

AI is a technology like no other in human history. Advancing AI is the right thing to do, but advancing as quickly as possible, just because we can, without thinking of the consequences, isn't the smart thing to do.

These evaluations give governments an insight into the risks developing at the frontier of AI, and an empirical basis to decide if, when, and how to intervene.

Humanity risks extinction from its very creations — AIs. … If they succeed in creating AIs outsmarting humanity, it would be game over -- no one knows how to keep control of these AIs.

The risk is that if we are lazy, we are just the meat to the machine. We will not be the decision makers any more.

In short, neither OpenAI nor any other frontier lab is ready, and the world is also not ready.

We will enter a world that is post-truth, where nothing we see online is believable or trustworthy.

I think and talk a lot about the risks of powerful AI. … I think that most people are underestimating how bad the risks could be.

You're going to have to pardon my French, but that's complete B.S.

The companies developing advanced AI systems acknowledge that the risks these models present to the public are real and rapidly increasing.

Let me be clear - I agree with the author - we cannot afford to wait for a major catastrophe to occur before taking action to protect the public. California will not abandon its responsibility. Safety protocols must be adopted. Proactive guardrails should be implemented, and severe consequences for bad actors must be clear and enforceable.

But AI also brings profound risks, from deepfakes to disinformation to novel pathogens to bioweapons.

The companies in question often say that it's too early for any regulation because the science of how AI works and how to make it safe is too nascent. I'd like to restate that in different words. They're saying, we don't have good science of how these systems work or how to tell when they'll be smarter than us.

AI could pose a variety of catastrophic risks to international security in several domains, including the proliferation and acceleration of cyberoffense capabilities

If we could really understand these systems, and this would require a lot of progress, we might be able to go and say when these models are actually safe. Or whether they just appear safe.

We're really building an enduring capacity here at the AI Safety Institute to make sure the state science of AI safety keeps up with the pace of AI development.

You really want a public interest body that is genuinely representing people to be making those decisions.

Safety is essential to fueling breakthrough technological innovation. With these agreements in place, we look forward to beginning our technical collaborations with Anthropic and OpenAI to advance the science of AI safety.

We are really focused on testing – building out evaluation suites so that we can test new models prior to deployment. … There are so many open questions in AI safety right now – how these models work, why they produce some of the outputs they do.

Existential risk from AI had finally gone from a speculative academic concern to a major new topic in global governance.

We have suddenly found a way to create a new species that is many, many, times more powerful than we are. And are we sure we can live with it? Certainly, if we don't do anything, we are going to be eliminated.

We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.

This whole scheme is terrible. Being in neck and neck race at this point is terrible.

Here, if we're talking about existential risks, you only get one chance. So you are really asking me what are the chances that we'll create the most complex software ever on the first try with zero bugs and it'll continue to have zero bugs for a hundred years or more.

Reliably controlling AI systems much smarter than we are is an unsolved technical problem… failure could easily be catastrophic.

If we're lucky, we'll be in an all-out race with the CCP; if we're unlucky, an all-out war.

But for years, Sam had made it really difficult for the board to actually do that job by withholding information, misrepresenting things that were happening at the company, in some cases outright lying to the board

safety culture and processes have taken a backseat to shiny products

Last year, Congress faced a momentous choice: either watch from the sidelines as artificial intelligence reshaped our world, or make a novel, bipartisan effort to enhance but also regulate this technology before it was too late.

I do think that frontier experiments are completely reckless. … I think the most plausible warning shot that we might get is AI hacking its way out from the data centers that it's being trained in.

risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems.

the normative framework that motivates much of this goal is rooted in the Anglo-American eugenics tradition of the twentieth century

With the rise of General Purpose Artificial Intelligence, technology is challenging humanity. … The risks that we experience today – right now – are also existential. They are existential to humans.

Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI.

When government agencies use AI tools, we will now require them to verify that those tools do not endanger the rights and safety of the American people.

Google DeepMind is fairly unique in that capability and safety go hand-in-hand and have to advance together.

the time to hit the car brakes is not when the front wheels are already over a cliff edge.

Europe is now a global standard-setter in trustworthy AI.

We finally have the world's first binding law on artificial intelligence, to reduce risks, create opportunities, combat discrimination, and bring transparency.

Harness the promise of A.I. and protect us from its peril. Ban A.I. voice impersonation and more!

Humanity again needs to coordinate to avert a catastrophe that could arise from unprecedented technology.

Unsafe development, deployment, or use of AI systems may pose catastrophic or even existential risks to humanity within our lifetimes.

Unsafe development, deployment, or use of AI systems may pose catastrophic or even existential risks to humanity within our lifetimes.

These risks from misuse and loss of control could increase greatly as digital intelligence approaches or even surpasses human intelligence.

Unsafe development, deployment, or use of AI systems may pose catastrophic or even existential risks to humanity within our lifetimes. These risks from misuse and loss of control could increase greatly as digital intelligence approaches or even surpasses human intelligence.

The full planetary costs of generative AI are closely guarded corporate secrets… Within years, large AI systems are likely to need as much energy as entire nations.

It's not like one day, we're going to build some gigantic computer and turn it on, and then the next minute it is going to take over the world. That's the preposterous scenario.

In the future, the most powerful AI models could pose serious dangers to public safety and national security if they are developed recklessly.

The Safety Institute's ambitious mandate to develop guidelines, evaluate models, and pursue fundamental research will be vital to addressing the risks and seizing the opportunities of AI.

My approach is always to regulate as little as possible, as much as needed. Which is why I promoted a proportionate risk-based approach. … We do not want any mass surveillance in Europe.

Thanks to the European Parliament's resilience, the world's first horizontal legislation on artificial intelligence will keep the European promise - ensuring that rights and freedoms are at the centre of the development of this ground-breaking technology.

The support of international governments and companies is an important validation of the work we'll be carrying out to advance AI safety and ensure its responsible development.

Guardrails are not barriers – they allow the traffic to keep to the road and proceed safely.

Our AI Safety Institute will act as a global hub on AI safety, leading on vital research into the capabilities and risks of this fast-moving technology.

From AI-enabled cyberattacks at a scale beyond anything we have seen before to AI-formulated bio-weapons that could endanger the lives of millions, these threats are often referred to as the "existential threats of AI" because, of course, they could endanger the very existence of humanity.

as history has shown, in the absence of regulation and strong government oversight, some technology companies choose to prioritize profit over the wellbeing of their customers, the safety of our communities, and the stability of our democracies.

All executive orders are limited in what they can do, so it is now on Congress to augment, expand, and cement this massive start with legislation. … We cannot afford to wait.

To realize the promise of AI and avoid the risks, we need to govern this technology, not — and there's no other way around it, in my view. It must be governed.

If your fear-mongering campaigns succeed, they will *inevitably* result in what you and I would identify as a catastrophe: a small number of companies will control AI

If the system is really capable, really intelligent, really powerful, trying to somehow contain it or limit it is probably not a winning strategy because these systems ultimately will be very, very capable.

Get this wrong, and AI could make it easier to build chemical or biological weapons.

We should not rely on them marking their own homework, as many of those working on this would agree.

I think AI developers are not prepared to work with very powerful AI systems. They don't have the scientific understanding to deploy superhuman AI systems without considerable risk

AI is a surveillance technology… AI is a way, I think, to entrench and expand the surveillance business model.

The kind of risks we are paying most attention to are augmented national security risks.

AI is a general technology that is accessible, powerful and adaptable for a vast range of uses - both civilian and military. And it is moving faster than even its developers anticipated. So we have a narrowing window of opportunity to guide this technology responsibly.

It's the first tool in history that can make decisions by itself. Atom bombs could not make decisions.

Potentially we are talking about the end of human history—the end of the period dominated by human beings.

This bipartisan framework is a milestone—the first tough, comprehensive legislative blueprint for real, enforceable AI protections. It should put us on a path to addressing the promise and peril AI portends.

It's not that they'll believe something that is outrageous or fake, as such. It's even more fundamental: that you stop believing what you see, because it may be fake.

The Venn diagram of privacy concerns and AI concerns is a circle.

Legislating on AI is certainly not going to be easy. In fact, it will be one of the most difficult things we’ve ever undertaken, but we cannot behave like ostriches sticking our heads in the sand when it comes to AI.

Containment is the overarching ability to control, limit, and, if need be, close down technologies at any stage of their development or deployment. … As the power of our tools grows exponentially and as access to them rapidly increases, so do the potential harms.

The greater the technological capability becomes, the greater risks it will also have.

how do we maintain power, forever, over entities more powerful than ourselves?

Social media has shown us the harm that powerful technology can do without the right safeguards in place.

In the short-term and the long-term, the risk of AI replacing and causing the extinction of humankind will be present.

Generative AI has enormous potential for good and evil at scale. Its creators themselves have warned that much bigger, potentially catastrophic and existential risks lie ahead.

We don't only regulate AI. We have decided to organise our digital space.

Artificial intelligence is not inherently neutral, trustworthy, nor beneficial. This technology is not a mystery, it is not magic, and it is not alive. While it has immense capability, like many other high-potential technologies, it can also be used for harm by both malicious and well-intentioned actors.

But there are real dangers too: job displacement, misinformation, a new age of weaponry, and the risk of being unable to manage this technology altogether.

The challenge is to act quickly and take responsibility for exploiting all the advances while controlling the risks.

All eyes are on us today. While Big Tech companies are sounding the alarm over their own creations, Europe has gone ahead and proposed a concrete response to the risks AI is starting to pose.

There has to be regulation. You're going to need laws…there have to be consequences for creating deep fake videos which cause harm to society.

Congress can and should act to protect Americans’ privacy, stave off the harms of unchecked AI development, insulate kids from harmful impacts, and keep this valuable technology out of the hands of our adversaries

Humans and machines are different species. Unless we understand ourselves very well, it is difficult to control human-machine interactions.

We will select against AIs that are easy to turn off, and we will come to depend on AIs that we are less likely to turn off. … Once we hand over control, we won't get it back.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Current systems are not transparent, they do not adequately protect our privacy, and they continue to perpetuate bias.

A potentially rogue AI is an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere.

put AI safety and responsibility above all technical and industrial advances.

My worst fears are that we cause significant, we, the field, the technology, the industry cause significant harm to the world. … I think if this technology goes wrong, it can go quite wrong.

We are deploying this stuff far faster than we understand what the capabilities are.

We have built machines that are like bulls in a China shop, powerful, reckless, and difficult to control. … They can and will create persuasive lies at a scale humanity has never seen before.

Is it gonna be like the printing press that diffused knowledge, power, and learning widely across the landscape… Or is it gonna be more like the atom bomb, huge technological breakthrough, but the consequences severe, terrible, continue to haunt us to this day?

Too often we have seen what happens when technology outpaces regulation, the unbridled exploitation of personal data, the proliferation of disinformation, and the deepening of societal inequalities.

It may be that this exact algorithm and methodology suffice as to, as I would put it, go all the way--get smarter than us and then to kill everyone.

the private sector has an ethical, moral, and legal responsibility to ensure the safety and security of their products

These things are totally different from us. Sometimes I think it's as if aliens had landed and people haven't realized because they speak very good English.

Probability of an AI takeover: 22% … Probability that humanity has somehow irreversibly messed up our future within 10 years of building powerful AI: 46%

If superintelligence drives humanity extinct, it probably won't be because it turned evil or conscious, but because it turned competent.

the pace at which we can think and adapt as societal institutions, compared to the pace at which the technology's evolving, there seems to be a mismatch.

God-like AI could be a force beyond our control or understanding, and one that could usher in the obsolescence or destruction of the human race.

They do not yet know how to pursue their aim safely and have no oversight. They are running towards a finish line without an understanding of what lies on the other side.

I think we are all going to die. But having heard that people are more open to this outside of California, it makes sense to me to just try saying out loud what it is that you do on a saner planet.

It is dangerous to distract ourselves with a fantasized AI-enabled utopia or apocalypse which promises either a "flourishing" or "potentially catastrophic" future. … The harms from so-called AI are real and present and follow from the acts of people and corporations deploying automated systems.

It is dangerous to distract ourselves with a fantasized AI-enabled utopia or apocalypse which promises either a "flourishing" or "potentially catastrophic" future. … The harms from so-called AI are real and present and follow from the acts of people and corporations deploying automated systems.

the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die.

Scalable alignment—aligning superhuman AGI systems—is a real, unsolved problem. Current alignment techniques rely on human supervision, but as models become superhuman, humans won't be able to reliably supervise them.

Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.

We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.

But when we're in an arms race to deploy AI to every human being on the planet as fast as possible with as little testing as possible, that's not an equation that's going to end well.

These systems, as they become more powerful, are not becoming less alien. If anything, we're putting a nice little mask on them with a smiley face.

I would advocate not moving fast and breaking things

Algorithms used across many sectors are plagued by bias and discrimination, and too often developed without regard to their real-world consequences.

Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much more likely failure mode than humanity killing ourselves with destructive physical technologies.

once AI capabilities start to generalize in this particular way, it's predictably the case that the alignment of the system will fail to generalize with it.

the kind of AI I've discussed could defeat all of humanity combined, if (for whatever reason) it were pointed toward that goal.

We don't know how they're doing the things they do, or why they're doing them, and really don't know how to reason about how they might behave in other unanticipated situations.

this century could determine the entire future of the galaxy for tens of billions of years, or more.

Let's actually look at who's running the levers of these systems. That means shifting away from just focusing on things like ethical principles to talking about power.

The higher the risk that a specific AI may cause to our lives, the stricter the rules. … There is no room for mass surveillance in our society.

The pandemic is being used as a pretext to push unproven artificial-intelligence tools into workplaces and schools.

we ask whether enough thought has been put into the potential risks associated with developing them and strategies to mitigate these risks.

A methodology that relies on datasets too large to document is therefore inherently risky.

This potential, unknown risk is fairly concerning for everyone.

Countries need to exercise restraint in the military field to prevent humankind from suffering catastrophic damages from the weaponization of AI technology.

With the advent of nuclear weapons, humanity entered a new age, gaining the power to destroy ourselves, without the wisdom to ensure we won't. Since then, these dangers have only multiplied, from climate change to engineered pandemics and unaligned artificial intelligence.

there is no question in my mind that artificial intelligence needs to be regulated. It is too important not to.

Success in creating effective AI, could be the biggest event in the history of our civilization. Or the worst. We just don't know.

Recent landmarks such as self-driving cars or a computer game winning at the game of Go, are signs of what's to come. The rise of powerful AI will either be the best or the worst thing ever to happen to humanity. We do not yet know which.

Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity.

The development of full artificial intelligence could spell the end of the human race. It would take off on its own and re-design itself at an ever increasing rate. Humans, who are limited by slow biological evolution, couldn't compete, and would be superseded.

If I had to guess at what our biggest existential threat is, it's probably that. … With artificial intelligence we're summoning the demon.

success in creating AI would be the biggest event in human history … it might also be the last, unless we learn how to avoid the risks.

Superintelligence is a challenge for which we are not ready now and will not be ready for a long time

Eventually, I think human extinction will probably occur, and technology will likely play a part in this.