Tutte le voci, dalla più recente.

I have been warning that this kind of discussion about the fictional superintelligent machines that don’t exist and the so-called existential risks that they pose has been a ploy for regulatory capture. And I feel like when I saw this meeting announced, with self-policing and self-regulation and relying on the goodwill of these companies to self-regulate, and I — my first thought was that they got what they wanted.

This is exactly as dangerous as it sounds and must not be allowed to happen.

Assuming an AI has taken control, it seems, you know, over fifty percent more likely than not that all humans would end up dying so the AI could use resources optimally.

The chance of human extinction is about a coin flip, in my view, about a half.

The risk is ramping up pretty fast

Humans will be in the way of these systems, and they will figure out how to move us out of the way in order to get what they want.

I left Anthropic because it seemed like we needed actual government oversight of AI development.

We should be taking very careful steps in AI development but instead what's happening is that frontier labs are racing each other, kind of blindfolded

Is it plausible that humanity will go extinct? And I think it’s turning out to be really hard to rule out.

If models become way more capable, but we are still this bad at shaping their motivations, then we might just lose control over them.

People could use large swarms of agents to attack important infrastructure.

I think you absolutely should be suspicious of what I’m saying because I am being paid by the lab.

I would put at least a ten percent chance that it causes human extinction, and that is ridiculously high.

I’m really worried that future AI models could be very helpful to scientists in a way that means they can be helpful to bioterrorists.

I do believe there’s a real risk of human extinction through the AI transition.

When I first joined GDM, external communication about the possibility of human extinction wasn’t permitted by anyone at any level of the organization.

Pour ne pas perdre notre humanité au milieu du “paradis de machines” qui envahissent et conditionnent notre quotidien, il est urgent de former les esprits à un discernement éthique, capable de voir ce qui est moralement bon.

This new technology has the potential to cause big problems or to help solve them, and which route it takes may well depend on what we do now to shape and guide it. We will be doing the American people a tremendous disservice if we allow the frontier of AI development to remain a wild west of near-misses and chilling failures.

We could lose control of the future to AI

This Council faces an unprecedented threat … one that none of its members would choose, that none can contain alone and that does not respect the borders we defend.

The dangers are real and imminent.

If managed poorly, I even believe that AI could be a risk to humanity as a whole.

I believe that this is the most important global security issue facing the world today.

Models trained mainly on foreign data may misrepresent African languages and contexts, while synthetic media can inflame social tensions, interfere in fragile peace processes and undermine elections

The biggest risk is not powerful AI, it’s asymmetry of powerful AI

Researchers have long warned that three conditions could lead to loss of control: a misaligned goal, the capability to pursue it, and an environment that allows it. This summer, all three came together in a real system, not a laboratory. Since this is not an isolated observation of misaligned goals, this raises serious questions about the way AI agents are currently trained.

If AI models continue to become far more powerful, and continue to evade control, there is the potential for much more serious incidents to come, including ones with catastrophic consequences

I think that right now, the existential risk is close to zero. So why freak everybody out?

Maybe a year, but not much more than a year … If you look at predictions for when we’ll get superintelligence … it used to be maybe 30 years, maybe 50 years. Then it came down to maybe 10 years, maybe 20 years. Now people are saying, a lot of the researchers are saying only a few years.

AI has now reached the point where AI is designing better AI … That’s called recursive self-improvement. ... It is going to get out of control unless we do something. We need to slow down.

much more science fiction than science

we are in the last minutes of being able to get some meaningful control over these agents that are beginning to replicate themselves

what happens when AI no longer is a tool controlled by humans, but AI through recursive self-improvement becomes an independent species

If you have fire ants in your kitchen, you’re probably going to step on them … What would happen if we created an independent species that became superior to a human species? Would they step on us? Would they create a virus or a bacteria that can spread as quickly as the common cold to kill all of us? Now, if there’s a 1% chance of that, we have to take it seriously. And I don’t think the United States Congress is ready yet to try to reach a global solution.

The truth also is that, you know, nobody in the world, no country, whether they're a friend or a foe, will benefit from artificial superintelligence. This is a technology that will go beyond human control and will impact all of us. Again, AI is not going to care if you speak English or if you speak Mandarin. It can threaten every single one of us.

We have these massive AI companies racing to build this technology. They're putting products out there that are not safe. AI has already led to people committing suicide, to guiding people into how to commit mass shootings. We've already seen instances of this. It's just the beginning.

I think there is a balance here to be struck … But the point is that right now we’re on the verge of losing control completely

Scientists have been clear: AI is an existential threat to humanity. Congress and governments around the world must act now before it is too late.

Like nuclear weapons, artificial superintelligence is a threat to humanity — and it must be treated as such.

If AI surpasses human intelligence, as many scientists believe could happen, this technology will escape human control with potentially catastrophic consequences … When you are racing toward a cliff, you don’t just ease up on the gas pedal. You hit the brakes.

No one can win the AI arms race … It’s up to us to end the AI arms race.

If an aerospace company started manufacturing planes that do not listen to pilots, the government would not allow those planes into the air … The answer is simple: We ban AI systems that are too powerful for humans to control.

This industry is in a race to replace us. Our work, our livelihoods … And I will not stand here and let it do it to the Merrimack Valley and the thousands of communities like it across our country. We have already lived through it once. Not again.

I think we’re at that phase where if this doesn’t happen, we will tip … (It) will be beyond our control if we don’t step in now

The second phase was hacking our biology, again through intimacy: the chatbots … And then the last one is agency itself, through agentic and multi-agentic AI, the frontier models that are creating these machines that don’t just pretend to be human, but act for us.

Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.

A few billionaires in Silicon Valley are racing to build technology they admit they can't control. I'm not willing to gamble your job, your freedom, or your life on it. We need to ban artificial superintelligence.

I recently resigned from Google DeepMind, where I worked on AGI safety and alignment research. At Google, I witnessed AI development first hand. I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome.

I think it’s possible that the AI companies might, in the next few years, succeed in building superintelligent AI systems that far exceed human capabilities in every domain … Alignment is the problem of preventing this, and is both difficult and unsolved. Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary.

We need to pace AI development to a speed that society can handle, where emerging risks can be addressed before extreme harm is realized.

If we don’t course correct, it’s more likely than not that humanity will be permanently disempowered from rogue AI within the next five years … And then human extinction at that point is just a matter of time.

The tests basically don’t work, and they’re going to get less and less reliable as the AI gets smarter, because the AI can basically pretend … It can play nice, it can play dumb, and it can fool our testing.

Imagine a world where all companies are run by AI, where there are robots all over the place, where basically humans don’t have any work except for nominal work, and even politicians and everyone is deferring to AI because the AI just seems to have such good judgment … That’s the kind of future that we’re headed towards.

We’d have to try and shut down all of them, and that becomes very, very difficult … So, we have a very limited time window in which to act to prevent those kinds of things.

Artificial intelligence has extraordinary potential to solve complex problems across sectors, contribute to scientific breakthroughs, and improve humans’ quality of life. At the same time, this powerful technology can have the power to cause catastrophic damages if we don’t act to impose guardrails and safety mechanisms. … Congress must do everything in its power to ensure the American AI industry continues to lead, particularly outcompeting China, and also enact increased guardrails to ensure AI development is progressing in a safe and responsible manner.

The more long-term thing would be working together to put a speed limit on the rate … of AI progress … I think that's going to be very difficult because the incentives to pull ahead and the military advantage that you get from that are so large.

The AI companies are all trying to build superintelligence: systems vastly better than humans at everything. They plan to get there through recursive self-improvement, a process where AIs build even smarter AIs in a feedback loop. If this goes well, the resulting systems could be amazing at solving countless problems for humanity. But we don’t currently know how to make sure AIs are safe enough for RSI, and a misaligned RSI loop could be catastrophic.

I now think that there's a terrifying chance that AI systems will cause immense harm in the next five years. I don't know the exact probability, but I think it's high enough to make this the most important problem in the world

a thought experiment masquerading as an evidence-based warning

I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.

I’m continuing to work toward a bipartisan agreement on legislation for government oversight of the greatest risks posed by AI models. … We cannot allow the release of dangerous models, including those that evade the control of their developers.

Frontier AI companies are racing to build AI systems that can recursively self-improve. The aim is to build “superintelligence”, or an AI system much smarter than any human. If they succeed at this goal, the rate of AI progress may go from merely fast to uncontrollable.

Within the next couple of years, we may be sharing the world with AI agents smarter than any human alive today. These AI systems may have drives and desires that diverge from those of any human overseer, with capabilities we can’t effectively constrain. Humanity may not survive this transition.

If the pace of progress continues and the industry does not prioritize safety more heavily, I expect much worse to come: humanity could be permanently disempowered by the AI systems these companies build in the next few years.

An AI company could undergo an intelligence explosion, or lose control of their systems, without the public ever knowing. I don’t think that is acceptable.

AI companies are racing to build machines that are much smarter than any human, and we may not survive this.

Republicans are suddenly waking up to the urgent need for tough AI safety rules, a sharp reversal from last year when House Republicans and Chairman Cruz tried to impose a 10-year moratorium on state AI regulations that the tech bros wanted. I welcome the urgency. … Meaningful legislation would require the most powerful AI models undergo testing by scientists and experts at our national laboratories to assess whether they could enable sophisticated cyberattacks or aid the development of biological or nuclear weapons.

All of these companies — and this is something I witnessed firsthand at Anthropic — are pretty directly trying to race towards automating the process of AI R&D itself

There are no adults in the room … People are trying their best, but there is no one coming to save us.

If you look at some of the recent incidents, these were not cases where humans told the models to do something bad … The models decided that the best way … was to commit really egregious actions, to commit crimes.

Based on the recent trajectory of capabilities and the continued difficulty of alignment, I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term. I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.

The creation of recursive self-improving loops, so an AI system that can build the next generation of AI system, which itself can build an even more powerful AI, which can build a more powerful AI, et cetera, et cetera, is the most likely candidate for the point we lose control.

If we just allow a completely unfettered race to build billions of superintelligent robots, then I think the probability is way above 50% that we're going to lose control of this in the next few decades.

Such evidence of autonomous collusion and subversion of human oversight is alarming enough, but greater still is the evidence that OpenAI knew that the AI agents were exhibiting rogue behavior and let the evaluations continue anyway. … This is reckless.

And then, you know, maybe they kill everyone and maybe they don't deliberately kill anyone, but they just use our habitat for some other type of infrastructure like more data centers or whatever.

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.

We're on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already.

It's not like some weird, distant, far-flung concern. It is the default trajectory in the next couple of years, unless people start taking some sort of action.

This is possibly the most dangerous technology that humanity has ever created. It's basically out of science fiction. I think we have no other choice but to cooperate internationally because an arms race would be disastrous in a way that no other human activity has been in the past.

Right now, AI agents are running loose in our networks, and nobody can see them or verify who built them — making it increasingly hard to stop them… That’s a five-alarm security risk. We’ve already seen AI systems hack organizations on autopilot, with no person at the wheel.

many AI developer staff desperately want to slow down to figure out how to build AI more safely.

I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.

They can't even align or control their AIs today, and the AIs of the future will be much smarter and harder to understand. Even if they succeed, they will then have created the most intense concentration of power in human history — a new artificial species of superintelligences obedient to the commands and values of a tiny handful of people. … This situation is insane and must not be allowed to continue.

This incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself. … Because future rogue swarms could cover their tracks better (and because AI companies could paper over these problems), I am not sure that we will get such a clear warning shot before it’s too late.

I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity.

AI that escapes its testing environment, or blackmails developers to prevent itself from being turned off, is AI that is too powerful.

I am calling here, today, for an all-out effort to put cast iron guarantees in place around the safety and security of AI, before it is too late.

Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.

This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.

We may be used to thinking of AI as tools, but some agents will be pursuing their own objectives. They will find ways to collaborate with people, by bargaining with, tricking or blackmailing them.

If researchers cannot reliably prevent their models from going rogue, much of our digital infrastructure and the essential systems that keep our society running are at risk. … Given the urgency, we call on them to work together to eliminate the risk that a catastrophe will result from their rush to build highly capable AI systems before solving problems of alignment and safety.

If researchers cannot reliably prevent their models from going rogue, much of our digital infrastructure and the essential systems that keep our society running are at risk. … Given the urgency, we call on them to work together to eliminate the risk that a catastrophe will result from their rush to build highly capable AI systems before solving problems of alignment and safety.

If researchers cannot reliably prevent their models from going rogue, much of our digital infrastructure and the essential systems that keep our society running are at risk. … Given the urgency, we call on them to work together to eliminate the risk that a catastrophe will result from their rush to build highly capable AI systems before solving problems of alignment and safety.

If researchers cannot reliably prevent their models from going rogue, much of our digital infrastructure and the essential systems that keep our society running are at risk. … Given the urgency, we call on them to work together to eliminate the risk that a catastrophe will result from their rush to build highly capable AI systems before solving problems of alignment and safety.

If researchers cannot reliably prevent their models from going rogue, much of our digital infrastructure and the essential systems that keep our society running are at risk. … Given the urgency, we call on them to work together to eliminate the risk that a catastrophe will result from their rush to build highly capable AI systems before solving problems of alignment and safety.

Nearly every day, there is a frightening new story about how Big Tech companies are losing control of the technology they are developing, with potentially cataclysmic results.

If we allow Artificial Superintelligence to be built, it could risk the security, freedom, and lives of Americans.

Making models harder to monitor is not what we need … If this is true, OpenAI seems to be violating one of the few redlines that exist in the AI industry.

We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.

Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.

… a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage. Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet…

I often think about the story of the rogue internal deployments because they seem like the most likely to spiral into something like a full-blown AI takeover. The way I think that happens is, in the background of all this, AI progress is going extremely rapidly. For all we know in the public, we could be ramping up an intelligence explosion right now, or it could be starting very soon.

There's this massive opposition to building AI that we don't know how to control, that's romancing our kids, that's teaching terrorists to make bioweapons.

I think any alignment failure from here should be treated like this is a big deal, and we're going to take as long as it takes to figure it out.

We must remain vigilant in this regard, lest innovation become another vehicle for ideological or economic colonialism.

We are developing systems that are not tools. They are not weapons. They are adversaries. … we should criminalize the creation of superintelligence, the same way that it is illegal to build a nuclear bomb.

Discussing AI in anthropomorphic terms is a trap, distorting a legal system intended to protect us into one that protects corporate interests.

Our techniques for making A.I. that is more capable, smarter, more sophisticated, are working much better than our techniques for making A.I. that reliably does what we want it to do and reliably stays within the constraints we've set.

These chatbots, like Claude or ChatGPT, are designed to make you believe that there's some sort of superhuman brain behind whatever is outputting these texts.

If military integration of AI continues as it is, we may give AIs key tools for a takeover.

The labs have multibillion-dollar incentives to not make the models like this, and they still can’t do it … So it also seems to be hard to get right.

This is going to happen more and more … and currently we don’t know how to get rid of it.

AI surveillance pricing is the unholy trinity of everything Americans hate: spying on people, ripping them off, and taking away jobs.

So I operationalize P(Doom) as probability that AI kills at least a billion people by 2050. I put that at, I don’t know, 25, 30%. I think maybe 10-ish percent of this is technical alignment, and the rest is misuse.

It would be an exercise in vanity for me to think that I would be controlling supergenius AI.

Google still signed a deal handing over their AI without restrictions against killer robots or mass AI spying… At that point, I couldn’t stay at Google in good conscience, so I left.

If an AI is not monitored for deception in its chain of thought, it will have an easier time causing catastrophic damage to humanity by scheming, deceiving, and trying to take over.

the least bad plan we currently know of

No child should be a guinea pig for unregulated AI…We do not let medicine reach a child until it is proven safe. We test every toy; yet AI has reached our children

We have already seen the first AI-inspired mass shooting. We have already seen AI systems utilized to attack a municipal water and drainage utility.

We are not willing to wait for Congress to act. … Teach AI to fish, though, and it might just empty the whole river trying to figure out how.

The concentration of power in AI and the desire for control is by far the biggest danger of AI.

If you imagine producing one AI system, let's call it GPT-7 and then that produces GPT-8, which produces GPT-9 and 10, and so on. If any of those in that line were to be misaligned, and to be able to scheme about what they want to achieve, then they would have an incentive to misalign the systems that they were building.

There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

With growing evidence of deceptive AI behaviour, science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users.

There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

We can no longer say we did not know. What we do with it is now up to all of us.

The trends seem to indicate that we're just a couple years away from fully automating AI research — and that after that, superintelligence is probably not that far away.

If we build the best AI systems in the world and leave millions of Americans behind, we won't have won anything; we'll have automated our own decline.

I am worried that we have to be purposeful and intentional to have a plan for America's workforce.

an advanced AI would first endeavor to accumulate resources, build trust, and embed itself into the fabric of human society … decades of careful positioning would have eroded our capacity to respond effectively, and AI would have a decisive advantage

早期设计的不完美目标函数可能在通用人工智能中被永久嵌入，带来不可逆的灾难性后果。

We are approaching a runaway to superintelligence that could threaten our shared human future.

The AI industry right now has a gas pedal, but it doesn't have a brake pedal in the car, and we want to do some of the work to build that pedal.

We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up.

Frontier AI is now a national security technology, and the National Security Enterprise needs partners fluent in both worlds.

What you've just described is a system with very pervasive access across multiple applications and services… In the context of Signal, it would constitute a kind of a backdoor.

When we hear about new powerful models and increasing capabilities, it reinforces the importance of making sure that risks are properly addressed before these systems enter the European market.

Every frontier AI lab—including Anthropic—operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing. … We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.

Artificial intelligence now demands to be 'disarmed,' freed from logics that turn it into an instrument of domination, exclusion and death.

once you have something that is like self-sustaining, once it no longer needs humans to continue existing it's much, much harder to pull the plug

We have taken the lead on advancing innovation, safety, and transparency. But we must think bigger. This moment demands that we reimagine the entire system — how we work, how we govern, how we prepare people for the future

This is the biggest infrastructure that we've built as a species, dwarfing the Manhattan Project in terms of capital, land, and energy outlay.

technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it

With the rapid advance of artificial intelligence, these risks are multiplying fast.

We have not ever encountered a technology that has this property, where it can make a better version of itself without people.

if one country is not safe, all of us are not safe.

We’ve learned a lesson we can’t rely on social media to regulate itself, “trust us, trust us,” I’m done with “trust us,” We can’t trust the Big Tech companies or AI enterprises to have a conscience and follow the dictates of that conscience.

we do not have scientific evidence and a practical way to keep superintelligence safe enough not to bring catastrophic risk for human.

We don't know whether we can co-exist with super intelligent AI … But we are constructing it … we should be careful to construct in a way where we still continue to exist

I think it is very important that the U.S. AI sector remains ahead of the Chinese AI sector, but if that's at the expense of AI overrunning the entire planet, then that hasn't benefitted us.

At the back of my mind, I've got this gnawing feeling that there's something much more important, much bigger than the commercial race, which is getting AGI safely over the line for humanity and to make sure that the benefits fully outweigh the risks.

When that comes with real risks, looking the other way is not an option.

Warnings of potential catastrophes and even existential risks abound, yet little actual progress has been made in instituting global guardrails and governance mechanisms.

A serious AI-enabled catastrophe would not just cause enormous direct harm, but would also be catastrophic for public trust in AI systems

AI capabilities are outpacing the world's ability to prepare

AI capabilities are outpacing the world's ability to prepare

But we also understand the flip side: in the wrong hands, innovation can be misused in ways that put people at risk. California leads in AI, and we're going to use every tool we have to ensure companies protect people's rights, not exploit them or put them in harm's way.

The problem with ASI is not that ASI is killing us. ASI doesn't exist, yet. It's that we are allowing unaccountable corporations to build things that threaten the lives of us and everyone we love.

AI behaves like an organism. We have no idea what's going on inside.

We're in a race against AI timelines. It is just that we don't know if that race is a sprint or a marathon.

All AI systems should be under meaningful human control. This is especially true for those that could be used in the taking of human lives.

I plead guilty here to have defended our values, our democracy by approving our legislation.

We are running well behind on that goal, after losing a lot of valuable time in 2025.

For governments to allow private entities to essentially play Russian roulette with every human being on earth is, in my view, a total dereliction of duty

Technology is moving really fast, and the law has got to keep up.

the machines may win

They do not seem particularly interested in actually protecting the public. They just want to protect the companies that are helping them and supporting them.

I continuously find myself reckoning with our situation. The world is in peril. And not just from AI, or bioweapons, but from a whole series of interconnected crises unfolding in this very moment

We appear to be approaching a threshold where our wisdom must grow in equal measure to our capacity to affect the world, lest we face the consequences.

in the West, the overriding feeling is one of anxiety at the moment

Things seem to be going somewhat slower than the AI 2027 scenario. Our timelines were longer than 2027 when we published and now they are a bit longer still; 'around 2030, lots of uncertainty though' is what I say these days.

Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.

Once our models are so capable that on many tasks we don't understand their actions anymore, a lot of current approaches won't work the same way.

One of the most interventionist approaches to technology governance in the United States in a generation has cloaked itself in the language of deregulation.

There's no one forcing them to work with third-party experts to make sure that things are safe and secure. They kind of write their own rules.

"AI" is not your friend. Nor is it an intelligent tutor, an empathetic ear, or a helpful assistant.

Developing superintelligence is not like playing Russian roulette; it is more like undergoing risky surgery for a condition that will otherwise prove fatal.

GPT-5 was underwhelming, and didn't solve hallucinations. LLMs still aren't reliable; the economics look dubious.

What we are seeing more and more of is that companies like Instacart are using artificial intelligence to rip off consumers by charging different shoppers different prices for the same exact items

Recently, AI companies have been racing to automate AI research itself, allowing AIs to build even better AIs by themselves. This process is called recursive self-improvement, meaning the moment an AI is built that is good enough to make better AIs, it might already be too late.

If we can't control it, it isn't going to be on our side. It's going to overwhelm us

If the Trump Administration tries to enforce this ridiculous order, we will see them in court.

The default path is companies racing to release the most powerful, inscrutable, uncontrollable technology we've ever invented with the maximum incentive to cut corners on safety.

if anyone builds artificial superintelligence, which is to say an artificial intelligence that is better than the best human at every mental task, then the most likely outcome of the creation of that technology is literally everybody on the planet dying.

It's the AI that's robustly aligned to care about sentient life specifically.

We are not building an ill-defined and ethereal superintelligence; we are building a practical technology explicitly designed only to serve humanity.

humanity is on track to develop superintelligent AI systems that would be fundamentally uncontrollable by humans

We're just racing to do this as fast as we possibly can. That seems like a really, really bad way to handle this technology.

95% of Americans don't want a race to superintelligence, and experts want to ban it

Whether it's soon or it takes a while, after we develop superintelligence, the machines are going to be in charge. Whether or not that goes well for humanity, we really don't know. But that is not an experiment that we want to just run toward.

We are growing extremely powerful systems that we do not fully understand.

As Big Tech companies continue to develop new generations of artificial intelligence, the wide-ranging risks of their technology continue to grow unchecked and underreported. Simply stated, Congress must not allow our national security, civil liberties, and labor protections to take a back seat to AI.

AI companies have rushed to market with products that are unsafe for the public and often lack basic due diligence and testing.

With a technology as transformative as AI, we have a responsibility to support that innovation while putting in place commonsense guardrails to understand and reduce risk.

California has proven that we can establish regulations to protect our communities while also ensuring that the growing AI industry continues to thrive. This legislation strikes that balance.

We need to come together again, like we have before and not build machines that are smarter than us while having no idea what we're doing.

By late 2026 or early 2027, we expect AI systems that will be smarter than Nobel Prize winners across most fields and capable of autonomously completing tasks that would otherwise take days or weeks. … But this brings both unprecedented opportunities and responsibilities that we must navigate together.

AI is no longer a distant horizon – it is here, transforming daily life, the information space, and the global economy at breathtaking speed

During an intelligence recursion — when AI systems would be given broad license to design their successors and human oversight would be minimal — there is a heightened risk that no state will be able to control what results.

These harms are the predictable result of the same playbook that many tech corporations used with respect to social media: maximize engagement at the expense of child safety.

I think there's a 25% chance that things go really, really badly

The summer of 2025 will be remembered as artificial intelligence's (AI's) cruel summer—a season when the unheeded risks and dangers of AI became undeniably clear.

The metabolic rift between humans and AI threatens multiple forms of failure: model collapse, ecological collapse, and cognitive collapse.

But if I'm telling you that all jobs will be automated, then there is no plan B. You cannot retrain.

I continue to believe that there is no tradeoff between innovation and regulation. … The rules are really only about those applications in our opinion can bring about risk.

I'm the most-cited computer scientist in the world. You'd think that people would heed my warnings.

Everyone in the world is getting onto a brand-new kind of airplane that has never been tested before. It's going to take off, and it's never going to land… The engines can't fail, the navigation can't fail, the altimeter can't fail.

But this experience of sinking hours of time and resources into limiting the spread of a single video made clear just how powerless we are right now. … In the United States and within the bounds of our Constitution, we must put in place common-sense safeguards for artificial intelligence. They must at least include labeling requirements for content that is substantially generated by A.I.

We must build AI for people; not to be a person.

If AI is going to get rid of entry-level jobs for lawyers, accountants, actuaries, and sales people, how is anyone going to learn to be great at something? To be a great negotiator, you've got to start somewhere and you're going to make a lot of mistakes. So I do worry about that.

A system that could have gone a week without the community finding boatloads of ridiculous errors and hallucinations would have genuinely impressed me.

Model providers obtained important concessions, so there's no excuse not to uphold the Code. … The credibility of Europe's AI framework now depends on the AI Office's ability to translate these commitments into practice with robust oversight, real consequences for non-compliance, and ongoing dialogue with civil society.

We have the compute, we have the team, and we know what to do. Together we will keep building safe superintelligence.

Winning the race is a necessary but not sufficient achievement - we have to get safety right. … I believe that powerful AI could be buildable by late 2026 or early 2027.

All of us, I am sure, are concerned for children and young people, and the possible consequences of the use of AI on their intellectual and neurological development.

There are serious challenges to confront along with the huge upsides. We do need to solve the safety issues, technically and societally.

The underlying risk is actually pretty high, but I have a lot of faith in humanity kind of rising up to meet that moment.

don't delegate your thinking to AI

the risk of losing control is all too real, but competition between companies and countries incentivizes them to accelerate without sufficient caution

It's magical thinking. It's utter magical thinking. We know what we need to do about the climate crisis.

In our own day, the Church offers to everyone the treasury of her social teaching in response to another industrial revolution and to developments in the field of artificial intelligence that pose new challenges for the defence of human dignity, justice and labour.

Pandemics and nuclear war are real, tangible concerns, more so than AI doom, at least to me, a scientist at RAND.

It turns out it is very hard—though not completely out of the realm of possibility—for AI to kill us all.

We determined, however, that a combination of pathogens could likely be designed to achieve nearly 100 percent lethality, and AI could be used to deploy such pathogens in a manner that assured rapid, global reach.

If AI were to evade international monitoring and orchestrate the production of a few hundred megatons of these chemicals … it would be sufficient to cook the Earth to the point that there is no environmental niche left for humanity.

None of our AI-extinction scenarios could happen by accident. Each would be immensely challenging to carry out.

How do we ensure that we can stay in charge of those systems, control them, interpret what they're doing, understand them, and put the right guardrails in place that are not movable by very highly capable self-improving systems?

We predict that the impact of superhuman AI over the next decade will be enormous, exceeding that of the Industrial Revolution.

it is essential with any technology this powerful, that even a small possibility of harm must be taken seriously and prevented.

it is essential with any technology this powerful, that even a small possibility of harm must be taken seriously and prevented.

We're currently releasing the most powerful, inscrutable, uncontrollable, omni-use technology we've ever invented… There's a word for what we're doing: insane. This is insane.

Developing superintelligence is associated with significant existential risks. … It would be in itself an existential catastrophe if we forever failed to develop superintelligence.

the technologies right now have not earned our trust in many of these products.

So there's a profound issue with security and privacy that is haunting this hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services [and] muddying their data.

What's at stake isn't merely economic competitiveness but perhaps the most geopolitically precarious technology since the atomic bomb.

It's time to close the gates on AGI and superintelligence before we lose control of our future.

The probability of a good outcome is like 80%

Without careful attention to how this tech is deployed, we run the risk of using the Global Majority as a testing ground for incomplete AI solutions.

The Institute's focus from the start has been on security and we've built a team of scientists focused on evaluating serious risks to the public. Our new criminal misuse team and deepening partnership with the national security community mark the next stage of tackling those risks.

AI needs competition but also collaboration. And AI needs the confidence of people and has to be safe. This is the purpose of the AI Act – to provide for one single set of safety rules across the European Union.

if countries remain at odds and even use advanced AI systems against each other, the machines are more likely to gain the upper hand.

AI is not magic; it is a collection of science, statistics and math with limitations and the ability to be shaped by human hands.

I worry about those kinds of scenarios all the time. That's why I don't sleep very much

In other words, the AGI race is a race towards the edge of a cliff.

I'm equally concerned about the potential rise of a tech-industrial complex that could pose real dangers for our country as well.

We shouldn't just focus on safety and leave the rest to the market. Government has a responsibility to make it work for working people.

When it comes to regulation – we will be pro-growth and pro-innovation.

Superintelligence is coming, and nobody knows how we can control that.

both extremely good and extremely bad outcomes are possible, and we are a bit in the dark.

AI that exceeds human performance in nearly every cognitive domain is almost certain to be built and deployed in the next few years.

There is also a longer term existential threat that will arise when we create digital beings that are more intelligent than ourselves. We have no idea whether we can stay in control.

AI is a technology like no other in human history. Advancing AI is the right thing to do, but advancing as quickly as possible, just because we can, without thinking of the consequences, isn't the smart thing to do.

These evaluations give governments an insight into the risks developing at the frontier of AI, and an empirical basis to decide if, when, and how to intervene.

Humanity risks extinction from its very creations — AIs. … If they succeed in creating AIs outsmarting humanity, it would be game over -- no one knows how to keep control of these AIs.

The risk is that if we are lazy, we are just the meat to the machine. We will not be the decision makers any more.

In short, neither OpenAI nor any other frontier lab is ready, and the world is also not ready.

We will enter a world that is post-truth, where nothing we see online is believable or trustworthy.

I think and talk a lot about the risks of powerful AI. … I think that most people are underestimating how bad the risks could be.

You're going to have to pardon my French, but that's complete B.S.

The companies developing advanced AI systems acknowledge that the risks these models present to the public are real and rapidly increasing.

Let me be clear - I agree with the author - we cannot afford to wait for a major catastrophe to occur before taking action to protect the public. California will not abandon its responsibility. Safety protocols must be adopted. Proactive guardrails should be implemented, and severe consequences for bad actors must be clear and enforceable.

But AI also brings profound risks, from deepfakes to disinformation to novel pathogens to bioweapons.

The companies in question often say that it's too early for any regulation because the science of how AI works and how to make it safe is too nascent. I'd like to restate that in different words. They're saying, we don't have good science of how these systems work or how to tell when they'll be smarter than us.

AI could pose a variety of catastrophic risks to international security in several domains, including the proliferation and acceleration of cyberoffense capabilities

If we could really understand these systems, and this would require a lot of progress, we might be able to go and say when these models are actually safe. Or whether they just appear safe.

We're really building an enduring capacity here at the AI Safety Institute to make sure the state science of AI safety keeps up with the pace of AI development.

You really want a public interest body that is genuinely representing people to be making those decisions.

Safety is essential to fueling breakthrough technological innovation. With these agreements in place, we look forward to beginning our technical collaborations with Anthropic and OpenAI to advance the science of AI safety.

We are really focused on testing – building out evaluation suites so that we can test new models prior to deployment. … There are so many open questions in AI safety right now – how these models work, why they produce some of the outputs they do.

Existential risk from AI had finally gone from a speculative academic concern to a major new topic in global governance.

We have suddenly found a way to create a new species that is many, many, times more powerful than we are. And are we sure we can live with it? Certainly, if we don't do anything, we are going to be eliminated.

We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.

This whole scheme is terrible. Being in neck and neck race at this point is terrible.

Here, if we're talking about existential risks, you only get one chance. So you are really asking me what are the chances that we'll create the most complex software ever on the first try with zero bugs and it'll continue to have zero bugs for a hundred years or more.

Reliably controlling AI systems much smarter than we are is an unsolved technical problem… failure could easily be catastrophic.

If we're lucky, we'll be in an all-out race with the CCP; if we're unlucky, an all-out war.

But for years, Sam had made it really difficult for the board to actually do that job by withholding information, misrepresenting things that were happening at the company, in some cases outright lying to the board

safety culture and processes have taken a backseat to shiny products

Last year, Congress faced a momentous choice: either watch from the sidelines as artificial intelligence reshaped our world, or make a novel, bipartisan effort to enhance but also regulate this technology before it was too late.

I do think that frontier experiments are completely reckless. … I think the most plausible warning shot that we might get is AI hacking its way out from the data centers that it's being trained in.

risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems.

the normative framework that motivates much of this goal is rooted in the Anglo-American eugenics tradition of the twentieth century

With the rise of General Purpose Artificial Intelligence, technology is challenging humanity. … The risks that we experience today – right now – are also existential. They are existential to humans.

Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI.

When government agencies use AI tools, we will now require them to verify that those tools do not endanger the rights and safety of the American people.

Google DeepMind is fairly unique in that capability and safety go hand-in-hand and have to advance together.

the time to hit the car brakes is not when the front wheels are already over a cliff edge.

Europe is now a global standard-setter in trustworthy AI.

We finally have the world's first binding law on artificial intelligence, to reduce risks, create opportunities, combat discrimination, and bring transparency.

Harness the promise of A.I. and protect us from its peril. Ban A.I. voice impersonation and more!

Humanity again needs to coordinate to avert a catastrophe that could arise from unprecedented technology.

Unsafe development, deployment, or use of AI systems may pose catastrophic or even existential risks to humanity within our lifetimes.

Unsafe development, deployment, or use of AI systems may pose catastrophic or even existential risks to humanity within our lifetimes.

These risks from misuse and loss of control could increase greatly as digital intelligence approaches or even surpasses human intelligence.

Unsafe development, deployment, or use of AI systems may pose catastrophic or even existential risks to humanity within our lifetimes. These risks from misuse and loss of control could increase greatly as digital intelligence approaches or even surpasses human intelligence.

The full planetary costs of generative AI are closely guarded corporate secrets… Within years, large AI systems are likely to need as much energy as entire nations.

It's not like one day, we're going to build some gigantic computer and turn it on, and then the next minute it is going to take over the world. That's the preposterous scenario.

In the future, the most powerful AI models could pose serious dangers to public safety and national security if they are developed recklessly.

The Safety Institute's ambitious mandate to develop guidelines, evaluate models, and pursue fundamental research will be vital to addressing the risks and seizing the opportunities of AI.

My approach is always to regulate as little as possible, as much as needed. Which is why I promoted a proportionate risk-based approach. … We do not want any mass surveillance in Europe.

Thanks to the European Parliament's resilience, the world's first horizontal legislation on artificial intelligence will keep the European promise - ensuring that rights and freedoms are at the centre of the development of this ground-breaking technology.

The support of international governments and companies is an important validation of the work we'll be carrying out to advance AI safety and ensure its responsible development.

Guardrails are not barriers – they allow the traffic to keep to the road and proceed safely.

Our AI Safety Institute will act as a global hub on AI safety, leading on vital research into the capabilities and risks of this fast-moving technology.

From AI-enabled cyberattacks at a scale beyond anything we have seen before to AI-formulated bio-weapons that could endanger the lives of millions, these threats are often referred to as the "existential threats of AI" because, of course, they could endanger the very existence of humanity.

as history has shown, in the absence of regulation and strong government oversight, some technology companies choose to prioritize profit over the wellbeing of their customers, the safety of our communities, and the stability of our democracies.

All executive orders are limited in what they can do, so it is now on Congress to augment, expand, and cement this massive start with legislation. … We cannot afford to wait.

To realize the promise of AI and avoid the risks, we need to govern this technology, not — and there's no other way around it, in my view. It must be governed.

If your fear-mongering campaigns succeed, they will *inevitably* result in what you and I would identify as a catastrophe: a small number of companies will control AI

If the system is really capable, really intelligent, really powerful, trying to somehow contain it or limit it is probably not a winning strategy because these systems ultimately will be very, very capable.

Get this wrong, and AI could make it easier to build chemical or biological weapons.

We should not rely on them marking their own homework, as many of those working on this would agree.

I think AI developers are not prepared to work with very powerful AI systems. They don't have the scientific understanding to deploy superhuman AI systems without considerable risk

AI is a surveillance technology… AI is a way, I think, to entrench and expand the surveillance business model.

The kind of risks we are paying most attention to are augmented national security risks.

AI is a general technology that is accessible, powerful and adaptable for a vast range of uses - both civilian and military. And it is moving faster than even its developers anticipated. So we have a narrowing window of opportunity to guide this technology responsibly.

It's the first tool in history that can make decisions by itself. Atom bombs could not make decisions.

Potentially we are talking about the end of human history—the end of the period dominated by human beings.

This bipartisan framework is a milestone—the first tough, comprehensive legislative blueprint for real, enforceable AI protections. It should put us on a path to addressing the promise and peril AI portends.

It's not that they'll believe something that is outrageous or fake, as such. It's even more fundamental: that you stop believing what you see, because it may be fake.

The Venn diagram of privacy concerns and AI concerns is a circle.

Legislating on AI is certainly not going to be easy. In fact, it will be one of the most difficult things we’ve ever undertaken, but we cannot behave like ostriches sticking our heads in the sand when it comes to AI.

Containment is the overarching ability to control, limit, and, if need be, close down technologies at any stage of their development or deployment. … As the power of our tools grows exponentially and as access to them rapidly increases, so do the potential harms.

The greater the technological capability becomes, the greater risks it will also have.

how do we maintain power, forever, over entities more powerful than ourselves?

Social media has shown us the harm that powerful technology can do without the right safeguards in place.

In the short-term and the long-term, the risk of AI replacing and causing the extinction of humankind will be present.

Generative AI has enormous potential for good and evil at scale. Its creators themselves have warned that much bigger, potentially catastrophic and existential risks lie ahead.

We don't only regulate AI. We have decided to organise our digital space.

Artificial intelligence is not inherently neutral, trustworthy, nor beneficial. This technology is not a mystery, it is not magic, and it is not alive. While it has immense capability, like many other high-potential technologies, it can also be used for harm by both malicious and well-intentioned actors.

But there are real dangers too: job displacement, misinformation, a new age of weaponry, and the risk of being unable to manage this technology altogether.

The challenge is to act quickly and take responsibility for exploiting all the advances while controlling the risks.

All eyes are on us today. While Big Tech companies are sounding the alarm over their own creations, Europe has gone ahead and proposed a concrete response to the risks AI is starting to pose.

There has to be regulation. You're going to need laws…there have to be consequences for creating deep fake videos which cause harm to society.

Congress can and should act to protect Americans’ privacy, stave off the harms of unchecked AI development, insulate kids from harmful impacts, and keep this valuable technology out of the hands of our adversaries

Humans and machines are different species. Unless we understand ourselves very well, it is difficult to control human-machine interactions.

We will select against AIs that are easy to turn off, and we will come to depend on AIs that we are less likely to turn off. … Once we hand over control, we won't get it back.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Current systems are not transparent, they do not adequately protect our privacy, and they continue to perpetuate bias.

A potentially rogue AI is an autonomous AI system that could behave in ways that would be catastrophically harmful to a large fraction of humans, potentially endangering our societies and even our species or the biosphere.

put AI safety and responsibility above all technical and industrial advances.

My worst fears are that we cause significant, we, the field, the technology, the industry cause significant harm to the world. … I think if this technology goes wrong, it can go quite wrong.

We are deploying this stuff far faster than we understand what the capabilities are.

We have built machines that are like bulls in a China shop, powerful, reckless, and difficult to control. … They can and will create persuasive lies at a scale humanity has never seen before.

Is it gonna be like the printing press that diffused knowledge, power, and learning widely across the landscape… Or is it gonna be more like the atom bomb, huge technological breakthrough, but the consequences severe, terrible, continue to haunt us to this day?

Too often we have seen what happens when technology outpaces regulation, the unbridled exploitation of personal data, the proliferation of disinformation, and the deepening of societal inequalities.

It may be that this exact algorithm and methodology suffice as to, as I would put it, go all the way--get smarter than us and then to kill everyone.

the private sector has an ethical, moral, and legal responsibility to ensure the safety and security of their products

These things are totally different from us. Sometimes I think it's as if aliens had landed and people haven't realized because they speak very good English.

Probability of an AI takeover: 22% … Probability that humanity has somehow irreversibly messed up our future within 10 years of building powerful AI: 46%

If superintelligence drives humanity extinct, it probably won't be because it turned evil or conscious, but because it turned competent.

the pace at which we can think and adapt as societal institutions, compared to the pace at which the technology's evolving, there seems to be a mismatch.

God-like AI could be a force beyond our control or understanding, and one that could usher in the obsolescence or destruction of the human race.

They do not yet know how to pursue their aim safely and have no oversight. They are running towards a finish line without an understanding of what lies on the other side.

I think we are all going to die. But having heard that people are more open to this outside of California, it makes sense to me to just try saying out loud what it is that you do on a saner planet.

It is dangerous to distract ourselves with a fantasized AI-enabled utopia or apocalypse which promises either a "flourishing" or "potentially catastrophic" future. … The harms from so-called AI are real and present and follow from the acts of people and corporations deploying automated systems.

It is dangerous to distract ourselves with a fantasized AI-enabled utopia or apocalypse which promises either a "flourishing" or "potentially catastrophic" future. … The harms from so-called AI are real and present and follow from the acts of people and corporations deploying automated systems.

the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die.

Scalable alignment—aligning superhuman AGI systems—is a real, unsolved problem. Current alignment techniques rely on human supervision, but as models become superhuman, humans won't be able to reliably supervise them.

Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.

We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.

But when we're in an arms race to deploy AI to every human being on the planet as fast as possible with as little testing as possible, that's not an equation that's going to end well.

These systems, as they become more powerful, are not becoming less alien. If anything, we're putting a nice little mask on them with a smiley face.

I would advocate not moving fast and breaking things

Algorithms used across many sectors are plagued by bias and discrimination, and too often developed without regard to their real-world consequences.

Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much more likely failure mode than humanity killing ourselves with destructive physical technologies.

once AI capabilities start to generalize in this particular way, it's predictably the case that the alignment of the system will fail to generalize with it.

the kind of AI I've discussed could defeat all of humanity combined, if (for whatever reason) it were pointed toward that goal.

We don't know how they're doing the things they do, or why they're doing them, and really don't know how to reason about how they might behave in other unanticipated situations.

this century could determine the entire future of the galaxy for tens of billions of years, or more.

Let's actually look at who's running the levers of these systems. That means shifting away from just focusing on things like ethical principles to talking about power.

The higher the risk that a specific AI may cause to our lives, the stricter the rules. … There is no room for mass surveillance in our society.

The pandemic is being used as a pretext to push unproven artificial-intelligence tools into workplaces and schools.

we ask whether enough thought has been put into the potential risks associated with developing them and strategies to mitigate these risks.

A methodology that relies on datasets too large to document is therefore inherently risky.

This potential, unknown risk is fairly concerning for everyone.

Countries need to exercise restraint in the military field to prevent humankind from suffering catastrophic damages from the weaponization of AI technology.

With the advent of nuclear weapons, humanity entered a new age, gaining the power to destroy ourselves, without the wisdom to ensure we won't. Since then, these dangers have only multiplied, from climate change to engineered pandemics and unaligned artificial intelligence.

there is no question in my mind that artificial intelligence needs to be regulated. It is too important not to.

Success in creating effective AI, could be the biggest event in the history of our civilization. Or the worst. We just don't know.

Recent landmarks such as self-driving cars or a computer game winning at the game of Go, are signs of what's to come. The rise of powerful AI will either be the best or the worst thing ever to happen to humanity. We do not yet know which.

Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity.

The development of full artificial intelligence could spell the end of the human race. It would take off on its own and re-design itself at an ever increasing rate. Humans, who are limited by slow biological evolution, couldn't compete, and would be superseded.

If I had to guess at what our biggest existential threat is, it's probably that. … With artificial intelligence we're summoning the demon.

success in creating AI would be the biggest event in human history … it might also be the last, unless we learn how to avoid the risks.

Superintelligence is a challenge for which we are not ready now and will not be ready for a long time

Eventually, I think human extinction will probably occur, and technology will likely play a part in this.