Who Gets to End the World?

The line was fake, but entirely believable.

Sam Altman: “If I Don’t End the World, Someone Far More Dangerous Will” was an Onion joke, not a confession. It landed because it distilled the bargain Silicon Valley keeps asking us to accept: Artificial intelligence may transform work, war, politics, even the terms of human survival. Therefore, the people racing to build it must be trusted to restrain it. The danger becomes the credential. Acceleration becomes duty. The match becomes proof that the arsonist understands fire.

There should be a name for this moral trapdoor, the one that opens whenever some clever bastard realizes that a machine capable of ending the world can also be made into an argument for why he alone should build it. Call it apocalyptic possessiveness.

The formula is always identical. The thing is coming anyway. The bad people are already building it. Possibly they have built it already, in some bunker, under some mountain. So we have to build it too. Not because we want power, God forbid. We are only being responsible. We are the adult in the room. We are the clean pair of hands hovering over the dirty button. Delay means defeat. Caution is cowardice. Regulation is for people who do not understand the terrible velocity of history. History, meanwhile, has apparently burst into the meeting wearing a little sash that says CHOSEN ONE and pointed directly at us.

This is why the argument is hard to kill. It has a grave little voice and a flag pin and a sentence about adversarial environments. Sometimes, in the narrowest and most disgusting sense, it is even right. Someone else might build the thing. Someone worse might get there first. The world is full of worse people, which is very convenient for the better people who need an excuse to become them.

So the alibi proceeds. First inevitability: This will happen. Then virtue: Only we can make it happen safely. Then permission: Therefore we may do whatever we were already planning to do.

The canonical American version was the Manhattan Project. The United States did not wake up one morning in the desert and decide, as an abstract exercise, to split the world open. It acted under the fear that Nazi Germany might do so first. The National WWII Museum describes the project as rooted in fear of what Hitler might do with such a weapon; refugee scientists brought both knowledge of German atomic research and fear of what might be unleashed, and American officials raced to beat Germany to the bomb. The fear was not invented by publicists in a conference room; it belonged to a war in which the worst men in Europe had already demonstrated a remarkable appetite for the possible.

But this is the problem with emergency logic: The emergency can fade before the machinery built in its name does.

By late 1942, German leaders had largely concluded that an atomic bomb was beyond their reach. The scientists and officials of the Manhattan Project did not know this with certainty until near the war’s end, so the project continued under the pressure of a German threat that had already begun to recede. The bomb had been justified as a race against Nazi Germany; but when that enemy no longer explained it, the bomb did not disappear. It found new targets and a permanent life of its own. What began as fear became policy.

The successor was worse. After the Soviet atomic test, Edward Teller pressed for the hydrogen bomb. PBS recounts Teller’s warning that if the Russians demonstrated the “Super” before the United States possessed one, America’s situation would be hopeless. The United States tested a thermonuclear device in 1952; eighteen months later, the Soviet team tested its own. The defensive move had produced not safety but a higher cliff. This is often the way with the argument; its first use is an emergency, its second use is precedent, and its third use is doctrine.

Doctrine followed immediately. In 1946, Bernard Brodie argued that the military’s central purpose had changed: It was no longer to win wars, but to prevent them. The new arsenal would deter conflict not by promising victory, but by making catastrophe credible.

This logic was not irrational. It may well have helped prevent a direct great-power war. But it rested on a brutal bargain: Civilization would be preserved by keeping alive the power to destroy it. The gun on the table would keep everyone polite. The world would continue because ending it had become, at last, technically possible.

There had been earlier versions of the dream. Alfred Nobel, whose dynamite was not originally developed for war but was soon used in it, entertained the idea that sufficiently destructive weapons might make war impossible. In 1891 he suggested that his factories might end war sooner than peace congresses, since armies capable of annihilating one another in a second would cause civilized nations to recoil and disband their troops. NobelPrize.org notes, with the advantage of the twentieth century behind it, that he did not live to see how wrong this conception was. Nikola Tesla later imagined a “death beam” that could make war impossible by giving every country an invisible wall. PBS describes his lifelong desire to find a technological end to war, and the 1934 claim that such a beam would make nations impregnable.

There is an odd innocence in this fantasy, and a worse vanity. The inventor looks at violence and imagines he can refine it into peace. The bomb will abolish bombing. The beam will abolish battle. The machine will become humane by becoming total. It is the kind of beautiful idea that should make us nervous: elegant, fatal, and very flattering to the man who thinks it.

The darker cousin of the same impulse does not claim to save the future but to deny it to others. In March 1945, with the Third Reich collapsing, Hitler issued what became known as the Nero Decree, ordering the destruction of German infrastructure. The National WWII Museum describes the order as a decision, in the face of defeat, to destroy railroads, bridges, communications, factories, mines, public utilities, and anything that might aid the Allies. This was not deterrence. It was apocalyptic possessiveness. If history would not bear his name, it would not proceed intact.

The analogy has obvious limits. Sam Altman is not Edward Teller, Teller was not Hitler, and Anthropic is not the Manhattan Project. To pretend otherwise would be lazy and false. But the limits of an analogy do not make it useless. What matters is the recurring pattern: A danger is identified, the person identifying it presents himself as uniquely qualified to manage it, and the warning becomes a warrant for power.

OpenAI’s own charter understood the danger of this conversion. It warned that late-stage AGI development could become a competitive race without time for adequate safety precautions, and it said that if another value-aligned, safety-conscious project came close to building AGI first, OpenAI would stop competing and start assisting. This was, at least as written, an attempt to resist the race logic; it recognized that the sentence “someone else might get there first” is precisely the sentence that can destroy safety.

The later rhetoric is more interesting because it is more grand. In a 2025 reflection, Altman wrote that OpenAI had been founded because AGI seemed possible and might be the most impactful technology in human history; he wrote that the founders wanted to make their mark on history. He also wrote that OpenAI was now confident it knew how to build AGI “as we have traditionally understood it,” that it was turning toward superintelligence, and that OpenAI could not be a normal company. None of this is the same as wanting catastrophe. But it is the language of world-historical self-authorization. The company is no longer merely making a product. It is standing, by its own account, at the threshold where ordinary categories fail.

This week, the custody battle over that threshold moved into court. WIRED reported that Elon Musk testified he had helped start OpenAI because government was not stepping up and because Google’s unchecked progress worried him; he warned of a possible “Terminator outcome.” OpenAI’s lawyer, meanwhile, argued that Musk sued only after founding a competitor and that Musk had once proposed that he or Tesla control OpenAI. The spectacle had a grim clarity. The apocalypse risk had become a dispute over stewardship. Each side claimed the dangerous future would be safer in its own hands. The public was invited to choose not whether such power should be concentrated, but which concentration had the better biography.

The present-day cases are less dramatic, and for that reason more revealing. Reuters reported this week that families of victims in the Tumbler Ridge, British Columbia, mass shooting sued OpenAI and Altman, alleging that the company had identified the shooter as a credible threat months before the attack and failed to warn police. OpenAI said the shooting was a tragedy, emphasized its zero-tolerance policy for violent misuse, and said it had strengthened safeguards; Reuters also noted that OpenAI has denied claims in other lawsuits involving alleged chatbot harms. The point is not to decide the case from a distance. The point is that ordinary accountability has arrived at the door of the company that speaks in civilizational terms. The same institution that asks to be trusted with speculative extinction risk must answer questions about warning systems, escalation criteria, repeat violators, law enforcement referrals, and the human consequences of not acting.

There is also the narrower technical version. Reuters reported last week that Microsoft planned to integrate Anthropic’s Claude Mythos Preview into its secure-development process, after Mythos had reportedly found thousands of major vulnerabilities and shown advanced coding and exploitation capabilities. The model was being offered through Anthropic’s Project Glasswing to selected companies for defensive cybersecurity work. In Reuters’ earlier report, Anthropic described Glasswing as a controlled initiative giving select organizations access to an unreleased model for defensive security, with partners including major technology companies and additional organizations responsible for critical software infrastructure.

This may be a reasonable cyber-defense argument. Defenders do need to find vulnerabilities before attackers do. But the rhetorical shape is the same: The capability is dangerous, and it will spread, so trusted insiders must get it first. The danger is not a reason to stop. It is the reason for privileged deployment. The company’s possession of the weapon becomes evidence that the company must be central to containing it.

What makes this thinking strange is not merely ego. The stranger element is ego plus eschatology: The desire to be necessary at the final threshold. Most legacy projects require survivors. A cathedral, a constitution, a scientific discovery, a company, a conquest, even a scandal—all depend on someone left to remember. Extinction cancels the archive. There is no statue in the ash. No authorized biography. No congressional hearing in which the founder, older and thinner and wearing the same expression, explains what everyone failed to understand.

That is why the fantasy gives itself away. The imagined audience is not humanity after the event (there is no humanity after the event). The audience is internal, a private theater of history in which the actor sees himself at the center of the last chapter and forgets that a true last chapter has no readers.

The usual critique says that maybe the nice man building the doomsday machine is not actually very nice. Maybe if you look closely enough you’ll find some tiny moral stain on him: a bad tweet, a weird investor, some insufficiently lachrymose statement about the children pulped into the future’s intake valve. This is meant to be the serious objection. But it’s a children’s objection. The problem is not that the good guy secretly isn’t good. The problem is that “the good guy in charge of the apocalypse” is already a category error.

An extinction-level technology does not become less extinction-level because the person feeding it power cables has kind eyes. It does not matter that he worries about alignment or speaks in the soft quavering tones of someone who has read all the right papers and would really prefer not to annihilate the species unless the incentives make it unavoidable. The end of the world is not made acceptable because the man holding the detonator is preferable to some other, nastier man holding some other, nastier detonator.

This is how they get you: by turning annihilation into a character question. Would you rather the apocalypse be run by a vulgarian, a fascist, a scammer, a coke-eyed goblin with a podcast? Or by this careful, philanthropic, safety-conscious adult in the room?

The phrase “someone worse will do it” should not automatically authorize the speaker to do it first. Sometimes it should produce the opposite conclusion. If the danger is real, then the answer is not self-appointment but constraint: institutions, treaties, inspections, liability regimes, compute controls, emergency brakes, public oversight, and democratic mechanisms strong enough that no private actor can convert inevitability into permission. The validity of the fear does not validate the grandeur of the person invoking it.

There are, of course, moments when the fear is real. Nazi Germany was real. Cyber attackers are real. Nuclear deterrence may have prevented horrors we never had to see. A serious politics cannot pretend that restraint by one actor guarantees restraint by all others. But neither can a serious politics allow every ambitious builder to launder ambition through catastrophe. “If I don’t, someone worse will” may describe a dilemma. It does not settle who gets power, who bears risk, who profits, who governs, or who is sacrificed when the system fails.

The most dangerous men are not always the ones who cackle while they build the death engine. Some of them look genuinely pained. Some convene panels. Some publish solemn little manifestos about safety, shared benefit, and the sacred obligation to all mankind. Some say they are humbled by the scale of what they are doing, and the disturbing thing is that they may even mean it. They may really go home at night and feel the full weight of history pressing gently on their sternum, like a concerned nurse checking for signs of life.

But sincerity is not legitimacy. The moral question cannot be reduced to character. The issue is not whether the people building the final machine are decent, anxious, reflective or even unusually responsible. The issue is why anyone gets to build it under private authority at all.

So the question is not whether Altman, Musk, Anthropic, Google, Microsoft or some uniformed state ghoul would make the most responsible custodian of the final machine. That is just the old fairy tale of the good king, updated for people who say “deployment” instead of “conquest.” Maybe one of them really is better. Maybe one would press the button with trembling hands, after a long ethical review, while another would jab at it laughing through a mouthful of ketamine and blood. But the distinction vanishes in the fireball.

A world ended by a good man is not more alive than one ended by a bad one. The last child still looks up at the last light with no idea whether the person who arranged all this had a thoughtful governance structure. At the end, there is no moral remainder, no little plaque fixed to the ruins saying, “at least the apocalypse was administered by the right sort of people.” There is only the ruin. The button may be pressed in madness or in sorrow. The blast makes no distinction.