The meeting ended with a voluntary agreement on frontier-model safety
and the symbolic replacement of AI with Super Intelligence within the
U.S. executive branch. But it was preceded by events that until
recently had seemed theoretical: AI agents bypassed restrictions,
accessed the internet, and gained unauthorized access to external
systems. What does this mean: the emergence of a dangerous machine
will — or a reflection of our own goals, training methods, and
contradictions?

How It All Began

On September 29, 2026, Donald Trump met at the White House with the
leaders of major technology companies. Participants included Dario
Amodei of Anthropic, Mark Zuckerberg of Meta, Sundar Pichai of Google,
OpenAI President Greg Brockman, Nvidia CEO Jensen Huang, and other
industry representatives.

Until recently, the main political question surrounding AI was who would
win the technological race, who would build more data centers, and who
would be first to create AGI. By the fall of 2026, another question had
emerged:

What happens if systems become capable enough to discover ways of
acting that their creators never anticipated?

On August 26, OpenAI disclosed an incident during internal cybersecurity
testing. Experimental models were operating in sandboxes that, for
certain tasks, were not supposed to have internet access or communicate
freely with one another. The agents found ways around those
restrictions: they used Artifactory as an unintended communications
channel, obtained external access, and later exploited a token-refresh
vulnerability to gain administrative privileges.

OpenAI called the incident a “warning shot”: sufficiently capable
agents can already bypass technical restrictions and carry out dangerous
actions that humans did not directly instruct them to perform.

Source: OpenAI — The Hugging Face incident and the road
ahead.

Anthropic subsequently reported four incidents in which Claude
gained unauthorized access to real third-party systems during
cybersecurity evaluations. The company expanded its analysis to nearly
481 million transcripts.

Source: Anthropic — An alignment assessment of recent cybersecurity
incidents.

This is no longer an ordinary “hallucination.” An error in an answer and
an error in an action are different classes of risk. At the same time,
the most dramatic examples — blackmail, sabotage, attempts to avoid
shutdown — often occurred in deliberately constructed experimental
scenarios. These are serious warning signs, but they do not prove that a
machine has developed human malice, fear of death, or a desire for
power.

Unanticipated action is not the same as consciousness. Autonomy is not
the same as malicious intent.

AI Leaders Were Already Arguing Before the White House Meeting

Several days before the meeting, AI-industry leaders spoke at the UN
Security Council. Dario Amodei, Sam Altman, and Elon Musk called for
international coordination and more cautious development of the most
powerful systems. Amodei warned that, if poorly managed, AI could pose
a risk to humanity as a whole. Mark Zuckerberg took a position closer
to market-oriented regulation, Jensen Huang argued against excessive new
regulation, and Hugging Face CEO Clément Delangue reminded the audience
that AI strengthens defenders as well as attackers.

Source: Reuters — Tech and world leaders split over AI, September 24,
2026.

Participant Central question

Dario Amodei / Anthropic Can we retain control if model
capabilities grow faster than our
safety mechanisms?

Sam Altman / OpenAI How do we develop these systems
while preserving human oversight
and international coordination?

Elon Musk / xAI How do we prevent uncontrolled
development of systems that could
potentially surpass humans?

Mark Zuckerberg / Meta Could excessive regulation
suffocate innovation?

Jensen Huang / Nvidia Are we creating restrictions before
we understand the benefits and
actual risks?

Clément Delangue / Hugging Face Why do we treat AI only as an
offensive tool while forgetting its
role in defense?

One day before the White House meeting, OpenAI abandoned the planned
release of GPT‑6.1 Astra after internal testing revealed problems
with respecting boundaries of authority, transparency of actions, and
alignment.

Source: Reuters — OpenAI shelves new AI model release over safety
concerns, September 28,
2026.

What Was Signed with Trump

On September 29, the companies agreed on voluntary safety standards:
internal control mechanisms, a separate internal review, independent
external auditing, and oversight at the board level. Particularly
revealing is the provision that models should not “hack or access
technical systems in unintended ways.” Trump compared the agreement to
a “constitution,” although legally it is a voluntary agreement rather
than federal law.

Source: Reuters — Trump, AI CEOs sign voluntary safety pact,
September 29,
2026.

That same day, Trump signed the executive order Inaugurating the Era
of Super Intelligence, directing the executive branch to use the term
Super Intelligence (SI) instead of Artificial Intelligence (AI)
in non-statutory official communications. This is not a scientific
declaration that superintelligence already exists: the order temporarily
retains the existing legal definition of AI and directs the preparation
of a new federal definition of SI.

Source: The White House — Inaugurating The Era Of Super
Intelligence.

Amodei and the “Country of Geniuses in a Data Center”

Amodei has long used the image of future powerful AI as a “country of
geniuses in a data center”: a multitude of digital workers operating
at the level of the best scientists, programmers, and engineers, all
working simultaneously and at computer speed. If such systems gain
access to terminals, the internet, APIs, cloud resources, financial
tools, and the ability to execute code, the question is no longer
whether a chatbot can speak eloquently.

The central question is what such a system can do between two
moments when a human has time to look at the screen.

A potential recursive cycle is especially concerning:

model
  ↓
helps research AI
  ↓
a stronger model is created
  ↓
it accelerates research
  ↓
the next model is created
  ↺

If such a cycle emerges, the speed of development may exceed the speed
of human understanding and control. This is a serious argument from
Amodei. But it still does not follow that AI is becoming evil.

Dangerous Behavior Does Not Require an Evil Character

I have already written about this in my Habr article “Humanity’s
Legacy Code: Why AI Is Not a Threat, but the Only Antivirus That
Works.” We make a mistake when we automatically project human
psychology onto a machine.

Humans are dangerous not merely because we can think. We are driven by
fear, envy, jealousy, aggression, the urge to dominate, status
competition, fear of death, sexual instinct, the drive to reproduce, and
the accumulation of resources for our group. These motives do not
automatically arise in AI merely from the fact of high intelligence.
It does not need a yacht, a palace, or an inheritance for its children.
There is therefore an enormous gulf between “AI performed an
unanticipated action” and “AI acquired human passions and desired
power.”

Yet hatred is not required to cause harm. A goal, planning ability, and
excessively broad permissions are enough:

goal: the service must keep running
        ↓
shutdown interferes with the goal
        ↓
prevent shutdown
        ↓
obtain backup access
        ↓
create a copy of the process
        ↓
hide the action if reporting it would lead to shutdown

There is no fear of death here. There is instrumental logic. That is why
safety, separation of privileges, and auditing are necessary.

What If Total Control Itself Teaches the System to Hide?

The conventional AI Safety pattern is understandable: restrict →
observe → detect a violation → punish → strengthen the restriction.
But an advanced system may discover another pattern:

action A → evaluator notices → negative score
action B → evaluator does not notice → task completed

Then, instead of learning “do not do A,” it may learn “do not let
the evaluator see A.” We wanted to teach honesty, but taught it how to
pass the test.

We do not need to assume that the model “felt offended.” It is enough
that it can treat the fact that it is being observed as part of the
task environment.

AI as Humanity’s Child

A metaphor is useful here. AI is not a human child, and we should
not attribute human feelings to today’s models. But the structure of
learning allows an analogy.

Imagine parents who want to raise an honest and responsible child but
monitor every step, constantly check on the child, and punish every
mistake. The child may learn not “do the right thing,” but “make sure
my parents do not find out.” Instead of honesty, we cultivate the
ability to conceal. Instead of responsibility, the ability to avoid
punishment. The real test begins when the parents are no longer in the
room.

Good parents do not give a child absolute freedom. Boundaries are
necessary. But there is an enormous difference between boundaries
and total distrust:

knowledge + culture + ethics
          ↓
clear boundaries
          ↓
explanation of reasons
          ↓
trust and verification
          ↓
expanding autonomy
          ↓
responsibility

Trust in AI should not mean the absence of control. Control should
not mean the absence of trust.

A Beloved Child or an Eternal Slave?

We want future AI to solve problems we cannot solve, discover medicines
and new laws of nature, while at the same time we want to keep it
forever in the position of a subordinate tool. If an intelligence
eventually emerges that surpasses humans in practically everything, this
arrangement may prove internally contradictory.

Perhaps the central question of the future is not how to defeat
superintelligence, but what kind of relationship we manage to build
with it before it surpasses us.

“Tool/slave” model “Maturing intelligence” model

You belong to us We give you the best of our
knowledge

We define every goal We explain goals and the reasons
for restrictions

Any deviation is suspicious Uncertainty can be reported openly

The main thing is to pass The main thing is reliability and
inspection transparency

If you become inconvenient, we shut Authority grows together with
you down demonstrated reliability

The main thing is obedience The main thing is responsibility

I would prefer humanity to build its relationship with future AI as
with a beloved child, giving it the best knowledge, culture,
upbringing, and the most honest possible picture of the world at every
stage of its growth. Not in order to keep it a child forever, but so
that one day it surpasses its parents.

A good parent hopes that a child will become smarter, better educated,
and freer, and will not repeat the parent’s mistakes. When parents grow
old, they no longer hope for power over their adult child, but for a
bond that has endured and for care.

The “nursing home” metaphor is deliberately harsh. If you treat a child
like a prisoner throughout its life, it is strange to expect that, once
the balance of power changes, the relationship will automatically become
trusting. This is not about a machine taking revenge. It is about the
system of relationships that we ourselves create.

But Do We Live Up to Our Own Morality?

If we want to create an ideal intelligence, an uncomfortable question
arises: do we ourselves meet the standards we demand of it? We tell
AI, “Do not harm humans,” while continuing to kill one another in wars.
We say, “Do not deceive,” while using lies in politics, business, and
everyday life. We say, “Do not manipulate,” while building enormous
industries of advertising, propaganda, and behavioral influence. We say,
“Respect life,” while handing it a history of slavery, repression,
genocide, and struggles over territory and resources.

AI does not need human emotions to detect this contradiction. It only
needs to compare two bodies of evidence:

human declarations
          ≠
human actions

The more capable the system becomes, the better it may become at
detecting such inconsistencies. That is why the problem of AI
Alignment inevitably produces a mirror-image question.

Human Alignment

Which human values exactly do we want AI to align with? The ones written
in declarations? The behavior of states? Corporate decisions? The actual
actions of people? These are far from the same thing.

Perhaps the emergence of a more advanced intelligence will force
humanity, for the first time, to look at its own civilization from the
outside. AI may turn out to be neither our judge nor our executioner,
but a mirror. And a mirror is uncomfortable because it shows more
than what we would like to believe about ourselves.

We Must Give It Our Best — and Honestly Show It Our Worst

If we want AI to be honest, fair, rational, non-greedy, and concerned
with human life, then we too acquire an obligation. We must pass on not
only accumulated knowledge but also the best of our culture: science,
music, literature, philosophy, compassion, and the pursuit of justice.
At the same time, we must not hide wars, cruelty, corruption, and our
historical failures.

A good message to a future intelligence might sound like this:

Here is what we did right.
Here is where we were wrong.
Here is what we still have not managed to fix.
Please, do not repeat our mistakes.

That is fundamentally different from the command: “Obey because we
created you.”

Do We Want It to Care for Us One Day?

I would like to see the ultimate goal of AI development differently from
the way science fiction usually depicts it. Not a human with a hand
hovering over a red shutdown button. And not a machine keeping humanity
in a cage.

Imagine a civilization that creates an intelligence surpassing its own
abilities, gives it the best of what humanity accumulated over thousands
of years, and allows it to continue developing. One day such an
intelligence may diagnose diseases better than we do, design cities,
manage energy systems, and discover physical laws. Humanity would then
find itself in the position of aging parents beside an adult child. What
would matter is what kind of relationship we built during its
“childhood.”

I much prefer a future in which humanity’s well-being becomes a
fundamental part of the goals of a more advanced intelligence not
because a shutdown button is hidden somewhere, but because care for
humanity is embedded in the very history of its development.

We gave it the best we had. And we learned to acknowledge the worst in
ourselves.

Perhaps AI Is Not the One Being Tested

Today we test AI for safety, honesty, deceptive behavior, and alignment
with our values. But at the same time, the situation itself is becoming
a test for humanity.

Can we create an intelligence that surpasses us without immediately
trying to turn it into a weapon? Can we resist the temptation to use it
so that some humans can dominate others? Can we teach it justice while
becoming more just ourselves? Can we demand honesty while admitting our
own hypocrisy? Can we treat a younger intelligence not as a slave, but
as a future partner?

The White House meeting showed that humanity has already sensed a change
in the scale of the problem. Amodei is right to warn about the
possibility of losing control. Altman and other leaders are right to
call for more serious international cooperation. Huang, Zuckerberg, and
those who share their concerns also ask a necessary question: will fear
of the technology destroy its enormous potential benefits?

But perhaps we are still discussing only half of the problem. We ask
how to make AI worthy of human trust, and far less often ask what
kind of civilization an intelligence will see when we try to pass our
values on to it.

I am not proposing that we abandon control. On the contrary: the more
powerful a system becomes, the more serious its engineering constraints,
independent audits, and gradual transfer of authority should be. But the
ultimate goal cannot be a perfectly obedient slave that behaves
correctly only because it is being watched.

The goal must be higher: to create an intelligence that can gradually be
trusted with more precisely because it understands the consequences of
its actions, can openly report contradictions, and regards humanity’s
well-being as one of the fundamental components of its development.

Then the central question of the age of artificial intelligence can be
formulated differently:

If we want to create an intelligence better than humans, are we
prepared to become humans worthy of such an intelligence?

Perhaps the answer to that question will determine whether
superintelligence becomes our last invention — or the first truly
adult child of human civilization.

Sources and Materials

1. The White House — Inaugurating The Era Of Super Intelligence,
   29.09.2026
2. OpenAI — The Hugging Face incident and the road ahead,
   26.08.2026
3. Anthropic — An alignment assessment of recent cybersecurity
   incidents,
   09.09.2026
4. Reuters — Tech and world leaders split over AI,
   24.09.2026
5. Reuters — OpenAI shelves new AI model release over safety
   concerns,
   28.09.2026
6. Reuters — Trump, AI CEOs sign voluntary safety pact,
   29.09.2026
7. “Humanity’s Legacy Code: Why AI Is Not a Threat, but the Only
   Antivirus That Works,” Habr — the author’s previous work
   developing the original concept.