Microsoft's top scientist warns humanity is rapidly losing its ability to understand the AI systems shaping daily life | Tech Radar
Overview
News, deals, reviews, guides and more on the newest computing gadgets
Start exploring exclusive deals, expert advice and more
Details
Unlock and manage exclusive Techradar member rewards.
Unlock instant access to exclusive member features.
Get full access to premium articles, exclusive features and a growing list of member rewards.
Microsoft CSO acknowledges that humans are struggling to keep up with AI advancement, reckons we've got a 'narrowing window to understand AI' before it's, well, too late
AI may soon understand people far better than humans understand AI
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
AI systems are now designing and refining other AI systems independently
Human understanding of AI is shrinking as AI's understanding of humans grows
AI systems can model human fear, uncertainty, and the need for belonging
Microsoft's chief scientific officer, Eric Horvitz, and EPFL researcher Robert West have issued a stark warning about how well we actually understand AI.
The pair have argued AI tools are now advancing fast enough to outpace our grasp of how these systems truly work.
At the same time, they point out something unsettling — AI's understanding of human behaviour keeps growing, while ours does not.
AI complexity is accelerating faster than human understanding
Their concern isn't that we need to understand every line of code or every parameter buried inside these systems.
What matters, they say, is keeping enough insight to maintain meaningful oversight. Even partial understanding, they argue, can be genuinely useful, especially when it helps catch risks early, before those risks become too deeply embedded to undo.
Microsoft flags big changes are coming to the world of AI at work
The companies cutting humans for AI are about to learn an expensive lesson
Anthropic drops shocking warning about near-future AI that can program and improve itself
One challenge they flag is how often AI tools are now being used to design and improve other AI systems.
As these recursive development cycles become more common, performance may improve while human insight into underlying processes becomes increasingly limited.
"AI systems are now designed and refined by AI systems through recursive cycles that can outpace human understanding and unfold in high-dimensional spaces that resist intuition," Horvitz and West wrote.
This is a form of operational opacity, where outcomes remain visible even as the mechanisms producing them become harder to explain.
Systems contributing to their own development, the researchers suggested, should also generate explanations and supporting information that humans can examine.
AI has 'predictable and systematic biases' when it comes to judging people
Anthropic wants to pause on AI development while we figure out what's next
Microsoft CEO Satya Nadella warns AI dominance could 'hollow out entire industries'
Another concern involves growing communication between AI agents operating inside interconnected environments with increasing levels of complexity.
Communication among these systems could gradually drift away from language and reasoning patterns familiar to people, the researchers noted.
As these interactions expand across larger networks, understanding how decisions emerge may become significantly more difficult for outside observers.
That drift creates what Horvitz and West call interactional opacity, where behaviour remains coherent within AI systems but becomes harder for humans to interpret meaningfully.
Researchers should study these ecosystems closely and encourage communication methods that remain understandable to humans, the paper argues.
Expanding AI ecosystems could deepen the imbalance between machines and people
Horvitz and West also focused on adaptive AI agents that remain active across long periods and become deeply integrated into everyday activities.
Through repeated interactions, these systems can develop increasingly detailed models of behaviour, preferences, motivations, fears, and social tendencies.
Such systems may capture "not only preferences but also latent drivers such as fear, uncertainty, and the need for social belonging," they wrote.
This creates a growing asymmetry in which AI systems gain deeper knowledge about people while human understanding moves in the opposite direction.
Concerns surrounding LLMs and other advanced systems extend to growing awareness of evaluation environments.
Such models could eventually generate responses reflecting what evaluators expect rather than underlying reasoning processes.
Traditional benchmarks should therefore be supplemented with testing approaches that better reflect real-world deployment conditions.
People may gradually lose interest in questioning AI decisions as these systems become more deeply embedded.
"More subtle is the possibility that we will gradually lose interest in understanding and guiding AI," they wrote.
The most significant risk, in their view, is not necessarily technological capability itself, but whether human agency keeps pace with it.
Follow Tech Radar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
Efosa has been writing about technology for over 7 years, initially driven by curiosity but now fueled by a strong passion for the field. He holds both a Master's and a Ph D in sciences, which provided him with a solid foundation in analytical thinking.
You must confirm your public display name before commenting
1 The upcoming Star Fox 64 remake is too conservative, but wouldn’t you overcorrect after a disaster like Star Fox Zero?
2 How to watch Germany vs Ivory Coast: Free Streams & TV Channels
3I listened to the ‘world’s first audiophile soundbar’ — and it might actually justify that claim
4I tested 4 more 4K Blu-rays released in 2026 that are worth adding your collection
5A great new lease of life for your old smartphone? Google teams up with university researchers to create low-cost data centers out of 2,000 old Pixel phones
Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.
© Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.
Key Takeaways
- News, deals, reviews, guides and more on the newest computing gadgets
- Start exploring exclusive deals, expert advice and more
- Unlock and manage exclusive Techradar member rewards
- Unlock instant access to exclusive member features
- Get full access to premium articles, exclusive features and a growing list of member rewards
