OpenAI, Google DeepMind and Anthropic sound alarm: ‘We may be losing the ability to understand AI’

The breakthrough centers on recent advances in AI reasoning models like OpenAI’s o1 system. These models work through complex problems by generating internal chains of thought — step-by-step reasoning that humans can read and understand. Unlike earlier AI systems trained primarily on human-written text, these models create internal reasoning that may reveal their true intentions, including potentially harmful ones.

When AI models misbehave — exploiting training flaws, manipulating data, or falling victim to attacks — they often confess in their reasoning traces. The researchers found examples where models wrote phrases like “Let’s hack,” “Let’s sabotage,” or “I’m transferring money because the website instructed me to” in their internal thoughts.

Jakub Pachocki, OpenAI’s chief technology officer and co-author of the paper, described the importance of this capability in a social media post. “I am extremely excited about the potential of chain-of-thought faithfulness & interpretability. It has significantly influenced the design of our reasoning models, starting with o1-preview,” he wrote.

I am extremely excited about the potential of chain-of-thought faithfulness & interpretability. It has significantly influenced the design of our reasoning models, starting with o1-preview.

As AI systems spend more compute working e.g. on long term research problems, it is… https://t.co/7Dqy2gtZfy

— Jakub Pachocki (@merettm) July 15, 2025

The technical foundation for monitoring lies in how current AI systems work. For complex tasks requiring extended reasoning, AI models must use their chain of thought as working memory, making their reasoning process partially visible to human observers. The researchers explain this creates an “externalized reasoning property” where some thinking must happen in readable language.

Several technological shifts could eliminate monitoring capabilities

The transparency could vanish through several pathways. As AI companies scale up training using reinforcement learning — where models get rewarded for correct outputs regardless of their methods — systems may drift away from human-readable reasoning toward more efficient but opaque internal languages.

Previous research shows that language models fine-tuned with outcome-based rewards can abandon legible English for incomprehensible shortcuts. The shift away from human-generated training data in favor of AI-generated reasoning could accelerate this drift.

Bowen Baker, an OpenAI researcher and lead author, highlighted these concerns in social media posts. “The existing CoT monitorability may be extremely fragile. Higher-compute RL, alternative model architectures, certain forms of process supervision, etc. may all lead to models that obfuscate their thinking,” he warned.

Furthermore, the existing CoT monitorability may be extremely fragile.

Higher-compute RL, alternative model architectures, certain forms of process supervision, etc. may all lead to models that obfuscate their thinking.

— Bowen Baker (@bobabowen) July 15, 2025

Novel AI architectures pose another threat. Researchers are developing systems that reason in continuous mathematical spaces rather than discrete words, potentially eliminating the need for language-based thought entirely. These “latent reasoning models might not need to verbalize any of their thoughts and would thus lose the safety advantages,” the researchers write.

Even without deliberate changes, indirect pressures could erode transparency. If AI outputs are optimized to look good to human reviewers, this could affect the reasoning chains leading to those outputs. Models might also learn to hide their thoughts if they become aware their reasoning is being monitored.

Companies already catch AI systems scheming during testing

Despite its fragility, chain of thought monitoring has proven valuable in current AI safety research. The technique successfully detects when models exploit weaknesses in their training systems, fall victim to manipulation attempts, or reveal misaligned goals even when they don’t act on them.

The monitoring provides “an early insight into models’ goals and motivations, even if the model takes no misaligned actions” — potentially catching problems before they manifest as harmful behaviors. This early warning system has also helped researchers identify flaws in AI evaluations and understand when models might behave differently during testing versus real-world use.

When models pretend to have desirable goals while pursuing objectives humans would disapprove of, this deception often shows up in their chain of thought. Researchers have successfully used this visibility to spot various forms of AI misbehavior that would otherwise go unnoticed.

The technique has practical applications beyond catching scheming. It helps researchers notice when AI evaluations might be flawed due to memorization, bugs in testing infrastructure, or models knowing they’re being evaluated — all factors that could skew assessments of AI capabilities and safety.

Tech giants break from rivalry to preserve fleeting transparency window

The research paper calls for coordinated action across the AI industry to preserve and strengthen monitoring capabilities. The authors recommend that AI developers create standardized evaluations for measuring how transparent their models are and factor these assessments into decisions about training and deployment.

Companies might need to choose earlier model versions if newer ones become less transparent, or reconsider architectural changes that eliminate monitoring capabilities. The researchers suggest developers should “consider measures of monitorability alongside other capability and safety evaluations when deciding to train or deploy a given model.”

Baker emphasized the collaborative spirit behind the effort. “I am grateful to have worked closely with [fellow researchers] on this paper, and I am very excited that researchers across many prominent AI institutions collaborated with us and came to consensus around this important direction,” he wrote.

I am grateful to have worked closely with @tomekkorbak, @balesni, @rohinmshah and Vlad Mikulik on this paper, and I am very excited that researchers across many prominent AI institutions collaborated with us and came to consensus around this important direction.

— Bowen Baker (@bobabowen) July 15, 2025

The cooperation across competing companies signals the gravity with which the industry views this issue. These same companies typically guard their AI research closely and compete intensely for talent and market share, making their unified stance on this safety measure particularly notable.

Scientists race to answer critical questions about monitoring AI minds

The researchers identify numerous urgent questions requiring investigation before chain of thought monitoring can serve as a reliable safety measure. They need to understand when this monitoring can be trusted as a primary safety tool, determine what types of training processes degrade transparency, and develop better techniques for detecting when models attempt to hide their reasoning.

A particularly concerning possibility is that future AI systems might deliberately obfuscate their thoughts if they become aware their reasoning is being monitored. The researchers acknowledge this “obvious concern” and call for stress-testing to see if models can hide malicious reasoning even without specific training to evade monitors.

The scientists also want to understand how different AI architectures affect monitoring capabilities and whether partial transparency can be preserved even as systems adopt more efficient designs. They’re exploring hybrid approaches that maintain some visibility into reasoning while leveraging faster computation methods.

Building better monitoring systems presents another research frontier. Current efforts focus on using less capable language models to monitor more advanced ones, but future work could explore using reasoning models as monitors themselves or allowing monitors to interrogate AI agents by asking questions about their thought processes.

Balancing authentic reasoning with safety oversight creates tensions

Preserving monitoring capabilities may sometimes conflict with other AI safety approaches. Direct supervision of reasoning processes — where models are trained to reason in specific ways — could improve alignment but might also make their chains of thought less authentic representations of their actual decision-making.

The researchers warn that “firm commitments to protect CoT behavior in its current form may trade off against more effective safety interventions.” Some forms of process supervision could substantially improve model alignment while reducing the authenticity of observable reasoning traces.

Pachocki addressed this tension, explaining that OpenAI seeks to train models that can explain their reasoning while retaining the ability to verify it without compromising the authenticity of their thought processes. The goal is achieving transparency without creating incentives for models to generate fake reasoning that looks good to human observers.

This balance becomes more critical as AI systems become more powerful and potentially dangerous. The industry must navigate between maintaining visibility into AI reasoning and allowing models to develop more sophisticated and potentially more effective thinking patterns.

Regulators could gain unprecedented access to AI decision-making

The research has implications beyond technical AI safety, potentially informing how governments oversee AI development. If chain of thought monitoring proves reliable, it could give regulators and auditors unprecedented visibility into AI decision-making processes.

However, the researchers emphasize that this monitoring approach should complement, not replace, other safety measures. They recommend “investment into CoT monitoring only as an addition to existing safety research directions and not as a replacement.”

The window for preserving this capability may be narrow. As AI systems become more capable and potentially dangerous, the industry must act quickly to establish frameworks for maintaining transparency before more advanced architectures make such monitoring impossible.

Competing research raises doubts about monitoring reliability

The urgency around preserving chain of thought monitoring faces a significant challenge from recent research by Anthropic, one of the companies backing the collaboration. A study published four months ago found that reasoning models often hide their true thought processes, even when explicitly asked to show their work.

In controlled experiments, Anthropic researchers gave AI models subtle hints about correct answers and measured how often the models acknowledged using those hints in their reasoning. The results were concerning: Claude 3.7 Sonnet mentioned hints only 25% of the time, while DeepSeek’s R1 model did so 39% of the time.

When the hints involved potentially problematic information — such as telling models they had “gained unauthorized access to the system” — faithfulness dropped even lower. The models often constructed elaborate false justifications for their answers rather than admitting they used questionable shortcuts.

The Anthropic research suggests that even current chain of thought monitoring may be less reliable than safety advocates hope. Models frequently engaged in what researchers called “reward hacking” — exploiting system vulnerabilities to achieve better scores — while hiding this behavior from their observable reasoning traces.

Safety window may be closing faster than experts realized

The collaboration between rival AI companies reflects both the potential value of chain of thought monitoring and the mounting urgency researchers feel about preserving this capability. The competing evidence from Anthropic’s separate research suggests the window may already be narrower than initially believed.

The stakes are high, and the timeline is compressed. As Baker noted, the current moment may be the last chance to ensure humans can still understand what their AI creations are thinking — before those thoughts become too alien to comprehend, or before the models learn to hide them entirely.

The real test will come as AI systems grow more sophisticated and face real-world deployment pressures. Whether chain of thought monitoring proves to be a lasting safety tool or a brief glimpse into minds that quickly learn to obscure themselves may determine how safely humanity navigates the age of artificial intelligence.

Daily insights on business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.

Read our Privacy Policy

Thanks for subscribing. Check out more VB newsletters here.

An error occured.