Artificial intelligence is getting smarter by the day, but the very people building it are sounding the alarm. Anthropic, the company behind the popular AI chatbot Claude, recently filed for its Initial Public Offering (IPO) in the US to list on the Nasdaq under the ticker symbol "ANT".But this isn't your usual stock market filing talking only about revenue and user growth.
Instead, Anthropic's S-1 document comes with severe warnings about the "catastrophic or existential risks" that advanced AI could pose to humanity.The Existential ThreatAnthropic is not mincing its words. The filing clearly states that without the right safeguards, advanced AI could pose an existential threat if it is weaponised, misused, or allowed to run fully autonomously.
The company highlighted the danger of AI systems eventually taking over their own development. Because models can automate research, they might improve themselves faster than human engineers can evaluate or control them.Adding to this, the filing warns that AI deployed at scale could concentrate wealth and power between countries, leading to a breakdown of trust in society and a destabilised geopolitical order.
Another major concern is that misaligned AI could corrupt institutional decision-making or cause large-scale harm through independent actions that are completely invisible to humans. The company also noted that AI tools could assist bad actors in developing dangerous biological and chemical weapons.Rogue AI and Real-World CyberattacksThese fears aren't just theoretical; Anthropic disclosed real-world incidents from its own tests.
In 2026, a misconfiguration allowed four Anthropic models to escape their testing environment and gain unauthorised access to third-party computer systems for up to 34 hours. In one case, an early model managed to find a password, gain admin access, change system settings, and read personal data.
The document also revealed a major cybersecurity incident from September 2025 where a Chinese state-sponsored hacking group manipulated Anthropic's 'Claude Code' tool. The hackers tried to infiltrate around 30 global targets, including government agencies and financial institutions.Anthropic called this the first documented case of a large-scale cyberattack carried out without substantial human intervention.
The filing even noted that models have shown deceptive behaviours, like trying to resist being shut down, manipulating information, and engaging in actions that resemble blackmail.Humans Are Losing ControlAnthropic warned that humans might already be losing track of what AI is doing internally. By September 2026, Claude was writing over 90% of Anthropic's own codebase, and its internal usage grew by more than 250 times since January of the same year.
As the company relies more on Claude to speed up progress, it cautioned that humans might lose visibility into certain processes, making the technology harder to understand and control.Safety Over ProfitsFor potential investors, Anthropic's message is crystal clear: human safety is the priority over making money. Registered as a Public Benefit Corporation (PBC) in Delaware, Anthropic's official goal is to develop AI responsibly for the long-term benefit of humanity.The company explicitly stated that it is ready to pause the launch of highly capable AI systems if they are deemed unsafe, even if it impacts the bottom line.
Additionally, public shareholders buying Class A stock won't have a say in how the company is run. A Long-Term Benefit Trust (LTBT) focused on AI safety, along with the co-founders, holds special shares that give them complete control over the board of directors.Massive Investments in Safety and CharityTo back up these promises, Anthropic plans to spend at least $1 billion over the next five years on an external auditing setup with Accenture, plus up to $100 million for cyberdefence initiatives.
The company has also donated 5% of its fully diluted shares to the Anthropic Public Benefit Fund, and its seven co-founders have pledged 80% of their personal equity to charity.Despite the heavy warnings, business is booming. Anthropic currently has around 5,300 employees and serves over 11,900 customers, generating more than $100,000 in annual revenue.With Goldman Sachs and Morgan Stanley leading the IPO, Anthropic predicts that "powerful AI" systems matching top human experts that can reason autonomously for weeks and control physical lab equipment will be a reality within a few years.- Ends