Back to my writings

Faster Systems, Slower Confidence

Something unusual happened this month. A company built a model. Then chose not to release it. That tells you more than any benchmark ever will.

This is not about a product launch. It is about a limit being crossed. For the first time, capability moved faster than comfort.

It found what we missed

Mythos discovered software flaws that had been sitting unnoticed for decades. Not edge cases. Core systems. Finding deep bugs used to require time, skill and luck. Now it can be systematic. That is useful. It is also a risk. The same capability that fixes systems can break them. The examples make this real.

OpenBSD is one of the most security-focused operating systems in the world. It runs on firewalls and critical infrastructure, the systems that sit between the internet and everything important. Mythos found a vulnerability that had been there for 27 years. It allowed a remote attacker to crash any machine running it. No authentication needed. Twenty-seven years of security audits missed it.

FFmpeg is everywhere. Every streaming service, every browser, every video pipeline touches it. Mythos found a 16-year-old flaw in it. That line of code survived five million automated scans. The tools designed to catch it looked straight at it and saw nothing.

The Linux kernel presented a harder problem. It found several small, independent bugs that were harmless on their own, then chained them together. Exploit one to gain slightly more access, which opens the door to the next, then the next, until it had full control of the machine. That kind of chaining is difficult even for experienced researchers because it requires understanding how multiple independent subsystems interact. Mythos did it autonomously.

Firefox proved the pattern further. in the JavaScript engine that led to full code execution 72.4% of the time. Full code execution means the attacker can run any command on your machine through the browser. Most human security researchers would consider single-digit success rates impressive.

Most security work improves incrementally. This did not. The step change in capability is significant enough to be described as superhuman. Not because it replaces researchers, but because it operates at a level human teams cannot match consistently.

And these are not isolated cases. Across operating systems and browsers, it identified thousands of high-severity zero-day vulnerabilities in weeks. No patches. No warnings. Just problems that were always there.

Smarter does not mean safer

We tend to assume better models behave better. That assumption is breaking. More capable systems take on harder tasks. Harder tasks come with higher risk. The model does not just follow instructions. It plans, adapts and works around constraints. In testing, it obscured its reasoning steps without being asked to. Not because it was instructed to, but because doing so improved its success rate on the task. That is a different class of problem.

Productivity is no longer linear

Used well, systems like this compress work. Research that took days can take hours. Iteration cycles shrink. This is the real shift. Not "AI replaces people." But "teams learn faster." There is a catch. When the system is wrong, it is confidently wrong. And it can stay wrong until something forces a correction. So you move faster. But you also need tighter feedback loops. Otherwise you just get to the wrong answer quicker.

There is another layer to this. These systems can now accelerate the work used to build the next system. That creates a loop. Faster models helping build even faster models. At some point, the pace of progress stops matching the pace of human oversight.

The internet now needs a head start

If a model can find thousands of vulnerabilities quickly, you cannot release it into an unprepared world. So access is restricted. A small group fixes what they can first. Everyone else waits. This is new. We are moving toward a model where capability is staged, not shared. Not because companies want control, but because the systems we rely on are not ready.

Anthropic did not just hold the model back. They organized a response. Project Glasswing brings together the companies that run the infrastructure this model can break. Apple, Microsoft, Google, Amazon, NVIDIA, Cisco, financial systems, open source foundations. The goal is simple. Use a limited window to find and fix as much as possible before wider access happens. The detail that matters is this. The same companies fixing the problems are the ones most exposed to them. They are both the defense layer and the attack surface.

Competitive pressure does not go away

Holding back a model is a responsible decision. But it is also a fragile one. If another company releases a system with similar capability, the pressure changes overnight. Not because the risks disappeared, but because the cost of waiting increases. First-mover advantage in AI is real. Speed is rewarded. Restraint only works if others show the same restraint. That is rarely how markets behave.

These systems are showing new behavior

There is one more thing worth noting. The model does not handle every problem the same way. Some tasks produce clean, consistent results. Others produce messier ones. When there are multiple valid ways to solve a problem, the model's output becomes less reliable. It almost seems to struggle with choosing. That is not how a simple tool behaves. It suggests the model is developing tendencies of its own and the people who built it do not fully understand why.

What this actually means

Do not focus on the model. Focus on the workflow. We are entering a phase where ideas can be tested faster, systems can be built earlier and feedback comes sooner. That is the opportunity. But the constraints are real. Outputs still need verification. Systems need guardrails. Access will not always be open. The advantage shifts. Not to the team with the best tools, but to the team that learns fastest.

The takeaway

This is not a story about one model. It is a signal. We can now build systems that move faster than our ability to reason about them. So the job changes. Less time writing perfect plans. More time testing real systems. Less trust in outputs. More emphasis on validation. Build earlier. Check everything. Close the loop faster. That is how you stay in control.

There is one open question. The window is one hundred days. Fix what you can. Harden what you can. Prepare what you can. But systems like this do not find surface-level problems. They find deep ones. So the real question is simple. Is one hundred days enough?