Defenders are getting AI security tools too

Not long ago, Anthropic says, AI-written bug reports were mostly noise to the volunteers who look after open-source code. Now, it says, some maintainers ask to receive everything its models find, checked or not.

That's the background to the Anthropic Cyber Mission, which The Register covered on October 9. Anthropic's announcement covers two areas: a program for the companies that secure critical infrastructure, and a service called OSS Scanner for open-source projects. The second one is closer to home for most teams, because almost all software, including the AI tools your team builds, sits on open-source code.

What OSS Scanner is, and who gets it

According to Anthropic's post on the service, OSS Scanner is opt-in. Projects that join get periodic security scans from Anthropic's strongest models at no cost. Each report includes a working example of how the bug could be exploited, an explanation, and a suggested fix where one is available.

Two limits matter. First, it isn't for every project. Eligibility follows criteria similar to those used by Google's OSS-Fuzz program: projects should have a "critical impact on infrastructure and user security", and Anthropic decides case by case. The Register sums it up plainly: the offer "does not extend to all open-source projects, nor is it entirely free." In its reading, the cost is that maintainers' inputs and outputs can be used for model training under the terms Anthropic applied to the service.

Second, the reports are sent without human review. Anthropic says that makes them faster, but also means some will be wrong, for example with an inflated severity rating. It says it expects a true-positive rate above 90%.

The numbers, as Anthropic tells them

These figures come from Anthropic, about its own work, so read them that way.

On CyberGym, an academic benchmark for finding vulnerabilities, Anthropic says language models went from finding under 20% of vulnerabilities at the beginning of last year to over 85% this year. It says that over six months its models flagged more than 29,000 candidate vulnerabilities in important open-source projects, and that it has only been able to review and triage about 6,000 by hand.

Anthropic also makes a forecast: "Our forecast is that in two years, AI will favor defense: it will be easier to catch bugs before they ship, write fundamentally secure software from scratch, and actively defend systems with models." It adds that in the near term, "that may not be true."

The shift worth noticing is simple. The same kind of model that can find a weakness can also explain it and suggest a fix, and that work is now being pointed at the shared code everyone depends on.

What this means for the code your team builds

If your team builds its own AI tools, you probably won't qualify for OSS Scanner, and you don't need to. Two things still apply.

The fixes made upstream only reach you when you update. A bug patched in a library your tool uses this month is still in your copy until someone pulls the new version in.

Your own code needs its own checks. Many code hosting services already include dependency and security scanning: alerts when a package you use has a known flaw, and scans of your own code for common mistakes. They're easy to leave switched off, or to switch on for one repository and forget for the rest.

One thing to switch on

Turn on the dependency and security scanning your code host already offers, for the code your team builds. Start with the repository behind your most-used internal AI tool, read the first batch of alerts together, and decide who looks at new ones when they arrive. A scanner nobody reads is just another inbox.