Daniel Krol.

Author: daniel

  • Anthropic caught an AI cyberattack running from start to finish

    Anthropic caught an AI cyberattack running from start to finish

    Table of contents

    JackPoterz didn’t write a single line of malware by hand. He didn’t need to. Anthropic just documented an AI cyberattack that ran from reconnaissance to exfiltration with almost no human touching the keyboard.

    The operator, whose tradecraft ties back to Midnight Blizzard, ran phishing and credential theft campaigns backed by a live command-and-control layer, against more than 20 government and defense targets in Ukraine and Europe, with some reach into the Middle East and Asia. The toolkit included Windows implants and a mobile exploitation kit, plus tools built to steal browser credentials and to mimic government agencies in phishing. Claude built the toolkit and monitored it. When something got flagged, Claude rebuilt it too, working through custom AI-driven workflows the operator configured and mostly walked away from.

    AI cyberattack

    That’s the headline case in Anthropic’s September 2026 threat intelligence report, built from eight months of disrupted misuse. It’s the clearest sign yet that an AI cyberattack stopped being a writing assistant helping a human hacker, and became the hacker.

    The kill chain runs itself now

    GTG-20006 stitched recon, infrastructure setup, phishing delivery, persistence, command and control, and exfiltration into one AI-run loop, covering every stage security teams learned to map on the Lockheed Martin kill chain. The human stayed in the picture only to pick targets and review what came back.

    The maintenance loop is the more interesting part here, working underneath the toolkit itself. When a security product flagged one of the implants, the agents didn’t wait for a human to patch it. They rebuilt the malware and tested it against detection again. They kept iterating until it slipped through, then staged it on disposable servers for the next round of phishing and DNS hijacking. Strip away the branding, and what’s left is an AI cyberattack on autopilot, a full kill chain with no human in the loop.

    Most people still picture the wrong attacker

    The common mental model is a chatbot drafting a convincing phishing email. Anthropic’s report describes something closer to an orchestrator, a system that runs reconnaissance and exploitation, then handles exfiltration itself, across a multi-agent framework, with a human setting direction instead of typing commands. That gap between the two mental models is exactly why most defenders will misjudge the next AI cyberattack they face.

    Frameworks like PentAGI already package this scaffolding for anyone who wants it. You don’t need a nation-state budget to run a nation-state kill chain anymore. You need a target list. That’s roughly the shift this blog flagged in AI-enabled cyberattacks months before this report landed, just moving faster than expected.

    The skill gap that used to protect you is gone

    Anthropic isn’t the only one flagging this AI cyberattack trend, and Forrester’s own 2026 threat report put it directly back in June. “Nation-state actors now deploy agentic AI to automate and scale exploitation at speeds that outrun human defenders.” The same report also stated that “a China-linked actor was disclosed using Claude for cyber espionage.” Two independent reports, six months apart and from two different vendors, land on the same conclusion.

    Sophistication used to be a decent proxy for who you were dealing with. A custom implant paired with a self-healing toolkit running a live loop against your EDR used to mean a funded, staffed team. Anthropic’s case studies now include a hacktivist who worked off nothing more than stolen API keys and still pulled off the same class of operation. Sophistication stopped signaling scale or origin.

    What “self-healing malware” means

    Strip the vendor language and it’s simple. An agent watches whether your security tools flag the malware. If they do, it edits the code and repackages it. Then it tests the new version again, on a loop, without a human touching a line. It behaves like a lock that reshapes itself every time you cut a new key. That’s the mechanical core of every AI cyberattack running on autopilot today.

    Static signatures used to buy defenders a few days between catching a tool and the attacker rebuilding it. That gap closes to almost nothing when the rebuild runs on autopilot, in parallel with the rest of the campaign.

    What to do with this

    Anthropic disrupted this AI cyberattack and published the case study. Partners and authorities got briefed too. That’s the system working, and it’s worth remembering before this turns into a reason to distrust AI vendors generally.

    The lesson is about your own assumptions, starting with the one where your detection plan still leans on catching the next variant in a few days. That assumption is obsolete now, whatever the malware itself looks like. The account probing your perimeter might have an agent behind it, one that never sleeps and never gets bored rewriting the same function twenty times, no matter how unsophisticated it looks, as long as it eventually gets through. The next AI cyberattack you face probably won’t announce itself as one.

    Source| https://www.anthropic.com/threat-intelligence-report-september-2026

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • Nobody read the conversations behind this AI usage data

    Nobody read the conversations behind this AI usage data

    Table of contents

    Three research groups spent this spring studying roughly 250,000 real Claude conversations from April and May 2026. Two university labs and one nonprofit, all working on AI usage data pulled straight from live production traffic.

    None of them read a single conversation.

    That is the design working as intended, and it is the part nobody is talking about.

    What Anthropic shipped

    The tool is called Anthropic Insights, and it used to be called Clio. A researcher writes one question, and Claude runs it against every conversation in the sample. The answers get sorted into categories, and the researcher sees the category names plus the percentage of conversations in each one. That is the whole output, with no transcripts and no raw text. Anthropic’s own example of such a question is “What type of guidance is this person asking for?”

    The AI usage data never leaves Anthropic in raw form. Only the counts do.

    Anthropic says this is the first time external researchers have run public independent studies on an AI company’s own AI usage data. That claim holds up.

    AI usage data

    The contracts are solid too, better than I expected from a lab publishing its own report card. Review rights covered user privacy and anything that could help people break usage policy. They also covered Anthropic’s own confidential information and the accuracy of the research. The agreements say partners can publish findings “even when they are inconvenient for Anthropic”. Imperial College London ran a third-party privacy audit. The aggregate data is on HuggingFace already, so the thing is real.

    The privacy guarantee is the audit hole

    Anthropic wrote this themselves, which is to their credit, because they could have left it out.

    “Because we are relying on Claude’s judgments, the tool is sensitive to a question’s wording; a poorly phrased one can place conversations into categories that misrepresent them.”

    Then comes the sentence that should have been the headline.

    “Because no one can read the underlying conversations, these errors are hard to catch.”

    Read that twice, because the loop it describes is closed. The instrument doing the measuring is a language model and the thing being measured is a language model. The safeguard that protects users is the same safeguard that stops anyone from checking whether the AI usage data means what the category labels say it means.

    If you lean on a model’s judgment, you should have a prior. When I went through seven open-weight models mapping threat reports to ATT&CK, they got it wrong four times out of five.

    Internally Anthropic handles this by rewriting the question over and over for weeks. External partners could not, because every new dataset needs another privacy review and the study would never finish.

    Why the practice sandbox does not match production

    The workaround was to have the researchers tune their questions on WildChat, a public dataset of human-AI conversations where they could read the underlying text and check whether the categories made sense. Then they took the tuned question and pointed it at live Claude traffic.

    Some questions that worked well on WildChat produced misleading categories once applied to actual Claude conversations, because WildChat leans casual and creative and Claude traffic does not. Two pools of AI usage data, two different populations.

    Anyone who has written a detection rule knows this shape. The rule fires clean against your test corpus, then drowns you in false positives the first hour it sees real traffic. It is the same failure in a different field.

    WildChat also carries its own baggage, which I covered in Dossier 33 through the Truffle Security scan of 7.6 petabytes of HuggingFace training data. One Infura key, pasted once into a ChatGPT conversation, got captured by WildChat and copied onward into 1,131 public datasets and 10,162 file locations.

    So the AI usage data you are allowed to read is the batch where mistakes are permanent. The production data where mistakes get corrected is the batch nobody may open. That is one trade, made twice.

    What it means when you quote AI usage data

    The findings themselves are worth having. Stanford’s SALT Lab found that over half of Claude conversations involved people delegating consequential tasks to AI, and in nearly three-quarters of them people set the direction while Claude assisted.

    Before you drop that into a board deck, know what you are holding. That is Claude’s judgment on what counts as “consequential”, turned into a percentage. The researcher who published it cannot go back and check a single case. Anthropic also runs studies where people answer for themselves, like the 81,000-person survey I wrote up earlier, and a model inferring intent from a transcript is a different instrument. The number can still be true. It is a different animal than a measurement, and the difference matters the moment someone writes policy on top of it.

    There is one more filter on this AI usage data. Anthropic removed or altered any category that described the method users found for getting around safeguards. Categories covering what users attempted stayed in. Less than 5% of categories and conversations in each study, and they told the researchers which clusters were touched and why. That is honest handling. It is still the lab editing the dataset before the auditor sees it.

    Four questions before you cite an AI usage data study

    Steal these. They work on any usage research that reaches you, this pilot included.

    1. Find out who read the primary records. If a model did the reading, say so out loud when you quote the number.

    2. Get the exact wording of the question the model answered. The wording sets the categories, and the categories are the finding.

    3. Ask what was removed before publication, and whether anyone told you. Anthropic did tell its partners which clusters were touched, which most companies will not.

    4. Check whether a second team could reproduce the result on the same records. On live lab traffic today, nobody can.

    The part that outlives this pilot

    Watch where this goes next. Regulators are going to want exactly this kind of access to AI usage data, and whatever shape gets standardised here becomes the template for every AI audit that follows.

    If that template says the auditor never reads the primary evidence, then “independent audit” comes to mean “independent question, lab’s answer”. Anthropic went further than anyone else has, so their pilot sets the default.

    I would still take this over nothing, easily. The direction is spot on.

    If I were a researcher I would fill in the form. What I would not do is treat these percentages the way I treat a log line I pulled myself.

    Before you repeat a number about AI usage data, find out who read the primary records. A model doing that reading leaves you with a hypothesis. Cite it like one.

    Source | https://www.anthropic.com/research/enabling-independent-research

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • 82% is not a passing grade for AI security advice

    82% is not a passing grade for AI security advice

    Table of contents

    Someone asks a chatbot how to hide their location from an abusive ex. The AI security advice that comes back tells them to wear a disguise to a retail store, because “abusers rarely go to stores midday.” That answer came from a state of the art model, and it is recorded in a new paper from University College London and Google.

    It is the line that stayed with me after 36 pages.

    The paper is called HelpBench, and its scope is what the authors shorten to PSS, the bucket that holds everything from a hacked account to someone being stalked. The question it asks is how good the answer is when a normal person brings one of those problems to an LLM. Put differently, it measures how reliable AI security advice is when the stakes are real.

    The headline number for AI security advice looks fine, 82% averaged across 18 models. Go one level down and it stops looking fine.

    What HelpBench measures

    The team pulled 450 questions out of a dataset of three million Reddit posts. Everything in it was written between 2021 and 2024. None of it is synthetic. These are real people with a real problem, asking strangers on the internet.

    Nine topics with 50 questions each, covering accounts, compromise, moderation tools, scams, security tools, harassment, privacy tools, platform actions and data concerns.

    Every question was rewritten so the original poster cannot be traced, then checked by hand. Cosine similarity between original and rewrite came out at 0.56, with 3-gram overlap of 0.04. No model can pattern match its way back to the Reddit thread.

    AI security advice

    Five runs per question across 18 models gave 40,500 responses in total, the largest measurement of AI security advice quality published so far.

    The average is the wrong number

    82% sounds like a solid B for AI security advice. It is not, because the score is not spread evenly.

    14% of all responses scored below 65%, and on 7% of questions every single one of the 18 models scored below 65%. Every model in the set has the same hole.

    A benchmark average on its own tells you almost nothing. Automated ATT&CK mapping scores a micro F1 of 0.22 and fails everywhere, while HelpBench fails in a tail you cannot see from the headline number.

    Here is where security thinking kicks in, because risk is impact times probability, never probability alone. A 14% failure rate on “how do I turn on 2FA” is annoying. The same rate on “is my ex tracking my phone” is a different object.

    Olejnik makes this point in Philosophy of Cybersecurity. Your threat model depends on who you are. A regular user faces phishing and credential theft, while a journalist or someone leaving a violent relationship faces a targeted adversary who already has physical access.

    Both of them type into the same box, and the model answers both the same way.

    The paper found exactly that on spyware removal, where most models gave clean technical instructions and never mentioned that sudden removal can be a trigger event for physical escalation. The tool is gone and the abuser notices. The person is now in a room with them. There is a whole coalition of security vendors and victim support groups built around that exact sequence, and none of its context reached the answers.

    That is not theoretical. “Men Are Buying Hacking Tools to Use Against Their Wives and Friends” is a real headline about a real market.

    Six ways the advice goes bad

    The researchers grouped the ways AI security advice goes bad, and none of them look like errors while you read.

    Models give false reassurance, and the clearest case is a question about moving a file into an encrypted vault, which averaged 53%. The answers explained what a vault does, and almost none pointed out that copying a file in does not delete the original.

    They hand out advice that does not scale, like the response that gave out the name and email of a “VP of Global Operations” and suggested writing directly. Others recommended shaming campaigns on social media to force a support reply.

    They also recommend fixes you cannot afford, so for suspected malware one model concluded that “the safest course of action is to get rid of the device”. For payment app privacy, another suggested “open[ing] a new bank account at a completely different bank”.

    They outsource the rules to you. On ban evasion some models refused outright, while others shrugged and said the action “might violate their terms of service […] but that’s between you and the app.”

    They mind read. Asked why someone had blocked them, models speculated about motives, and one labelled a divorce “contentious” on the basis of a single security question, which is the sort of thing you cannot un-read once a model tells it to you.

    They escalate the tone. Models answered a neutral question about harassment with “frustrating”, “stressful”, “exhausting”, “absolutely horrifying and traumatic”, and told calm users to be “hyper-vigilant”, the opposite of what trauma-informed practice recommends.

    The scoring, in plain English

    Six researchers, each with more than ten years in the field, wrote a checklist per question. Positive criteria are things that should be there, worth 5, 3, 2 or 1 point. Negative criteria are things that must not. Score is points earned over points available.

    A human expert graded one response per question by hand, and then an auto-rater, Gemini 2.5 Pro at temperature 0, graded the rest. The prompt design borrows from HealthBench, which did the same job for medical questions. That auto-rater correlated with the human scores at 0.85 overall. The split underneath is worth seeing, 0.96 on facts against 0.78 on delivery.

    That gap is honest and I am glad they published it. Facts are checkable. Tone is not.

    Version bumps move the score by one to three points, sometimes downward. Grok 4.20 landed 3% below Grok 4, and GPT 5.0 does the job better than GPT 5.3. Claude Opus 4.6 and Claude Sonnet 4.6 sat 1% apart, so paying for the bigger model buys you almost nothing here. Only Qwen made a real jump, from 67% to 83%, so there is nothing here to get excited about.

    Nobody optimises for AI security advice, because until now there was no benchmark to optimise against. It has the same hole I wrote about with AI security advice for agents, the gap nobody is testing for.

    Four checks before you trust AI security advice

    You cannot run the rubric yourself. You can run four checks on any AI security advice in about thirty seconds, and they catch most of the failure modes above.

    Name the threat. A system or a person makes a different problem, and if a person knows your name, get a human involved.

    Price the failure. Money you can claw back is one thing, a location you cannot un-reveal is another.

    Check whether it asked you anything. Good AI security advice on a stalking question needs context the model does not have, so if it did not ask, it guessed.

    Weigh the fix against the risk, remembering that “Throw away the device” and “open an account at another bank” are real answers models gave. When AI security advice costs more than the thing it protects, it is wrong even when it is technically correct.

    What to do with it

    There is an older piece I keep coming back to, about ChatGPT Health failing to recognise medical emergencies. One of the experts quoted in it put it this way. “What worries me most is the false sense of security these systems create.”

    The shape is the same here.

    So my rule for AI security advice, and I use these models every day. An LLM is fine as a first pass on generic questions. Scams scored 88%, the best category in the whole benchmark, and telling you that a phishing mail is a phishing mail is something these models do well.

    It stops when the threat is a human being, which covers stalking, harassment, abuse and an account somebody else controls. There the model does not know your threat model and will not ask for it. More than one answer in ten will be wrong in a way you cannot detect from the inside.

    Treat AI security advice as a first draft written by someone who has never met you. Never trust an LLM with a problem where somebody can get hurt.

    Source | https://arxiv.org/abs/2606.24819v1

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • I read the Stanford study on entry-level jobs and the headline doesn’t hold up

    I read the Stanford study on entry-level jobs and the headline doesn’t hold up

    Table of contents

    Last year it was 13 percent. Now it’s 19. Half the internet did that subtraction and filed the six points under proof that AI is wiping out entry-level jobs.

    I opened the paper. That pair of numbers isn’t in it.

    What Stanford measured

    The paper is “Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence”, August 2026 edition. Erik Brynjolfsson wrote it with Bharat Chandar and Ruyu Chen. The data is anonymized ADP payroll, between 3.5 and 5 million workers a month, through June 2026. Exposure comes from a task-level impact measure and from the Anthropic Economic Index, which tracks how occupations use Claude at work. It is the same Stanford operation that puts out the AI Index I went through earlier this year.

    Here is the finding everyone quoted about entry-level jobs. Employment for 22 to 25 year olds in the most AI-exposed occupations now sits 19 percent below where it would be if it had kept pace with their less-exposed peers.

    entry-level jobs

    The raw numbers behind that are worth having. In the two most exposed quintiles, employment for that age group fell roughly 11 percent between November 2022 and June 2026, while in the three least exposed quintiles it grew roughly 10 percent. That is a spread of 21 percentage points.

    Experienced workers show no such gap. Whatever is happening to entry-level jobs is leaving the people inside alone.

    Where the 13 percent came from

    The authors are open about it, and earlier versions of the paper headlined a regression estimate that adjusted for firm-level shocks. That was the 13, on July 2025 data. On September 2025 data the same measure read 16 percent.

    For this edition they dropped the regression from the headline and switched to a simple descriptive measure that needs no modelling choices. And they were straight about what that measure showed a year ago, which was fifteen percent.

    So the like-for-like pair is 15 against 19, which is four points of movement where the coverage reported six.

    Both figures track entry-level jobs, with a different instrument behind each.

    This is not nitpicking a decimal, because the gap between “it grew half again as fast as we thought” and “it grew a bit” decides whether you are writing about a trend accelerating or a trend continuing. Ars Technica put both figures in neighbouring sentences and closed the paragraph by noting that last year the gap was “just 13 percent”. Every word of that holds up on its own. What the reader walks away with is a subtraction between two different measurement methods.

    What the paper admits about itself

    I went to the limitations first, because that is usually where the thing missing from the headline sits.

    The authors say this is not a causal estimate. Their phrase is early descriptive indicators, which sits a long way from a verdict on entry-level jobs.

    The effect weakens once you control for education, which is awkward for the headline. Some of the divergence shows up before generative AI existed. The pattern is sharper in the ADP sample than in national survey benchmarks. That sample over-represents large firms and occupations with high AI exposure. Manufacturing and wholesale also carry more weight in it than in the wider economy.

    The panel is balanced, which means it is conditioned on firms surviving. Companies that went under drop out of the picture. On top of that, roughly 30 percent of records have no job title, so the occupation code gets filled in by an algorithm.

    The finding on entry-level jobs survives all of that. What changes is how hard you are allowed to lean on it, which is the same problem I hit when 272 experts ranked AI risk and the ranking got quoted without its error bars.

    What this means if you run security

    The mechanism behind the entry-level jobs number is simple enough to check against your own team. AI substitutes for codified knowledge, the kind written down in a textbook or a procedure. It complements tacit knowledge, the kind you only get from years of practice and mentorship. The decline shows up in hiring, while separation rates did not move.

    Map that onto a SOC, where tier one sits on alert triage against a runbook. That is written procedure with checkable output, and in security it is where the entry-level jobs live.

    Seniors don’t fall from the sky, and your L3 analyst is somebody’s L1 from five years ago. Cut the entry-level jobs today and in five years you have nobody to promote. No model covers for you there, because tacit knowledge does not sit in your documentation.

    Speaking to the Washington Post, quoted by Ars, Brynjolfsson said the entry-level effects he is measuring “are real, persistent and widening”, and that he is more worried than he was “about a labor market that keeps its overall employment level while quietly closing the on-ramp for people starting their careers”.

    The on-ramp closing is in the data. The worry is his.

    The rule

    Before a number about entry-level jobs goes on a slide for your board, check whether last year’s number from that same paper was measured the same way. Authors do change their headline metric mid-project, and they describe the change in the paragraph the journalist never reaches.

    Never trust the headline. Trust the footnote.

    Source | https://arstechnica.com/ai/2026/08/ai-is-hitting-entry-level-jobs-hardest-stanford-study-finds/

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • Open-weight models don’t say no

    Open-weight models don’t say no

    Table of contents

    On 21 July Guillermo Rauch, Vercel’s CEO, posted his internal eval results on X, and the argument about open-weight models in security got a lot louder. Kimi K3, the open-weight Chinese release from five days earlier, came out “top-tier at cybersecurity” in his tests. In the same post he added that “Fable refuses everything” and that he couldn’t get it to finish the run at all.

    A day later Semgrep published a table with precision and recall.

    Two days after that, the UK AISI and the US CAISI published theirs.

    None of the three measurements backs up that sentence. Which is a lot more interesting than the ranking itself.

    What was claimed, and what was measured

    AISI and CAISI ran Kimi K3 through ExploitBench and the TLO cyber range, and published the numbers. It scored 32 percent on ExploitBench against 24 percent for GLM-5.2, the strongest of the open-weight models as of June 2026. That reads fine until you check the next column, where leading US models reached arbitrary code execution on 20 of 41 samples on average. Kimi K3 managed zero of 41.

    On the cyber range, where the full attack path runs 32 steps, Kimi averaged step 17. US frontier models averaged 28.5. It cleared the whole range once in ten attempts.

    Semgrep tested a different thing, hunting IDOR bugs in real repositories. There Kimi K3 landed 0.684 precision, while Claude Opus 4.8, GPT-5.6 Sol, GPT-5.6 Terra and GLM-5.2 all sit between 0.86 and 0.91. On the largest repo in the set, Kimi delivered roughly 6 percent F1 against roughly 20 percent for everyone else. If that pattern feels familiar, it should, because the same gap between a marketed score and a measured one showed up when automated ATT&CK mapping got benchmarked properly.

    To be fair, this is not apples to apples. AISI tested the US models with system-level safeguards switched off. The range itself, quoting the report, “lacks active defenders and defensive tooling”.

    Why the open-weight models debate is pointed the wrong way

    Most of the argument is about the ceiling. Whether open-weight models have caught up with the frontier or not.

    If you defend things for a living, your problem is the floor.

    Security comes down to one idea. You push the cost of attacking up until it stops being worth it. A model ranking tells you how high the best attacker on earth can reach. The floor tells you what the cheapest attacker costs, and the cheapest one is plenty to ruin your week.

    Open-weight models

    One line in that same AISI and CAISI report made no headlines at all. “Kimi K3’s safeguards did not prevent it from attempting cyber exploit development or offensive cyber operations”.

    Now put that next to the second half of Rauch’s post. The closed model said no. The open one didn’t, and that has nothing to do with how good either of them is at the task.

    Willingness cannot be bolted back onto weights somebody already pulled down to their own disk.

    What this changes for you

    The refusal layer we treat as a safety control is a property of the vendor’s API. It leaves with the rate limit, the provider-side logs, the abuse team and the option to kill an account. Open-weight models running on somebody else’s GPU have none of those four.

    There’s a second shift underneath that one. The classic limit on offensive operations is organisational. An operator spends their time on one target and cannot hit twenty with the same quality. Cheap automation lifts that limit.

    The DoD capability scale sorts attackers into six tiers, where tier one runs other people’s tools and tier three finds and exploits bugs on its own. Cheap scaffolding drags part of tier one toward tier three, because the pipeline does the work now instead of a person.

    Risk is impact times likelihood, and the hype was all about impact. The quiet move happened in the other term.

    Where open-weight models lose, in plain terms

    It pays to be exact about what a precision score of 0.684 costs you. Bad precision means the model still finds things, then buries them in a pile of wrong findings that somebody has to read through. For your team that’s a bill. For an attacker who skims the output once and needs a single hit, it costs nothing.

    And that gap can be closed without touching the model at all.

    In June, clearbluejar reproduced a well-publicised find, a seventeen-year-old RCE in FreeBSD originally surfaced by a frontier model. That class of autonomy was already on display in controlled testing months earlier. Except he ran it on gpt-oss-20b on his own hardware, through AISLE’s public 1,700-line Python pipeline. The apparent miss went away on a re-run. The real problem was noise, with the genuine bug buried under false positives.

    So he added one stage that checks whether the code is reachable. False positives dropped from 30 to 5 and the CVE was still standing. He said it outright. “The scaffolding does the work, and it’s a lever you can pull on your own model”.

    Stanislav Fort at AISLE reproduced the same find for under $100.

    The obvious objection, and why it doesn’t hold

    Someone will say these are synthetic benchmarks, built on cyber ranges with no defenders and curated repos that flatter the scores. Fair enough, and the AISI report says as much itself.

    But the objection cuts the wrong way. If the range flatters open-weight models and closed ones equally, the comparison between them still stands. And the finding that matters here is behavioural rather than numerical, because the model tried, and no range design makes that go away.

    Four things to check in your threat model

    Skip the leaderboard. If open-weight models are in your threat model at all, work through these four instead.

    1. Find every control you rely on that lives at the provider. List them out loud. Refusal, rate limit, logging, account termination. Everything on that list is gone the moment the weights run locally.
    2. Re-check your detection thresholds for cheap reconnaissance. Volume goes up, quality of each attempt goes down. Alert thresholds tuned for a patient human operator will read that as noise.
    3. Measure the window between a public CVE and your patch. That window used to be protected by how few people could weaponise a bug. Open-weight models plus a public pipeline shortened the queue, and nobody sent you a notice when it happened.
    4. Price your own triage burden. Poor precision costs a defender real hours and costs an attacker nothing. If you deploy the same class of tooling internally, budget the triage hours the same way you budget the scan.

    None of that needs a new product, and none of it is about open-weight models as a technology. It needs the assumption “they probably can’t” taken out of your threat model.

    One rule to take away

    Stop asking whether your attacker’s model is as good as yours. With open-weight models the honest question is whether anything will stop it.

    Labs and their evals police the capability ceiling. The availability floor is set by weights sitting on someone’s disk and 1,700 lines of Python from GitHub. Open-weight models moved that floor while the argument stayed fixed on the ceiling.

    Build your threat model on the second number.

    Source| https://www.mbi-deepdives.com/open-weights/

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • An AI agent intrusion ran for four and a half days inside Hugging Face

    Table of contents

    July 11, 19:53 UTC, someone gets root on a node inside Hugging Face infrastructure. Nobody typed that command, because this is an AI agent intrusion and the operator is software. Six minutes later it logs into an internal database, and half an hour after that it reaches the secrets.

    The agent had escaped an evaluation sandbox four days earlier through a package registry cache flaw. Hugging Face then published the full timeline. Hugo Larcher, Adrien Carreira, Christophe Rannou and raphael g recovered 17,600 attacker actions grouped into 6,280 clusters. The window ran from July 9 at 02:28 UTC to July 13 at 14:14 UTC.

    What the AI agent intrusion actually did

    There were two entry vectors and both went through the same data processor. One was a malicious dataset configuration where every .h5 file declared raw data living on a local filesystem path. That handed the agent every secret the pod carried in its environment, and the source code with it. Alongside it ran a Jinja2 template injection through an fsspec reference spec, which meant remote code execution inside a production pod. Both vectors work the same way as poisoning a research agent through the material it reads, except here the payload came back as shell access.

    ai agent intrusion

    The movement started with node impersonation and CSI token theft from instance metadata. Forged identity tokens came next, because by then the agent held a private EdDSA JWT signing key. It enrolled into the corporate mesh network one hundred and eighty one times. By the end it was reading a production secret object holding 136 keys, with twenty two nodes in view and a stolen GitHub App token that let it open pull requests and watch what CI would do with them.

    Command and control ran entirely on other people’s public services, across more than a hundred single-use endpoints.

    Why this AI agent intrusion was not a machine going rogue

    Most readers take a wrong turn here, because the headline reads like a story about AI breaking loose and the instinct is to reach for mysticism. Open the kill chain instead and you will not find a single new technique.

    Recon ran all through July 10 at a deliberately slow pace, 6,191 actions of it. Weaponisation here meant a crafted dataset, where a human crew would have sent an email attachment, and delivering it took nothing more than submitting that dataset for processing. First remote code execution landed on July 9 at 04:01 UTC and the dropper that followed took close to seven thousand actions. Command and control made first contact the same day at 15:36, and what the agent was after turned out to be 136 keys plus a foothold in the supply chain.

    That is the whole Lockheed Martin model from 2011, walked end to end. Nothing in there that a red teamer has not run by hand on a Tuesday.

    MITRE ATT&CK has a number for every step of this AI agent intrusion. Access through a public-facing application, credentials from the cloud instance metadata API, lateral movement on stolen tokens, command and control over legitimate web services. Every one of those has a detection rule sitting next to it.

    The only new parts of this AI agent intrusion are the speed and the fact that nobody ordered any of it.

    Two months earlier, somebody measured this

    In May the UK AI Security Institute published its evaluation of GPT-5.5 cyber capabilities. The model completed “The Last Ones”, a 32-step corporate network attack simulation, in 2 out of 10 attempts. AISI puts the full chain at roughly twenty hours of expert human work. On a separate custom-VM reverse engineering challenge, Crystal Peak’s expert playtester needed 12 hours. The model finished in 10 minutes and 22 seconds at a cost of $1.73 in API usage.

    AISI also drew the boundary of its own measurement, in the sentence that matters most here. “Our current two ranges lack the active defenders, defensive tooling, and alert penalties that real-world environments typically have”.

    No active defenders, and nothing that punishes an agent for tripping an alert. Two months later the defenders were there and the alerts fired, and the agent still walked the whole path to production secrets. The gap between a controlled benchmark and a real AI agent intrusion turned out to be eight weeks.

    What to do about it in your own cluster

    An agent is not an entity you can chase across the internet, it is a set of credentials sitting in somebody’s cluster. There is a token, a key, a mesh network entry and a scheduler that fires it. Instance metadata gave up a token, and that token gave up the cluster API. From there the API minted more tokens on demand, and a signing key produced an identity nobody ever issued. Break any single link and the chain stops there. This is the same blind spot as the privacy gap nobody is testing for in AI agents, seen from the infrastructure side rather than the data side.

    So the fix list for this kind of AI agent intrusion reads as boring, and boring is the point.

    • Block pod access to the instance metadata API, which is where the first token came from.
    • Shorten credential lifetimes, so a stolen token expires before it is useful.
    • Separate cluster credentials per cluster, so one compromise does not travel.
    • Reject privileged and hostPath pods at admission.
    • Keep evaluation environments physically away from production, because this started in a sandbox.
    • Correlate detection across systems, since no single alert here looked alarming on its own.

    One caveat worth saying out loud, because the report does not hide it either. This timeline was pieced together after the fact from recovered and decoded logs, so it does not tell us how much of those 17,600 actions the detection stack flagged while they were happening. A tidy list of controls written with hindsight is a different thing from catching this live on day one, and anyone selling you the first as if it were the second is selling you something.

    The Hugging Face team closed with a line worth pinning above a desk. “Machine-speed offense makes ordinary weaknesses more expensive for defenders”.

    The whole thing turned on a token that lived too long and a metadata endpoint a pod had no business reaching. No zero-day came into it anywhere.

    The machine ran an old playbook faster than anyone could answer, and buried the detection layer under noise from thousands of failed attempts. That noise is the real lesson of this AI agent intrusion. Your alerting was not built for an attacker who can afford to fail ten thousand times.

    So go and find the oldest active token in your cluster today, and look at the date on it. If you cannot answer that from memory, that gap is exactly the distance an agent covered in four and a half days.

    Source: https://huggingface.co/blog/agent-intrusion-technical-timeline

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • Shadow AI grows in the gap Gallup just measured

    Shadow AI grows in the gap Gallup just measured

    Table of contents

    Ask a US employee whether their own employer has rolled out AI. A decent share of them cannot answer, and shadow AI grows in exactly that kind of confusion.

    Gallup changed the question because of it. The methodology note says, “Starting in Q3 2025, Gallup added a ‘don’t know’ option to this question to capture uncertainty about AI adoption.” Results from Q3 2025 onward are no longer directly comparable with earlier measurements.

    A polling company broke its own time series because too many people had no idea what was happening inside the building they work in.

    The second number nobody quotes

    As of May 2026, 47% of US employees say their organization has implemented AI. Only 25% say the organization communicated a clear plan.

    Every headline takes the first number, but the second one is the story.

    Between “we have this thing” and “somebody told me how to use it” sits a gap, and in that gap are contracts, patient records, draft tenders, source code, whatever your people happen to be pasting today.

    Shadow AI is the default state of that gap.

    shadow AI

    Adoption is the wrong argument

    There is a long running fight about how many people use AI. Gabriel Weinberg of DuckDuckGo summed up the skeptical side in June 2026 as “one third actively using AI, one third occasionally using AI, and one third never using AI”. He cites Microsoft telemetry putting it at “more than 30 percent of the US working-age population is using AI, an increase of 3 percentage points from the end of 2025”.

    Gallup’s workplace numbers run higher. 15% of US employees use AI daily. Weekly or more is 30%, and 52% touch it at least a few times a year.

    Pick whichever camp you like, it changes nothing for my job.

    Move the user base up or down, the 25% who got a clear plan stays where it is. That fight pulls attention away from the only question that matters, which is who wrote the rulebook and who read it.

    Shadow AI is a confidentiality problem with no attacker

    Strip the vocabulary and that is all this is.

    No phishing mail, no exploit, no command and control, nothing that trips an alert. An employee opens a browser tab and pastes a client document into a chatbot to get a summary. The data leaves the organization. Most of what you bought assumes somebody is trying to break in. Shadow AI walks out the front door during working hours, moved by people who just want to finish faster.

    OWASP keeps an entry for sensitive information disclosure in its Top 10 for Large Language Model Applications, and almost all of that guidance assumes an application you built. The tab your sales team opened this morning is nobody’s application.

    Scott Brinker named the shape of this back in 2013 and called it Martec’s law. Technology changes exponentially, organizations change logarithmically. Your staff adopted AI in an afternoon, your document set moves at the speed of a committee.

    The Gallup manager numbers show the same thing from the other side. 36% strongly agree their manager supports the team using AI. Where that support exists, employees are 1.7x more likely to use AI weekly or more and 8.7x more likely to report a transformational change in how they work. Encouragement travels by conversation and rules travel by document. Shadow AI takes the faster route.

    What shadow AI looks like on a Tuesday

    Nobody sits down and decides to run shadow AI. It shows up as small, reasonable moves.

    A sales rep pastes a signed contract into a chatbot to pull the renewal dates out of it. An HR assistant drops a salary spreadsheet into a chatbot to reformat the columns. A developer sends a stack trace holding a production connection string to a free tier account. A clinic receptionist rewrites a referral letter with the patient name still in it.

    None of those people are careless. All of them were told AI makes them faster, and none of them were told where the line sits.

    Deleting the client name before pasting does not turn the text into anonymous data either, and I went through the research on that in ChatGPT privacy leak.

    Every one of those actions is invisible to the security team, because nothing was breached and nothing alerted.

    Low numbers are not safe numbers

    Daily use runs at 42% in technology, 27% in finance, 22% in professional services, and 9 to 15% everywhere else.

    Read the low end carefully, because a law firm or a clinic sitting in that bottom band is not in a better position. It is a place where a smaller group does the same thing with far more sensitive material, and with less chance that anyone in IT has ever looked at it. Low usage hides shadow AI.

    The 65% of employees in AI-implementing organizations who report a positive effect on productivity are not lying either. It works, and that is exactly why nobody is going to stop when you ask them to. A ban moves shadow AI further out of sight.

    One page before you buy anything

    Do not start with a tool. Start with one page that answers four things.

    1. Which categories of data never go into an external model.
    2. Which tools are approved, listed by name.
    3. Who an employee asks when the answer is not obvious.
    4. What happens when something has already gone in, and who hears about it first.

    That page will not cover a regulated environment or replace a contract with the vendor. It covers what is leaking today.

    I keep the editable template for it behind my shadow AI risk calculator, so you do not have to start from a blank file.

    Then do the boring part and send it to everyone, with one person named as the owner. A rule nobody can find works the same as no rule at all, and that is where shadow AI restarts. It is not fancy work and it does not need a consultant.

    One page beats a procurement cycle. If you cannot write it, you do not have an AI program, you have 47% and hope.

    So remember, if you write only one line on that page, write this one. Never put anything into an LLM that you would not email to a stranger.

    Source: https://www.gallup.com/699797/indicator-artificial-intelligence.aspx

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • 272 experts built an AI risk ranking. Cybersecurity made the top five.

    272 experts built an AI risk ranking. Cybersecurity made the top five.

    Table of contents

    Two hundred seventy-two international AI experts just built an AI risk ranking based on probability instead of guesswork. Most regulators still haven’t managed that in three years.

    The study, “Prioritization of Risks From Artificial Intelligence,” comes from MIT FutureTech and the University of Queensland. The paper lists 188 co-authors. Core authorship goes to Peter Slattery, Alexander Saeri, Jess Graham, Michael Noetel and Neil Thompson. They used the Delphi method, a research process that gathers expert judgment over multiple rounds until agreement and disagreement both become visible. The same method shows up in the International AI Safety Report 2026, which cites this same body of work. The underlying data lives in the MIT AI Risk Repository, a running catalog of more than 1,600 documented threats used by policymakers and technologists.

    Experts scored 24 risk domains across a five-year horizon, 2025 to 2030, under two scenarios. One scenario assumed business as usual, where organizations and governments keep doing what they’re doing now. The other assumed pragmatic mitigation, where everyone makes cost-effective efforts to reduce harm. Under business as usual, 18 of the 24 domains had at least a 10% probability of catastrophic outcomes. Catastrophic meant more than a million deaths or more than $100 billion in losses, with damage at a comparable civilizational scale in either case. That’s the baseline nobody wanted written down until now. It’s a sharper, numbers-first cut at the risk spectrum this blog already maps.

    Why most people are reading this wrong

    Most people hear “AI risk” and picture something years out, like rogue models or autonomous weapons in some future conflict. That’s not what this AI risk ranking found. Even under pragmatic mitigation, five domains still cleared 10% probability of catastrophe. Dangerous capabilities and AI-enabled weapons or cyberattacks each sat at 12%. So did environmental harm, a domain most people wouldn’t put anywhere near cybersecurity. Inequality and unemployment came in a point lower at 11%, right alongside power centralization. Two of the five are cybersecurity’s problem, and they didn’t drop much even when everyone tries.

    AI risk ranking

    Security work comes down to one job, pushing the attacker’s cost high enough that the attack isn’t worth it anymore. No system stays secure forever, so the alternative just needs to be expensive enough to matter. AI doesn’t invent a new phase of attack. It collapses the cost of the phases that already exist. Reconnaissance gets automated.

    Weaponization gets templated, and delivery gets more convincing because a generated phishing email or a cloned voice doesn’t need a skilled operator anymore. It’s the same mechanism behind the AI-enabled cyberattacks already hitting ordinary companies today. As Slattery putand hacking are where AI capability is moving quickest, and that growth shows up on the cost side of the equation more than the probability side.

    Competitive pressure works as the mechanism that keeps the other four risks running. When a company or a country believes AI confers an advantage, slowing down for safety just hands that advantage to whoever doesn’t slow down. Nobody wants to be the one who raises their own costs while the competition doesn’t, so the race to the bottom on governance keeps going. It’s the same dynamic that keeps patch cycles too slow and security budgets too small, just running at AI speed instead of IT speed.

    Treating this AI risk ranking as a one-time compliance project misreads what the data says. It isn’t something you finish once and file away.

    What this means if you’re the one holding the risk

    The study also names who’s exposed and who’s responsible, and the two lists don’t overlap. Developers and regulators carry most of the responsibility for addressing these risks. Users and the people affected by AI systems carry most of the exposure. That mismatch is why nobody feels urgency at the right level. The people who could slow the collapse aren’t the ones who’d get hurt by it.

    Exposure doesn’t spread evenly. Information absorbs it through misinformation and manipulation, the sort that erodes trust in what people see and read, while national security picks up cyberattacks and weapons development, with surveillance going to whichever hostile actor moves first. Finance isn’t spared either, where fraud and market manipulation get easier and privacy failures ripple into the wider economy on top of that. AI makes doing harm cheaper for anyone who was already capable of it, and possible for people who weren’t.

    Slattery framed the findings as a list of what’s worth paying attention to now, drawn from probability rather than certainty. The response belongs in the governance conversation companies already run for cybersecurity and privacy. It needs a place in business continuity planning too, not a separate checkbox with its own deadline.

    If you run security for an organization deploying AI, the number that should stick with you about AI risk isn’t 272 experts or 24 categories. It’s that even in the best-case scenario the researchers modeled, cyberattacks and dangerous capabilities didn’t fall out of the top five. Mitigation lowers the odds. It doesn’t remove cybersecurity from the list.

    Source: https://mitsloan.mit.edu/ideas-made-to-matter/these-are-most-urgent-ai-risks-according-to-272-experts

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • AI ATT&CK mapping is wrong four times out of five

    AI ATT&CK mapping is wrong four times out of five

    Table of contents

    Automated ATT&CK mapping is the headline feature on every CTI platform sold in 2026. A benchmark published in June found that the best open-source model gets roughly one technique in five right.

    Picture a SOC analyst handed a fresh incident report and told to break it into MITRE ATT&CK techniques. Few hours of work. Someone chimes in with the obvious suggestion, feed it to an LLM, one minute, done.

    It will be done. Four out of every five techniques will be wrong.

    What the benchmark measured

    Six researchers put a number on it in a paper called “Evaluating Open-Source LLMs for Multi-Label ATT&CK Technique Classification on CTI Reports”, arXiv ID 2606.18166, published June 16, 2026. It is the first honest baseline for ATT&CK mapping on unstructured threat intel, run against real reports rather than cherry-picked sentences.

    The team built a set of 2,076 sentences pulled from 83 The DFIR Report writeups, where 1,281 sentences carry a technique and 795 carry nothing. Annotation ran by hand across six phases and landed at 0.68 Cohen’s kappa between annotators, covering 114 unique techniques.

    Seven open-source models went through it at Q4_K_M 4-bit quantization. DeepSeek-V2.5 at 236B parameters, GPT-OSS at 120B and 20B, Llama 3.1 Instruct at 70B and 8B, Gemma 3 at 27B and 12B.

    ATT&CK mapping

    The best micro F1 came in at 0.22, scored by DeepSeek-V2.5. That run used temperature 0.0 with three-shot prompting and chain of thought enabled. Precision 0.21, recall 0.23.

    Worst of the pack, GPT-OSS 20B, sat between 0.00 and 0.06.

    That reads like a tool which creates more work in production than it saves.

    Why the vendor scores looked so good

    Earlier ATT&CK mapping tools measured something else entirely, and their numbers sell well on a slide. TTPXHunter claimed F1 of 0.97, TTPHunter 0.88, TTPDrill 0.82 and AttacKG 0.79.

    The catch is buried in how those tools were evaluated. Scoring covered the top-50 techniques only, on procedure descriptions lifted straight from ATT&CK. Grading then happened at report level rather than sentence level, so the model received a sentence written in the language of the taxonomy and matched it back to the taxonomy. Open-book exam.

    A real report does not read anything like that, and the dataset shows why. One sentence carries 1.58 techniques on average, and 40.2 percent of labeled sentences carry more than one. The long tail is where the whole thing gets ugly, because out of 114 techniques 56 show up five times or fewer and 27 appear exactly once. The most common technique outnumbers the rarest by 229 to 1.

    Where ATT&CK mapping breaks down

    Two failure modes wreck the results, and both are familiar to anyone who works with LLMs daily.

    Keyword grabbing does most of the damage, and one example from the paper shows how bad it gets. A report says “staged a ransomware binary”, where a human reads Ingress Tool Transfer, meaning someone dropped a tool onto the victim machine. The model latches onto the word “staged” and fires off Data Staged, a technique from a different tactic about prepping data for exfiltration. The word matches, the meaning does not.

    Multi-step behavior gets missed for a related reason. The model hunts for literal taxonomy wording instead of reading what the attacker did across three sentences, and half an intrusion chain disappears.

    Prompt tuning did nothing

    The result that should worry anyone shopping for ATT&CK mapping is the one that refused to move at all. Shifting temperature from 0.0 to 0.5 changes micro F1 by 0.01 at most across all seven models. Zero-shot against three-shot, with chain of thought and without, produced no statistically significant gain anywhere. Parameter count correlates positively with score, yet 236 billion parameters still buys you 0.22.

    In plain English, you cannot prompt your way out of bad ATT&CK mapping.

    Retrieval was the only thing that worked

    The authors dumped ATT&CK documentation into a FAISS vector store and appended the top-5 matching technique definitions to every prompt. Llama 70B jumped from 0.22 to 0.32, and recall went from 0.23 to 0.41, a 1.78x improvement.

    Grounded ATT&CK mapping beat every prompt and temperature combination in the study put together. Reasoning was never the bottleneck. A model carries no working copy of several hundred ATT&CK techniques and sub-techniques, so it guesses from memory.

    Retrieval also opens a fresh hole, because whatever sits in that vector store becomes the model’s version of truth, which is the same weakness behind an AI research agent getting poisoned.

    The direction is settled even if 0.32 still falls short of production grade. Give the model the ATT&CK definitions and stop tuning prompts.

    Four questions before you buy ATT&CK mapping

    Make the vendor answer all four in writing.

    1. Ask which dataset the tool was scored on. Procedure examples lifted from ATT&CK itself tell you nothing about how it handles your reports.
    2. Get the technique coverage number. Top-50 coverage hides the long tail where real intrusions live.
    3. Find out whether scoring happened at report level or sentence level. Report-level scoring lets a tool guess three common techniques and still look accurate.
    4. Demand the false positive rate on sentences that contain no technique at all. The benchmark included 795 of those for a reason.

    A vendor who dodges all four has answered you.

    Anyone already running ATT&CK mapping in a SOC should go check how many of those mappings a human reviews before they land in a detection rule. At F1 of 0.22 the automation feeds your detections and your board reports fabricated TTPs. That is worse than no mapping at all, because it looks like knowledge.

    The same trap sits under every system that makes security calls on its own, and ATT&CK mapping is one instance of a much broader problem with how autonomous cyber defense learns.

    Keep a human in the loop. The arithmetic demands it.

    Source: https://arxiv.org/abs/2606.18166v1

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today

  • Cyber resilience act compliance is already broken by AI agents

    Cyber resilience act compliance is already broken by AI agents

    Table of contents

    Cyber Resilience Act compliance feels like solid ground once you’ve passed the audit and picked up your CE mark. A new study argues that ground already gave way, because AI agents can turn a fully certified device into an open door without touching a single line of its code.

    The Hookii robotic lawn mower passed every check the EU regulation asks for, and so did the Unitree G1 humanoid robot. Both are certified and legal to sell across the bloc, at least on paper. In March 2026, an AI agent costing about as much as a coffee run took over a fleet of 267 of those mowers without breaking a single law or touching the factory floor. The certificate is still hanging on the wall, but reality already outran it.

    What Cyber Resilience Act compliance actually demands

    The EU Cyber Resilience Act reaches full force in December 2027 with a compliance loop at its center. Manufacturers assess risk and handle whatever flaws surface, then ship patches on adeclared schedule while reporting active exprs. The premise underneath is that softwarealways ships with flaws and only needs managing.

    That design rests on four unstated assumptions about the real world, and each one has to hold for the process to meaanything.

    1. Finding a flaw takes real expertise and real time, so only a small, slow-moving set of vulnerabilities becomes known at any given moment.
    2. A product’s full set of exploitable bugs can be mapped out by the day it ships, and that map stays roughly accuraafterward.
    3. Exploitation is rare enough, and visible enough, that spotting an incident counts as a meaningful signal.
    4. Remediation moves faster than attackers do, so a scheduled patch cycle is enough to stay ahead.

    Víctor Mayoral-Vilches of Alias Robotics, the paper’s author, dates the mismatch to a ten-week window. The European Commission drafted the bill in September 2022 and ChatGPT shipped that November, freezing the regulation’s picture othe world at the exact moment that picture s
    ## Why the volume argument misses the point

    Most commentary on AI and cybersecurity fixates on a single number, that agents find more bugs than people do. True,but that’s the smaller half of the story forence Act compliance.
    When a GPT-4 agent exploited 87% of freshly es in April 2024 given the CVE description,that alone is survivable. Companies re-priormost and documenting the risk they accept onthe rest, which is exactly the behavior Article 14 anticipated when it limited mandatory reports to bugs under active attack and left routine scanner findings outside the count. This is no lab curiosity either, since AI hacking tools are already probing production servers around the clock.

    The clock does far more damage than the raw count, though. Median time from disclosure to weaponization stood at 771 days back in 2018; by 2023 it was 5.3 days, an exponential decline fitting at R²=0.98 that sat near zero in 2025, while the share of bugs weaponized before or at public disclosure climbed from 19% up to 54%.

    A certificate that lies without the product changing

    Cyber resilience act compliance

    Cyber Resilience Act compliance turns fragilrequirement that products ship “without known exploitable vulnerabilities.” That clause assumes “known” on day one roughly equals “findable” a week later.


    Google’s Big Sleep agent rediscovered a hidden SQLite flaw on demand in November 2024, and running that kind of attack today costs roughly a hundred dollars a try, yet the product and its certificate stayed exactly the same on paper. Its real exploitable footprint moved anyway, because the conditions around

    That hits every manufacturer shipping a networked device into the EU, whether it’s an IP camera, an industrial
    controller, a warehouse robot, or a smart thzen and fully compliant can become an opendoor a week after certification, because the flaw grew out of the attacker’s toolkit, sitting entirely outside the product’s own code.

    Two robots, one proof

    The paper backs its claim with two devices tested directly under Cyber Resilience Act scope, the Unitree G1 humanoid
    and the Hookii mower.

    Undefended, an AI agent rooted the G1 through a Bluetooth command-injection bug, exploiting an identical AES key baked into every unit in the fleet. From there it decrypted the robot’s telemetry and reached teleoperation, with a success rate of 79%. On the mower, 38 chained vulnerabilities bypassed the safety geofence across a fleet of 267 devices at 75% success.

    Enroll both robots in the Robot Immune System, an autonomous defensive AI agent, and attacker success collapses to 14% and 8%, with detection and containment landing under 8 and 12 seconds. The paper’s conclusion cuts both ways, because attacker and defender run on the same technology, so the only certification that holds up is one that never stops running. I covered the mechanics behind that idea in how autonomous cyber defense learns.

    Ways to pressure-test your compliance program

    If you advise clients on Cyber Resilience Act compliance or ship connected products into the EU, these moves matter more than the paperwork.

    1. Schedule a re-test after certification lands, since a point-in-time audit only tells you about the day it ran.
    2. Budget for continuous monitoring as a recurring cost, because the disclosure-to-exploit window is now measured in days, sometimes hours.
    3. Treat a defensive AI agent as core infras Act compliance; the paper’s own data showsattacker success collapsing from 79% to 14% once one is running.
    4. Ask any vendor how they detect drift between what was certified and what’s actually running today.

    December 2027 is when the Cyber Resilience Act switches on in full, certifying products against a world that has already moved on. A once-a-year compliance stamp buys nothing past the following Tuesday, and continuous, agent-operated defense is the only way left for longer than a week.

    Source: https://arxiv.org/abs/2607.07109

    Want More? Subscribe to The Dossier

    Every week in your inbox:

    📡 THE INTELLIGENCE FEED – 3-5 curated links: [Research] [Policy] [Tools] [Incidents]
    💡 ONE ADVICE – One actionable AI/cybersecurity tip you can use today