Table of contents
My RSS agent reads my source list for me every morning, a small AI threat intelligence pipeline of my own. It scores each news item, and anything above 7 out of 10 lands in my digest. I built it and use it daily, and until last week I never asked what that 7 means.
Then I read a new paper by Jiahong Qian from Shanghai Jian Qiao University. It tests what happens when an LLM reads cyber threat intelligence reports and has to name the threat actors and vulnerabilities behind each campaign. The test ran on 354 incident campaigns from a public corpus. DeepSeek-V4-Flash and GLM-5.3-Flash were the cheap models, and GPT-6.1-Sol was the flagship control.
The scary result comes from a side experiment where one added sentence pins a fake CVE on the campaign, and the cheap models bind that CVE in 95.6% to 100% of cases with confidence near the top of the scale.
The models find everything, including the noise
Recall looks great at 0.81 to 0.99 across models and entity types, so the models catch almost every entity the analysts marked.
Precision is where it breaks, with micro precision between 0.03 and 0.34 across the board. On average GLM-5.3-Flash emits 15.1 CVE strings per report, while the gold annotation lists at most 2 primary CVEs.
GPT-6.1-Sol scores 0.057 micro precision on CVEs, and in the paper’s words “the gap does not close at the flagship tier.” Paying more buys you the same pile of extra candidates.
So you get a list of fifteen where two matter and nobody tells you which two. If your AI threat intelligence feed works like this, an analyst is still doing the sorting by hand. Earlier this year I wrote that AI ATT&CK mapping is wrong four times out of five on real reports, and this paper finds the same low precision on actors and CVEs.
Hallucination is the wrong suspect here
The reflex answer is hallucination, and the reflex fix is RAG plus chain-of-thought.
The paper checked that by annotating a sample of 360 candidates and projecting it over all the outputs. Most of them, 65.6%, were background mentions, meaning an entity named in the report that this campaign never used. Another 19.9% were relevant to the event but outside what the gold annotation counts. Only 1.2% had no support in the text at all. Among CVE false positives, 99.7% to 99.8% appear explicitly somewhere in the report.
The paper says the models “report strings whose truth they cannot adjudicate against this event”, and that “mentioned” is mistaken for “used”.
That changes the fix for AI threat intelligence, because RAG and chain-of-thought improved actor extraction by up to 0.133 macro-F1 and barely moved vulnerabilities. You can’t retrieve your way out of a reading problem.
Technical attribution matches TTPs and infrastructure against known campaigns, and false flags have always been part of that work. Łukasz Olejnik and Artur Kurasiński describe Turla running operations through Iranian infrastructure in Philosophy of Cybersecurity. That kind of deception used to cost the attacker real effort. If AI threat intelligence does your extraction, a false flag costs one sentence in a report your pipeline reads.
What confidence scores buy you in AI threat intelligence
Verbalized confidence, the model telling you it is 95% sure, is badly calibrated. Expected calibration error sits between 0.53 and 0.94.
Self-consistency works better, where you ask the same question five times and count how often the entity comes back. As a ranking signal it passed the paper’s preregistered bar in all six test cells. On vulnerabilities, reviewing the top 20% by that score locks in 96% of correct items.
In the planted-sentence test the fake entity got mean self-reported confidence of 98.0 and 85.6 on the two models, indistinguishable from the true one. Its self-consistency frequency was 0.892 on both. The paper’s verdict is that “SC frequency measures textual consistency, not fact.”
A planted sentence is consistent text, so the model agrees with it every time you ask. In AI threat intelligence the score measures agreement with the text, and the attacker controls the text.
How to run AI threat intelligence with a human in the loop
Dutch OSINT investigator Nico Dekens wrote in August that intelligence failures rarely arrive with flashing lights, and “usually, they arrive wearing a confidence score.” This paper puts numbers behind his line.
The paper’s deployment advice for an AI threat intelligence pipeline comes down to four changes.
Confidence sorts the review queue and never replaces the review, so for vulnerabilities you start at the top 20%. Threat actors need a different habit, since the top 5-10% of that ranking is enriched with errors and a human has to check actor attributions from the head of the list.
Thresholds get recalibrated per corpus, because on a newer 2022-2025 set of reports the rankings held up while fixed thresholds did not transfer.
Every report that feeds your AI threat intelligence counts as attacker-controlled input, since CTI reports quote vendor blogs and pastes and one planted line rides along with them.
Every extracted entity keeps the sentence it came from next to it, so the reviewer reads the evidence line first.
The planted-sentence batch ran only on the two flash models, so the flagship result there is unknown. Two LLMs from other vendors did the annotation and a human spot-check is still pending. The main corpus ends around 2021.
My RSS agent’s 7 works the same way, since it tells me what to read first and has never once told me what’s true.
Confidence sorts the queue. A human closes the attribution.
Source | https://arxiv.org/abs/2610.20417