Case studies · 13 October 2026 · updated 17 October 2026 · 7 min read
A polished report is not a checked report
I let my own OSINT engine write a brief on an arrest, then checked it by hand. The arrest was real, but much around it did not hold up. This is what the human step found.
I built an automated OSINT engine called FORGE. It collects open-source news signals about South Africa, scores them, groups them into cases, links the people and bodies involved and writes short analyses. For one case I let it run end to end. It collected the signal, handled it and wrote the brief, and I checked none of it while it ran.
Then I did the part the machine does not do. I went back to the original reporting and checked the brief, claim by claim.
The brief, as it stood on 13 October 2026, is about an arrest in April 2026. It looks finished: a headline, a date, a score, named actors and a forecast. This post is about the gap between that finish and what the sources say. It audits one report. It does not measure how often FORGE is wrong, and it does not explain why this report went wrong.
Correction, 17 October 2026
The table below says I found no source for a Polokwane tender probe in September 2025. That was wrong. On 17 September 2025 the Hawks visited Polokwane municipality’s offices with a court order to obtain documents for a corruption investigation (Sowetan). Nothing I found links it to the April arrests, so the brief’s claim stays unsupported. What I missed, and why, is in Checking the check.
What the machine did on its own
- It picked up a news item about the arrest and gave it a score it calls a gravity score.
- It filed the item in a case it had named for itself, and gave the case an evidence weight.
- It linked three actors to the case: a police unit, “the municipality” and the arrested man.
- It wrote an analysis, tied the arrest to other events and forecast more arrests.
What the human step found
I checked the brief against the reporting of the 18 April arrest by TimesLIVE, Eyewitness News and IOL. For the other arrests the brief mentions, I used TimesLIVE on the four officials, Eyewitness News on the Molemole arrests and Eyewitness News on the former legal manager.
| The brief said | The reporting says | Finding |
|---|---|---|
| The arrest happened on 18 April, by a SAPS task team, on charges of fraud and corruption | Same date and unit. Police also gave a third charge: defeating or obstructing the ends of justice | Partly supported |
| It was part of a multi-target HAWKS operation against Limpopo’s municipal procurement network | The arrest was at a home in Gauteng and police tied it to an investigation into corruption inside the Ekurhuleni Metro Police. The reports describe a SAPS task team, not a Hawks operation | Contradicted |
| The task team is a specialist unit against procurement fraud in Limpopo | The unit exists. It was set up to follow up evidence from a commission of inquiry’s interim report | Unit exists, purpose and place wrong |
| It was the third enforcement action in Limpopo’s municipalities within 72 hours | The Hawks did arrest municipal officials in Limpopo that week, twice. The 18 April arrest was in Gauteng | Partly supported |
| Four other officials were arrested on 16 April in a related HAWKS operation | Four senior Molemole Municipality officials were arrested by the Hawks on Wednesday 15 April, in an investigation the Hawks spokesperson said began with incidents reported in September 2024. No report I found links it to the 18 April arrest | Real arrests, wrong date, link unsupported |
| The case may follow a Polokwane tender probe of September 2025 | I first wrote that I found no source for it. That was wrong: the Hawks visited Polokwane municipality on 17 September 2025 (see the correction above). Nothing I found links that matter to the 18 April arrest. The Molemole case traces to September 2024, and its arrests were made in Polokwane | The probe is real; the link is unsupported |
| The charges fit procurement conduct, not operational misconduct | Police tied the charges to an investigation into corruption inside the metro police | Unsupported |
| Further arrests are expected | Police said the task team was still searching for other suspects | Weakly supported, on premises that are wrong |
The brief says it recorded the arrest from a TimesLIVE item. TimesLIVE’s own report of the arrest names the Ekurhuleni Metro Police and a home in Gauteng, and does not mention Limpopo. I cannot tell from the page whether FORGE used that exact article.
Real events, wrong links
The most interesting finding is the Limpopo arrests. They are real. In the days before the Gauteng arrest, the Hawks arrested four senior officials at one Limpopo municipality and a former legal manager at another. The brief put them on the same map as the Gauteng arrest and called them related. The reports I found describe separate investigations by different bodies, and none of them links the cases.
I cannot tell from outside how FORGE built that link, so I treat this as a finding about this report, not about how the engine works. A claim can be made of true parts and still be false as a whole.
A disclaimer does not fix a claim
FORGE’s page says its assessments are provisional and are not legal findings. That is a fair thing to say. It does not change the sentences above it, which state places, jobs and counts as facts. A reader who sees a caveat at the bottom and a confident claim at the top remembers the claim. If the facts are wrong, the caveat has not helped anyone.
Unsupported is not false, and two numbers share one label
Unsupported is not false. A claim with no source is a gap, not an error. I record it as unsupported and leave it out of anything I publish.
Two numbers, one label. The page gives 0.42 as the gravity score of one item and 0.42 as the evidence weight of the whole case. One weak signal’s score says nothing about how many claims survive checking. It is the tool’s rating, not evidence. The case record also lists “the municipality” as an actor without saying which one.
A name risk. A far better-known police figure shares the arrested man’s surname. A tool that links people by name can merge two people. I found no sign in this report that it did, so it is a risk to check for, not a finding.
The method
- Save a dated copy of the report before you touch anything. Live pages change.
- Write each claim as one line. Split compound sentences.
- Read the original reports in full, not search summaries.
- Mark each claim Supported, Partly supported, Contradicted or Unsupported, with the passage that decided it.
- Keep “no source found” apart from “proven false.”
- Write a finding that stays inside this report.
What this does not show
An arrest is not a finding of guilt. The man in this report faces allegations and charges, and this post says nothing about whether they are true. The case has also developed since April, and this audit covers only what was reported by 18 April and the Limpopo arrests of 15 and 16 April.
Finding errors in one report does not give a system-wide error rate. Nor does finding the corrected facts show what produced the mistakes.
A polished connection can be a useful lead. It becomes a finding only when you can show the steps.
If you want to try the method yourself, Exercise 004: One arrest, wrong map uses an invented scenario with the same kinds of errors.
Related posts
Checking the check: two mistakes in my own work
I audited a machine's report, then rebuilt it with stricter checks. My audit and my rebuild each held a mistake. Here is what they were and how to correct work in public.
Five reports can still be one account
I split 36 FORGE pins into 216 facts. The useful result was not a confidence score, but a clearer account of who said what, and what remains unknown.
Same handle, different person: the trouble with username matches
Username search tools tell you a page exists, not who owns it. Here is how to weigh handle matches, spot false positives and false negatives, and avoid naming the wrong person.