When AI Writes AI Policy: The Hallucinated Citations that Reset South Africa’s Draft National AI Policy

South Africa’s Draft National AI Policy was withdrawn this weekend. Good. That is the process doing what it should. And it opens a comparison worth making, because the countries that have built credible AI governance didn’t get there by moving fast. They got there by moving deliberately.

Other jurisdictions that have done AI policy well share three things, and none of them is speed.

Consider the EU AI Act. People look at its 2024 adoption and think it came together quickly. It didn’t. The groundwork started in 2018 when the European Commission stood up a 52-member High-Level Expert Group (HLEG). Every one of those 52 people was named. Their conflicts of interest were declared. That group produced the Ethics Guidelines for Trustworthy AI in 2019. Then came the White Paper on AI in 2020. The formal Commission proposal landed in 2021. From there, three years of trilogue negotiations between Parliament, Council, and Commission before the regulation was finally adopted in June 2024.

Six years. Every claim sourced. Every position traceable to a person willing to stand behind it.

Singapore went a completely different direction but arrived at the same standard of rigour. The IMDA published its Model AI Governance Framework in 2019. Revised it in 2020. Revised it again in 2024, this time specifically for generative AI. Each version was co-built with industry, civil society, and the Personal Data Protection Commission. Not consulted after the fact but co-built. The entire model treats governance as a living system. You ship a version, you learn, you update. No single document is expected to be permanent.

Brazil ran its process through a Senate commission of jurists who held months of public hearings before a single clause of PL 2338 was drafted. The AU’s Continental AI Strategy went through multi-year inter-ministerial consultation across all 55 member states before adoption in 2024.

The common thread across all four? Long lead times. Named experts with skin in the game. Iterative drafts. And a visible audit trail on every claim the document makes.

South Africa was reaching for the same outcome on a compressed timeline. The 2024 Framework drew 32 submissions, the leap from there to a gazetted policy in eighteen months meant fewer rounds of expert engagement than peer jurisdictions allow themselves. The withdrawal is, in a real sense, a delayed consultation finally happening.

Three things from the comparison set are worth borrowing for the next iteration.

1. Name the experts. The EU didn’t staff its HLEG with 52 people for ceremony. It did it because when someone later questions a claim in the document, you need a named human who can defend it. Anonymous drafting committees produce documents that nobody owns. And documents nobody owns are documents nobody can fix when they break.
2. Build in reversibility from day one. Singapore’s iterative framework is premised on a blunt admission: whatever you write about AI governance today will be wrong in interesting ways within 24 months. The institutional design should make revision routine. Not politically painful. Not treated as an admission of error. Just routine.
3. Separate the two jobs. Writing policy is one discipline. Synthesising the academic and technical literature that supports that policy is another. Different skills, different quality gates. A proposed Human-in-the-Loop workflow for the next iteration of South Africa’s National AI Policy.A proposed Human-in-the-Loop workflow for the next iteration of South Africa’s National AI Policy.A proposed Human-in-the-Loop workflow for the next iteration of South Africa’s National AI Policy.A dedicated research function with its own verification protocol would have caught this before gazette.

Here is what that protocol should look like:

The Human-in-the-Loop (HITL) Drafting Pattern

If we are going to use frontier technology to digitise policy formulation, it cannot be an open-ended prompt to a language model. AI is an assistant, not an oracle. The next iteration needs a modern, tools-based architecture:

• Step 1: Constrained Research (The RAG Pattern). Don’t use raw generative AI. Use Retrieval-Augmented Generation (RAG). You point the system only at a closed database of verified, peer-reviewed literature and existing global laws. The AI synthesises drafts strictly from that data, and every claim must auto-generate a hard link back to the source text. No link, no claim.
• Step 2: The Human “Red Team”. Before a draft is ever gazetted, it goes to a named, public-facing committee of domain experts. The AI does the heavy lifting of summarisation while the human experts do the heavy lifting of verification, debate, and contextual application.
• Step 3: AI-Assisted Public Discourse. Modern policies are co-created. When you open a draft to public comment, thousands of submissions flood in. Use Natural Language Processing (NLP) to map sentiment, group arguments, and surface critical legal conflicts, exactly like Singapore and Australia do. The AI synthesises the noise so human policymakers can actually address the substance.
• Step 4: Visible Traceability. Every time a clause is updated, it goes into a digital traceability matrix. If a rule changes, the system permanently logs who from the public suggested it, which expert approved it, and what verified research backs it up.

Here is what we should be clear on. The substance of the withdrawn draft is strong. The vision of AI for inclusive growth, capacity development, and ethical governance rooted in African specificity is exactly right. The proposed institutional architecture, **a National AI Commission, an independent Ethics Board, a Safety Institute, an Insurance Superfund **modelled on the Road Accident Fund, is genuinely inventive. I have not seen another developing country propose anything as structurally ambitious in a single policy document. That architecture deserves to survive the redraft.

What does need to change is the process underneath it. The rigour that makes the substance defensible when it reaches Parliament, when it reaches industry, when it reaches the public.

South Africa has something rare here: a public reset before any of these structures are funded, staffed, or politically locked in. Used well, that is worth more than the original draft could have delivered.

The next twelve months matter more than the last twelve did.