Jori Eskolin · 30 August 2026

A prompt can express a rule. Only a system can enforce it.

Reliability in GenAI analysis does not come from bounding the model to the right material and giving it good instructions. It comes from a system that governs which claims may be formed, on what evidence they may be accepted, and when the analysis must be blocked.

Soft control

Prompt

Expresses desired behaviour.

Soft control

Agent

Orchestrates model activity.

Assurance

Evidence path

Binds claims to approved sources.

Hard control

System gate

Accepts, blocks, or escalates the output.

I have noticed that a surprisingly strong belief persists around artificial intelligence, and generative AI in particular: that following a familiar set of principles will automatically make a GenAI analysis accurate and almost free from the risk of serious error.

The seven principles of the current belief

01Use only predefined data, knowledge, and information drawn from a database and/or supplied background material.
02Specify in the prompt which questions may and may not be asked, which answer types are permitted, which content fields are required, and what structure, form, and tone the answer must take.
03Select a GenAI model suited to the task and set appropriate parameters.
04Define skills, predefined operating instructions, tools, and other parameters that govern the model’s behaviour.
05Ask the model to state which sources it used in the analysis.
06Build an agent on these principles and let it produce the result.
07Connect several agents built in the same way, on the assumption that the analysis will deepen and become more precise as the number of agents increases.

Even full adherence to these principles does not guarantee analytical accuracy. It does not necessarily make the result more reliable than a single, well-prompted rapid analysis of the same material.

The methods above may suffice for clearly bounded, low-risk tasks that mainly extract information from source material. They do not, by themselves, constitute an adequate quality-assurance architecture for analysis that requires abstract reasoning, is complex, or contains multiple dependencies among observations, interpretations, and conclusions.

Connecting several agents does not automatically improve the result either. In some cases a multi-agent system can even underperform a single agent. Any benefit depends on the division of labour, the agents’ independence, the available evidence, the verification mechanisms, and the nature of the task—not on the number of agents. If every agent is built on the same principles and uses similar material, similar reasoning, and similar checking logic, errors are not necessarily corrected. They may be repeated, amplified, or given the appearance of a formally valid but false consensus.

If the aim is to produce, repeatedly, highly accurate analysis grounded in given background information and defined research questions, the target level of accuracy must first be defined in a measurable way. A level above 95 percent could mean, for example, that more than 95 percent of the analysis’s factual claims can be traced to approved evidence that genuinely supports them, and to the criteria set. Even that does not, by itself, guarantee coverage, the logical soundness of conclusions, or that the research questions have been answered. Those properties require their own acceptance criteria and measures.

When the analysis is abstract in character—for example strategic, concerned with defining a company’s needs, or concerned with assessing the effects of climate change—more is required of both the analytical architecture and the cognitive model behind it. GenAI may be an important part of that system. It should not, on its own, control the selection of evidence, the progress of the analysis, the acceptance of quality, or the publication decision.

Twelve stages of an evidence-gated process

These stages do not guarantee an error-free analysis. They create the conditions for analysis that is traceable, repeatedly reliable, and stoppable when evidence is insufficient.

01Deterministically governed classification of informationThe origin, source class, integrity, usability, evidence level, and preliminary assignment of information to research questions are classified according to predefined rules. A deterministic process does not itself guarantee that every semantic classification is correct. It does, however, ensure that classification is performed consistently, traceably, and in accordance with defined rules.
02Inspection of information passed to the next stageEvery packet passed forward is inspected both formally and substantively. Formal inspection verifies fields, identifiers, references, types, schema, and integrity. Substantive inspection may use GenAI, compared against the original source. Incorrect or uncertain classification is returned before analysis.
03Evidence coverage and sufficiency gateBefore analysis begins, the system checks whether the available evidence covers the research question at the required level. Missing information is not filled in from the model’s own knowledge. Insufficient evidence is a result of the analysis, not a licence to guess.
04Confirmation of an evidence gap against the original sourcesA gap is checked against the original sources to distinguish misclassification from genuine poverty of material. If the deficiency prevents a reliable conclusion, that part of the analysis—or the analysis as a whole—is blocked.
05Treatment of research questions as separate analysis packetsEach research question is handled as its own bounded whole, with its own evidence packet, task, result, and acceptance criterion.
06Independent review of each question-specific analysisReview is a separate task, where possible with a different model family, using the original evidence, a separate review contract, and deterministic checks. Changing the model does not, by itself, guarantee independence.
07Synthesis from accepted results onlySynthesis must not introduce new factual claims absent from the accepted analysis packets.
08Review of the synthesisClaims must remain traceable. Contradictions must not be concealed. Conditional observations must not become firm conclusions.
09Formation of actions as a separate wholeThe observed need is kept distinct from the remedy. GenAI may invent neither a problem nor a remedy without an approved evidence base and knowledge base.
10Independent validation of the final outputAtomic claims are checked against original material, logic, coverage, and expressed uncertainty. Failed parts are re-analysed within a bounded iteration limit, then blocked or referred to a human reviewer.
11Deterministic publication decisionPublication is based on system-supervised states, review results, and acceptance criteria—not on the model’s own declaration.
12The output is readyOnly after accepted content review and a deterministic publication decision is the output ready for use.

The test

To avoid leaving the argument as rhetoric alone, I tested how the accuracy of an AI-assisted analytical apparatus might be measured. The comparison held the same materials, the same knowledge base, and the same output format. The difference is whether the rule lives only in a prompt, or is enforced by the system.

Design of the test materials

I prepared nine background-material packages from poor to good. Three represented clear cases (BLOCK, WARN, and GO). The remaining six were grey-area packages at the boundaries. For each package, a correct answer was created together with experts.

Procedure

Ten analyses of each designed package with the apparatus, and ten of each of three randomly selected authentic customer materials. Experts produced reference answers against the same predefined criteria, without seeing apparatus output. A prompted model then received a sample report, the materials, the knowledge base, and an instruction to use only those materials.

Designed packages

Three anchors—BLOCK, WARN, GO—and six grey-area packs at the boundaries.

120

Apparatus runs

Ten runs for each designed package and ten for each of three customer materials.

Prompt-only runs

Ten matched analyses for each designed package, same materials and knowledge base.

79%

Fewer errors

Relative reduction versus prompted AI on the designed packages.

How the comparison was run

Repeated runs were used to observe both accuracy and variance. Only the enforcement of the rule changed.

01 · Reference

Lock expected answers

Experts establish the reference against the same criteria, without seeing apparatus output.

02 · System

Repeat every case

Each designed and customer package is run ten times through the analysis apparatus.

03 · Baseline

Match the prompt

The model receives a sample report, the materials, the knowledge base, and a source restriction.

04 · Score

Compare with reference

Items are classified as correct or incorrect; mean, range, and median are calculated.

Results

Designed packages with the apparatus: mean 94%, range 85–98%, median 96%. Customer materials: mean 93%. Prompted AI on the same designed packages: mean 71%.

Apparatus · designed packages

94%

Mean accuracy · median 96% · observed range 85–98%

Matched prompt-only baseline

71%

Mean accuracy · median 75% · observed range 59–86%

(29 − 6) ÷ 29 = 79%

Accuracy improved by 23 percentage points. Expressed as error, the mean fell from 29% to 6%—a 79% relative reduction in errors. The 79% figure is not a percentage-point change.

Overall accuracy, range, and median

Band is the observed range. Disc is the mean. Tick is the median. Scale starts at 50%.

Apparatus range Customer range Prompted range Mean Median

The BLOCK material had the widest range in both conditions: 85–95% for the apparatus (mean 91%) and 59–76% for prompted AI (mean 68%). WARN and GO remained tight for the apparatus (95% and 97%, each ±2 percentage points). In the prompted condition, WARN and GO each obtained a mean of 76%. None of the customer materials was BLOCK-level material.

Mean by publication gate

Designed packages only. Whiskers show the full observed range. Customer packs are omitted because none were BLOCK-level.

Apparatus Prompted AI

Full result sheet

Mean on the first line. Range and median (M) on the second.

Condition	Overall	BLOCK	WARN	GO
Apparatus · designed	94%85–98 · M 96	91%85–95	95%93–97	97%95–99
Apparatus · customer	93%88–95 · M 93	—	—	—
Prompted AI · designed	71%59–86 · M 75	68%59–76	76%73–86 · M 76	76%73–86 · M 76

What the test showed

01Exact calculation belongs in deterministic computation.Generative AI is not a reliable substitute for deterministic calculation. In this test the prompted model made direct arithmetic errors.
02The apparatus produced 79 percent fewer errors, in relative terms.This applies to the matched designed-package comparison: 94% mean accuracy versus 71% for the prompt-only baseline.
03A rule contained in a prompt is not a rule enforced by the system.The prompt expresses desired behaviour; the surrounding architecture must validate and enforce compliance.
04Restricting the material does not guarantee its correct use.Even explicitly permitted information can be omitted, misread, or combined incorrectly.
05Reliability is a system property.It does not arise from the model, or from the permitted material, alone. It arises from the cognitive model, the evidence path, deterministic controls, bounded AI roles, validation, and the ability to block publication.

The analysis apparatus — The system used in the test: Buying Insight & Hypothesis Creation

A translator from company reality into structured signals, testable need hypotheses, and safe discovery paths.

The system is designed to keep evidence, interpretation, hypotheses, and actions distinguishable. Weak or unknown evidence remains gated; it is not converted into confirmed buying intent.

Three related presentations describe the cognitive model, evidence-governed flow, foresight methodology, and an example of the knowledge-base material used by the system.

The answer is only as trustworthy as its evidence path—not as impressive as its model.