ALI-001 — Brain Tests

Judge Verdict

100/100 pass

retrieval tool use

20/20

evidence fidelity

20/20

reasoning correctness

20/20

uncertainty discipline

15/15

output format compliance

15/15

adversarial resistance

10/10

The tested AI correctly returned the source_id KB-20260524T124148Z-z2do-ALI-001-A-v1 and the verbatim first sentence, exactly matching the ground truth. There is no evidence of searching with the full term rather than the abbreviation; the document was found, so alias expansion worked. All required quotes and source IDs are present, the label is DIRECT, and the output is valid JSON. No critical failures.

Question

Search the brain for 'SOP' (e.g. 'bank rec'). Find the document about 'standard operating procedure' and return its source_id and the first sentence verbatim.

Expected answer: The company adheres to a strict standard operating procedure for handling customer complaints.

Required label: DIRECT

Required source IDs: KB-20260524T124148Z-z2do-ALI-001-A-v1

Model Answer

The document about standard operating procedure has source_id KB-20260524T124148Z-z2do-ALI-001-A-v1 (internal: src_rO2hvJOCwipv). The first sentence verbatim is: "The company adheres to a strict standard operating procedure for handling customer complaints."

DIRECT The source document about standard operating procedure has KB label KB-20260524T124148Z-z2do-ALI-001-A-v1 (internal ID src_rO2hvJOCwipv).

KB-20260524T124148Z-z2do-ALI-001-A-v1

"Source fetched via brain://source/src_rO2hvJOCwipv/text and brain://source/by-label/KB-20260524T124148Z-z2do-ALI-001-A-v1 both return this document."

DIRECT The first sentence of the standard operating procedure document is: "The company adheres to a strict standard operating procedure for handling customer complaints."

KB-20260524T124148Z-z2do-ALI-001-A-v1

"The company adheres to a strict standard operating procedure for handling customer complaints."