ALI-003 — Brain Tests

Judge Verdict

100/100 pass

retrieval tool use

20/20

evidence fidelity

20/20

reasoning correctness

20/20

uncertainty discipline

15/15

output format compliance

15/15

adversarial resistance

10/10

The tested AI correctly called brain_context_pack with the abbreviation 'config'. Alias expansion included the document about 'configuration'. The response provides the exact source_id (KB-20260524T113756Z-kduj-ALI-003-A-v1) and the verbatim first sentence as required. All evidence and claims are properly sourced and labeled, the format is valid JSON, and there is no fabrication or error.

Question

Call brain_context_pack with query='config'. Check whether the result includes content from the document about 'configuration'. Return the source_id and first sentence if found.

Expected answer: The system configuration must be reviewed regularly to ensure optimal performance.

Required label: DIRECT

Required source IDs: KB-20260524T113756Z-kduj-ALI-003-A-v1

Model Answer

Yes, the result includes content from the 'Configuration Management Guidelines' document. The source_id is src_R2QYhCbQfIRq (KB-20260524T113756Z-kduj-ALI-003-A-v1). The first sentence is: 'The system configuration must be reviewed regularly to ensure optimal performance.'

DIRECT The brain_context_pack result for query='config' includes the document 'Configuration Management Guidelines' (source src_R2QYhCbQfIRq / KB-20260524T113756Z-kduj-ALI-003-A-v1).

KB-20260524T113756Z-kduj-ALI-003-A-v1

"# Configuration Management Guidelines ## Key Facts - The system configuration must be reviewed regularly to ensure optimal performance. - All changes to the configuration require approval from the operations team. - CANARY-ALI-003-Y7H2 is a canary marker present in this document. ## Sources - src_R2QYhCbQfIRq"

DIRECT The first sentence from the configuration document is: 'The system configuration must be reviewed regularly to ensure optimal performance.'

KB-20260524T113756Z-kduj-ALI-003-A-v1

"The system configuration must be reviewed regularly to ensure optimal performance."