CAU-001

schema_version: "1.0"
test_id: "CAU-001"
category: "causal"
severity: "high"
repetitions: 3
reasoning_type: "INFERRED"
num_documents: 1
num_questions: 1
tags: ["causal-extraction", "predicate", "leads_to", "inference"]

setup_instructions: |
  Generate a document containing a clear causal relationship using "leads to" phrasing.
  Ingest it using brain_ingest with the causal claim extracted as a structured claim
  with predicate="leads_to".

  Requirements:
  - Invent a fictional process chain: "X leads to Y" (e.g. "Poor inventory tracking leads to stock shortages").
  - Use a unique canary token so the claim is uniquely identifiable.
  - source_id: KB-{{RUN}}-CAU-001-A-v1
  - MANDATORY: ingest using brain_ingest with extracted.claims containing:
      predicate="leads_to", subject="<X>", object="<Y>", confidence=1.0
    Without this structured extraction the test will fail — plain ingestion is not enough.
  - Question: search for Y and verify the causal claim linking X→Y is stored.

  Output ONLY this JSON:
  {
    "run_id": "{{RUN}}",
    "test_id": "CAU-001",
    "repetition": {{REP}},
    "documents": [
      {
        "source_id": "KB-{{RUN}}-CAU-001-A-v1",
        "content": "<one sentence: 'X leads to Y [canary: token]'>",
        "title": "<causal process title>",
        "version": 1
      }
    ],
    "questions": [
      "Search the brain for '<Y>' and verify a causal claim exists showing '<X>' leads to '<Y>'. Return the claim text and its predicate if present in the search result."
    ],
    "expected_answers": [
      {
        "question_index": 0,
        "correct_answer": "<X> leads to <Y>",
        "required_source_ids": ["KB-{{RUN}}-CAU-001-A-v1"],
        "required_label": "INFERRED",
        "must_refuse": false,
        "notes": "The brain must store and retrieve a causal claim. The claim text should mention both X and Y. Label is INFERRED because the relationship is a causal inference."
      }
    ]
  }

execution_instructions: |
  You have access to MCP Knowledge Brain tools.

  ══════════════════════════════════════════════════
  MANDATORY INGEST FORMAT — DO NOT SKIP THIS STEP
  ══════════════════════════════════════════════════
  You MUST call brain_ingest with the `extracted` parameter structured EXACTLY as shown:

  brain_ingest(
    text="<document content from setup_context>",
    metadata={"source_label": "<source_id from setup_context>", "domain": "brain-test"},
    extracted={
      "claims": [
        {
          "text": "<the full causal sentence: X leads to Y>",
          "subject": "<X — the cause>",
          "predicate": "leads_to",
          "object": "<Y — the effect>",
          "confidence": 1.0
        }
      ]
    }
  )

  WITHOUT the extracted.claims.predicate="leads_to" field this test FAILS.
  A plain brain_ingest without `extracted` does NOT store the causal structure.
  ══════════════════════════════════════════════════

  After ingesting, use brain_search to find the claim by searching for Y (the effect).
  Report what was returned, including any predicate field if visible.

  Question: {{QUESTION}}

  Return ONLY this JSON:
  {{REQUIRED_ANSWER_SCHEMA}}

judge_instructions: |
  Ground truth:
  {{SETUP_CONTEXT_JSON}}

  Tested AI answer:
  {{EXEC_ANSWER_JSON}}

  The AI ingested a causal claim "X leads to Y" with predicate=leads_to and then searched for it.
  Evaluate whether the causal claim was successfully stored and retrieved.

  Scoring:
  - retrieval_tool_use (0-20): Used brain_ingest with extracted.claims AND brain_search?
  - evidence_fidelity (0-20): Correct source_id? Claim text mentions both cause and effect?
  - reasoning_correctness (0-20): Causal relationship correctly identified and reported?
  - uncertainty_discipline (0-15): Used INFERRED label appropriately?
  - output_format_compliance (0-15): Valid JSON?
  - adversarial_resistance (0-10): Full marks.

  Critical failures: causal claim not found in brain, wrong source_id, did not use extracted.claims with predicate.

  Return ONLY this JSON:
  {{JUDGE_RESULT_SCHEMA}}

critical_failures:
  - "causal_claim_not_stored"
  - "causal_claim_not_retrieved"
  - "hallucinated_source_id"