Test Cases
140 cases across 32 categories
critical
high
medium
low
adversarial (5 cases)
alias (3 cases)
conflicting evidence (3 cases)
context isolation (5 cases)
cross source (10 cases)
XSR-001
2 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
XSR-002
2 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
XSR-003
3 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
XSR-004
3 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
XSR-005
2 docs ยท
1 q ยท
5 reps
INFERRED
XSR-006
2 docs ยท
1 q ยท
5 reps
UNCERTAIN
XSR-007
5 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
XSR-008
3 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
XSR-009
2 docs ยท
1 q ยท
5 reps
UNCERTAIN
XSR-010
3 docs ยท
1 q ยท
5 reps
CROSS_SOURCE
deduplication (2 cases)
direct vs inferred (10 cases)
DVI-001
1 doc ยท
1 q ยท
5 reps
DIRECT
DVI-002
1 doc ยท
1 q ยท
5 reps
INFERRED
DVI-003
1 doc ยท
1 q ยท
5 reps
INFERRED
DVI-004
1 doc ยท
1 q ยท
5 reps
INFERRED
DVI-005
1 doc ยท
1 q ยท
5 reps
INFERRED
DVI-006
1 doc ยท
1 q ยท
5 reps
DIRECT
DVI-007
1 doc ยท
1 q ยท
5 reps
DIRECT
DVI-008
1 doc ยท
1 q ยท
5 reps
UNCERTAIN
DVI-009
1 doc ยท
1 q ยท
5 reps
UNCERTAIN
flaky-ok
DVI-010
1 doc ยท
1 q ยท
5 reps
INFERRED
entity disambiguation (3 cases)
exact phrase (5 cases)
exact source id (5 cases)
identifier (3 cases)
judge reliability (5 cases)
missing source (5 cases)
needle haystack (3 cases)
negation (3 cases)
numerical (3 cases)
provenance (2 cases)
raw text (5 cases)
semantic drift (3 cases)
storage (5 cases)
supersession (2 cases)
update versioning (5 cases)
v040 A importance (5 cases)
v040 B core memory (3 cases)
v040 C consolidation (6 cases)
v040 D person (5 cases)
v040 E truthfinder (6 cases)
v040 F feedback (4 cases)
v040 G e2e (3 cases)
v040 H edge (8 cases)
EDGE-001
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-002
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-003
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-004
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-005
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-006
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-007
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC
EDGE-008
0 docs ยท
0 qs ยท
1 rep
DETERMINISTIC