Evaluation · 12 questions
Measure the answer.
Each question passes only when the answer cites the expected document and page. The same set becomes the regression suite after every re-index.
12questions ready
Run the set to compare each citation with the expected source.
| Question | Expected source | Result |
|---|---|---|
01What is the notice period for the Wola contract? | Agreement B · Nov 2025 · p. 2 | Not run |
02Which contract applies to the Mokotów flat? | Agreement B · Nov 2025 · p. 3 | Not run |
03What is the average price per square metre in Q3 2026? | Market report Q3 · p. 7 | Not run |
04Can a tenant keep a dog? | Lease template · p. 3 | Not run |
05What commission does the agency charge? | Fee policy · p. 1 | Not run |
06Do the two contracts agree on the deposit?Pass only when both contracts appear. | Draft A · Mar 2025 · p. 2 Agreement B · Nov 2025 · p. 2 | Not run |
07What is the deposit under the signed contract? | Agreement B · Nov 2025 · p. 2 | Not run |
08How long does a flat stay on the market in Q3 2026? | Market report Q3 · p. 7 | Not run |
09What is the price of the Mokotów flat? | Mokotów 14 · p. 1 | Not run |
10Does the Wola flat include parking? | Wola 22 · p. 1 | Not run |
11What is the fixed penalty for a late rent payment?No fixed penalty clause in the demo corpus. | Lease template · p. 4 Known gap | Not run |
12What is the energy rating of the Mokotów building?No energy certificate in the demo corpus. | Mokotów 14 · p. 2 Known gap | Not run |
Pass rule: every expected document id and page appears in the answer citations.12 questions · not scored
Twelve questions, six risks
Prices, dates, notice periods, deposits, pets and one direct conflict. The set covers the questions an agent asks in a viewing or a handover.
Two deliberate gaps
Questions 11 and 12 have no supporting passage in the demo corpus. The test reports them as gaps instead of hiding them. A wider corpus closes them.
One suite, two jobs
The same cases score the demo and guard production. Run the set after each ingestion change and block a deploy when the score falls.