Product guide · Architecture

Documentation

Evidence Layer is an evidence-selection layer between a retrieval system and the model that generates the answer. This guide distinguishes implemented code from service capabilities that remain disabled or planned.

In one sentence

Evidence Layer selects and traces evidence; it does not write the final answer.

A retrieval source or caller supplies candidate passages. Evidence Layer scores and ranks those candidates under a configured policy, then returns a smaller set with source identity and provenance so the caller’s language model can compose its own response.

How the flow works

Receive a task and eligible candidates. A retrieval layer or caller supplies a query and candidate records. A production source connector and hosted retrieval API are not part of the current service.
Score the candidates. The Python evidence core includes deterministic lexical scoring and a JEV scorer behind provider-selection policy. JEV use requires explicit policy enablement and a separately trusted provider-transfer authorisation; supported package paths currently fail closed for live transfer.
Rank and select evidence. Results are sorted and a configured evidence budget can return a focused subset. When a JEV result cannot be safely used, the policy recomputes the entire candidate set lexically rather than mixing scores.
Return evidence with provenance. The schemas preserve candidate identifiers and source references so callers can trace selected material back to the supplied evidence.
Generate the answer elsewhere. The caller’s chosen LLM consumes the selected context and writes the final answer. Evidence Layer does not contain a general-purpose answer-generation provider.

Implemented, constrained, and planned

Implemented in the repository

Python schemas, a lexical scorer, ranking, and evaluation metrics.
A JEV adapter and provider policy with whole-request lexical fallback behavior.
A local stdio MCP tool named retrieve_evidence.
Real-World and synthetic evaluation data, frozen manifests, result hashes, and offline verification scripts.
A Next.js Evidence Explorer that renders fictional local sample records and source references.

Constrained or unavailable in this application

The standard local MCP path is lexical-only and the corpus source boundary denies reads by default unless a trusted source boundary is injected.
Live JEV transfer is not enabled on the supported application package paths.
There is no production customer-document retrieval or answer-generation flow in the public Evidence Explorer.
The local deterministic source-routing library is not wired into a service.

Planned or requiring a separate release

Production source connectors, hosted retrieval, and a customer-data service.
Provider transfer authorisation and live JEV use in an authorised service path.
Integrated source routing, account usage reporting, billing, and a customer Comparison Lab.
A supported final-answer generation integration; callers currently own that step.

Evidence provenance and verification

Candidate IDs and source references travel through the schemas and selection results so a caller can identify where selected evidence came from. Evaluation manifests record input and output fingerprints; offline verifiers recompute saved rankings and metrics without provider calls. These repository checks verify the saved experiment artifacts. They do not independently verify the truth of source material or every future customer connector.

Source access and provider transfer are separate security decisions. A valid source-read boundary must establish access before reading protected content. Sending query or candidate text to a scorer requires a separate trusted transfer authorisation. Workspace identity, source ACLs, safe read boundaries, and fail-closed provider policy are distinct controls; a ranking result does not grant access to any source.

Token efficiency and cost optimisation

A focused evidence set can reduce repeated or irrelevant context sent to a downstream model. In the Phase 7 experiments, Evidence Layer passed three selected passages while the traditional baseline passed five. Fewer input passages reduced total tokens in those runs, but this does not guarantee savings on another dataset or model.

Selection and scoring also consume compute. A remote scorer can add API usage and latency; local ranking consumes CPU time. Total cost should include retrieval, scoring, retries, answer generation, and operations. Token savings alone do not establish lower system cost, and shorter context does not prove equal or better evidence coverage. See the separateCost & Performance report for results by experiment.

Integrations and security boundaries

The repository includes Supabase Auth and server-side workspace API code for account identity and workspace metadata. That is separate from evidence retrieval. Google Drive ingestion, a production source router, a customer-facing retrieval API, and an LLM answer provider are not implemented in the current service. The local service and evaluation harness should not be treated as a production SaaS endpoint.

Provider keys belong on a trusted server boundary. Source-access permission does not authorise sending content to a third party; transfer must be separately authorised and minimised. When remote scoring is unavailable or returns incomplete or invalid results, the policy must either recompute all scores locally or fail closed, according to the configured boundary.