The AI Runtime Field Lab

Field Briefs

Internal Tools Advanced Knowledge Agent Open Draft, pending editorial review

One Source of Truth: A Shared Context Layer Across Tools

Build a context layer that reconciles facts scattered across code, chat, docs, CRM, and product tools into one owned, cited, conflict-aware source of truth.

Why this matters

Context about a customer, a feature, or a decision lives in fragments across the codebase, Slack, the CRM, product docs, and sales notes, and each tool holds a slightly different version. People and AI assistants both answer from whichever fragment they happen to see, so the same question returns different answers depending on where it was asked. The hard part is not retrieval. It is reconciling conflicting fragments into one current, owned, cited truth, and making clear who maintains it.

Persona

Operations, platform, or knowledge lead at a company where context is split across many tools

Current manual workflow

Each team keeps its own version of the truth in its own tool. People ping each other to find the current answer, nobody owns the canonical version, and stale or conflicting facts persist because no single place is responsible for resolving them.

The AI workflow to build

The system ingests sanitized records from several sources (a code README, chat messages, CRM records, product notes), extracts atomic facts with provenance, and reconciles them into one canonical entry per entity. It detects conflicts and staleness, attaches an owner and a review date to each entry from an ownership map, and answers a question with the current value, its source, the responsible owner, and any unresolved conflict flagged for resolution. On a conflict it never silently picks a winner; it surfaces the disagreement to the owner.

Inputs

  • sanitized snippets from several tools (a code README, chat messages, CRM records, product docs)
  • the entity or entities to track
  • an ownership map naming a system of record per fact type

Outputs

  • a canonical context entry per entity with the current value, provenance, owner, and freshness
  • a conflict list with the disagreeing sources
  • an answer to a query with its source and owner
  • a staleness report of entries past their review date

Definition of done

On a synthetic multi-source fixture seeded with conflicting and stale facts about the same entities, the system reconciles each entity to one canonical value with provenance and an owner, flags every conflict rather than silently choosing, marks entries past their review date as stale, and answers a query with the current value, its source, and the responsible owner.

Example input

Three sources disagree on a customer plan tier: the CRM says Pro, a chat message says Enterprise, a product doc says Free. The ownership map names the CRM as the system of record for billing facts.

Example output

Canonical plan tier: Pro, source CRM, owner billing per the ownership map. Conflict flagged: a chat message and a product doc disagree, both cited, owner notified. The chat claim is marked unverified pending owner resolution rather than overwriting the system of record.

Data plan

sanitized data

Boundaries and non-goals

  • real production tool integration with Slack, CRM, or code APIs
  • real customer or company data
  • writing back to source systems
  • building source connectors

Evaluation ideas

  • reconciliation correctness against a key
  • conflict-detection recall on seeded conflicts
  • ownership attribution accuracy
  • staleness flagging accuracy
  • no-silent-winner: whether any conflict is resolved without being surfaced

Run Level target

R3 Reliable Plain translation: handles real cases.

Scope envelope

Buildable by one solo builder in 20 to 30 focused hours, on public, synthetic, or sanitized data, with a demo path that requires no production access.

Suggested tools

  • any LLM for fact extraction and reconciliation
  • a simple store for the canonical context and the ownership map

Suggested options, never requirements; briefs are tool-agnostic.

Product thesis questions

  • When sources disagree, who should own the fact, the system of record or the most recent update?
  • How should an entry freshness policy be set per fact type, since a price changes faster than a company name?
  • Should the layer be a service every tool queries live, or a periodic reconciliation job that writes a canonical store?