The federation gap in enterprise operational data

Between 2022 and 2025, enterprise teams broadly adopted retrieval-augmented generation as a solution to LLM hallucination. The predominant implementation pattern — documents chunked, embedded into a vector store, retrieved via cosine similarity — handles document-level static knowledge adequately. Thakur et al. demonstrated on the BEIR benchmark that dense retrievers underperform BM25 on out-of-domain corpora by 11.7 NDCG points on average,^[1] a gap that widens when the query targets operational state rather than indexed text.

Operational enterprise queries — "which at-risk deals have fulfilment issues?", "how many open P1 tickets do we have right now?" — require current state from systems of record, not historical text from indexed documents. The answer to such a query does not exist as a document. It exists as a join across live API endpoints that were never designed to communicate with each other. Dresner Advisory Services estimates that enterprise analysts spend 30–45 minutes per day context-switching across systems to manually assemble answers that cross system boundaries.^[2]

Three developments make a systematic federated synthesis approach tractable as of 2026. First, Schick et al.'s Toolformer demonstrated that LLMs can reliably self-direct API invocations from natural language input with minimal fine-tuning,^[3] establishing the theoretical basis for LLM-directed connector routing. Second, Gemini's two-tier model family — a lightweight Flash-Lite tier for high-volume routing decisions and a stronger Flash tier for reasoning-visible synthesis — makes cost-appropriate multi-step LLM orchestration viable against the Gemini Developer API at negligible per-query cost.^[4] Third, Promise.all() enables parallel async fan-out to multiple connectors with latency bounded by the slowest, not the sum, regardless of whether the fan-out runs in the browser or, as this revision requires, inside a Cloud Run service.

What document-indexed retrieval cannot answer

The failure mode of document RAG on operational enterprise queries is not noisy — it is silent. A document retrieval system returns an answer grounded in indexed text. When the indexed text is a policy guide or a sales playbook, the retrieval produces a fluent, confident response to a question that required live operational data. The document was the wrong unit of truth.

Guu et al. established that open-domain QA systems relying on static corpora degrade predictably when the query targets time-sensitive entity state — a finding that extends directly to enterprise operational queries where the correct answer is a function of current system state, not historical documentation.^[5] FederaQ addresses this class of query exclusively; static document lookup is explicitly out of scope.

2.1 — Query type taxonomy and failure modes

Six layers. One query. Grounded synthesis.

FederaQ is structured as six discrete layers. Each layer has a single responsibility. The boundary between layers is a well-defined interface. Layer 2 (Proxy) is a new architectural addition that resolves the credential isolation requirement: the browser client never holds API keys. The proxy layer is now realised as a Cloud Run service, fronted by Google API Gateway, which validates a Firebase-issued Google Sign-In JWT before any request reaches Cloud Run — Cloud Run itself grants run.invoker only to the Gateway's service account, never to allUsers. All upstream calls to Gemini and, in production, to enterprise connectors, are mediated through this layer.

3.1 — End-to-end latency model

The following latency budget is derived from documented component benchmarks. Gemini Flash-Lite and Flash token-generation figures are from Google's published Gemini Developer API benchmarks.^[4] API Gateway adds a JWT-validation hop in front of Cloud Run; Cloud Run itself scales from zero on a cold start of roughly 1–2s for a small Node container, amortised to near-zero on warm invocations within the same session.^[6] All figures are P50 estimates; P99 analysis follows.

3.2 — Connector federation model

3.3 — Adaptive output format selection

3.4 — Technology stack

Credential isolation via Cloud Run + API Gateway + Firebase Auth

4.1 — Cloud Run allowlist check

API Gateway validates the Firebase-issued JWT before a request ever reaches Cloud Run — that validation is declared in the Gateway's OpenAPI spec, not application code. Cloud Run still applies its own authorisation check on the forwarded identity, matching the least-privilege pattern used in the reference builds:

// server.ts · Cloud Run · federaq-service
// Gemini key comes from Secret Manager via env var — never in source, never in client code

function requireAllowlist(req, res, next) {
  // API Gateway already validated the Firebase JWT; it forwards the
  // decoded claims in this header. Cloud Run trusts Gateway, not the caller.
  const userInfoHeader = req.headers['x-apigateway-api-userinfo'];
  if (!userInfoHeader) {
    return res.status(403).json({ error: 'Missing identity header' });
  }

  const claims = JSON.parse(Buffer.from(userInfoHeader, 'base64url').toString());
  const allowlist = process.env.ALLOWED_EMAILS.split(',');
  if (!allowlist.includes(claims.email)) {
    return res.status(403).json({ error: 'Not authorized: ' + claims.email });
  }
  next();
}

app.post('/query', requireAllowlist, async (req, res) => {
  // Gemini call happens only after both checks pass — Gateway's JWT
  // validation, then this service's own allowlist
  const classification = await classifyIntent(req.body.query);
  // ... connector fetch, cache, synthesis — unchanged from §03
});

TTL policy with stateful connector mutation

5.1 — TTL value derivation

TTL values are not arbitrary. Each value is derived from documented source-system update frequency characteristics and operational staleness tolerance for the query type. The following derivations apply to the design phase; production values would be empirically calibrated against actual connector event logs.

5.2 — Cache routing decision tree

5.3 — Stateful connector mutation model

// Stateful connector — ERP inventory simulator
// State mutates on a deterministic schedule, exercising real TTL events

class ERPConnector {
  constructor() {
    // Seed from load timestamp — deterministic, reproducible
    this.seed = Date.now();
    this.state = this.initState();
    // Mutate every 15 seconds — faster than 60s TTL, guarantees TTL miss events
    setInterval(() => this.mutate(), 15000);
  }

  mutate() {
    // Deterministic delta: decrement SKU quantities, flip reorder flags
    this.state.skus = this.state.skus.map(sku => ({
      ...sku,
      quantity: Math.max(0, sku.quantity - Math.floor((Date.now() % 7))),
      below_reorder: sku.quantity < sku.reorder_threshold
    }));
    this.state.last_mutated = Date.now();
  }

  query(intent, filters) {
    return {
      data: this.state,
      source: 'ERP · SAP',
      fetched_at: Date.now(),   // stamped at query time, not mutation time
      ttl_policy: 60           // seconds
    };
  }
}

Eventual consistency with explicit staleness signalling

The system queries three connectors via Promise.all() at a point in time. Each connector returns its own fetched_at timestamp. When connector results are temporally inconsistent — for example, when ERP inventory state reflects a cache hit from 55 seconds ago while the CRM deal stage is live — the synthesis may join data from different operational moments. In enterprise operational contexts, this can produce materially incorrect conclusions for time-sensitive decisions.

The consistency model below defines the system's formal consistency posture, the consistency window threshold, and the handling rule for skew violations.

Intent classifier, rule-based planner, and failure taxonomy

7.1 — Classifier prompt (full specification)

// gemini/classify.ts · Cloud Run · gemini-3.1-flash-lite · temp=0.0

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });  // ← Secret Manager, never client code

// Classification taxonomy — unchanged from the design phase:
// intent:      "cross-system" | "live-status" | "inventory" | "pipeline" | "single-connector"
// connectors:  ["CRM"] | ["ERP"] | ["Ticketing"] | any combination
// format:      "TABLE" | "CHART" | "CARD" | "PROSE"
// sensitivity: "high" | "low"

const response = await ai.models.generateContent({
  model: 'gemini-3.1-flash-lite',
  contents: `${CLASSIFIER_GUIDANCE}\n\nQuery: "${query}"`,
  config: {
    temperature: 0,
    responseMimeType: 'application/json',
    responseSchema: {                      // ← decoding-level constraint, not just instruction
      type: Type.OBJECT,
      properties: {
        connectors:  { type: Type.ARRAY, items: { type: Type.STRING, enum: ['CRM','ERP','Ticketing'] } },
        format:      { type: Type.STRING, enum: ['TABLE','CHART','CARD','PROSE'] },
        sensitivity: { type: Type.STRING, enum: ['high','low'] },
      },
      required: ['connectors', 'format', 'sensitivity'],
    },
  },
});

// Confidence and format-confidence fallback rules (§7.2, F2/F3) are unchanged —
// they still govern what happens when the classifier is uncertain, not whether it is.

7.2 — Classifier failure taxonomy

7.3 — Rule-based query planner

The Query Planner takes the classifier output and generates per-connector sub-queries using a deterministic template, not a second LLM call. This removes ~180ms from the critical path and makes the planner fully unit-testable.

Architecture ceiling and three-stage graduation path

Cloud Run's scale-to-zero, Firestore's Always Free daily quota, and the Gemini Developer API's free/low-cost tier constitute a single-tenant, budget-capped architecture. At demo scale every component sits inside a perpetual free quota; the binding constraint is not user count but the deliberately tiny $0.01 hard cap on the Gemini billing account, which is a policy choice, not a technical ceiling. The following table defines what changes at each tier and, critically, which change is a technical requirement versus a budget policy that could be lifted without any architectural change.

ADM Phase B + Phase C positioning

This document operates as a Phase B + Phase C artifact within the TOGAF Architecture Development Method. Phase B establishes the business context — the federation gap, the query taxonomy, and the stakeholder concern (Sales Directors, Ops leads) whose cross-system queries this system addresses. Phase C defines the information systems architecture — the connector federation model, the data flow between layers, and the freshness policy as a data architecture constraint.

9.1 — Architecture Building Blocks

9.2 — Phase A (Architecture Vision) mapping

The Architecture Vision is captured in §01 (Motivation): enterprise operational teams are blocked from answering cross-system queries because no single system of record contains the required join, and no existing tooling automates the federation. FederaQ addresses this gap without requiring application migration, new enterprise software licenses, or changes to existing systems of record — consistent with the TOGAF ADM principle of maximising reuse of existing infrastructure.^[10]

All significant architectural choices · traceable rationale

Architectural parameter simulator · TTL and latency instrumentation

Citations

1. [1]Thakur, N. et al. (2021). BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS 2021 Datasets and Benchmarks Track. arXiv:2104.08663.
2. [2]Dresner Advisory Services. (2024). Wisdom of Crowds® Business Intelligence Market Study. Dresner Advisory Services LLC. [Analyst report; context-switching productivity data cited from executive survey findings.]
3. [3]Schick, T. et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS 2023. arXiv:2302.04761.
4. [4]Google LLC. (2026). Gemini API Documentation — Models, Rate Limits, and AI Studio Developer API. ai.google.dev/gemini-api/docs. [Gemini 3.1 Flash-Lite and Gemini 3.5 Flash; Developer API key auth via AI Studio; free-tier and pay-as-you-go pricing.]
5. [5]Guu, K. et al. (2020). REALM: Retrieval-Augmented Language Model Pre-Training. ICML 2020. arXiv:2002.08909.
6. [6]Google LLC. (2026). Cloud Run, API Gateway, and Firebase Authentication Documentation — Always Free Tier Limits and Scale-to-Zero Behaviour. cloud.google.com/run/docs, cloud.google.com/api-gateway/docs, firebase.google.com/docs/auth. [Cloud Run Always Free: 2M requests/month; scale-to-zero with cold start on first request; API Gateway JWT validation against Firebase issuer.]
7. [7]SAP SE. (2023). SAP Materials Management (MM) Configuration Guide — Batch Job Scheduling for Inventory Posting. help.sap.com. [Standard batch posting cycle documented at 60–300s for SMB deployments.]
8. [8]Atlassian Inc. (2024). Jira Cloud REST API Documentation — Webhooks: Delivery SLA and Retry Policy. developer.atlassian.com/cloud/jira. [Webhook delivery latency documented up to 60s; retry on failure.]
9. [9]Salesforce Inc. (2024). Salesforce Streaming API Developer Guide — Push Topics and Event Delivery Latency. developer.salesforce.com. [CRM Streaming API push latency 1–3s; deal stage human-update frequency empirically measured in minutes to hours.]
10. [10]The Open Group. (2018). TOGAF® Standard, Version 9.2. The Open Group. ISBN: 978-94-018-0-303-6. [ADM principles: maximise reuse, Phase B Business Architecture, Phase C Information Systems Architecture.]
11. [11]Sethi, R. et al. (2019). Presto: SQL on Everything. IEEE ICDE 2019. [Federated query execution model; connector interface design principles.]