QueryForge: Adaptive Retrieval
Optimization, Built Google-Native

§1 Problem Statement

Every enterprise search vendor that shipped RAG between 2023 and 2025 made the same architectural assumption: that any question an employee asks can be answered by finding the nearest document. That assumption holds for simple factual lookups. It fails, quietly and confidently, for roughly 40% of real enterprise knowledge-base queries — the ones that require synthesizing across multiple documents, reasoning about policy versions over time, comparing named entities simultaneously, or matching exact identifiers that semantic embeddings blur into a neighborhood. The failure is not a bad score. It is a high-confidence wrong answer, indistinguishable from a correct one until someone acts on it.

The research basis for this is settled. Thakur et al.'s BEIR benchmark (2021) showed dense retrievers underperform BM25 by 11.7 NDCG points on average on out-of-domain corpora. Yang et al. (HotpotQA, 2018) showed single-vector retrieval finds all required evidence for only 44.3% of multi-hop questions. The fix — query decomposition, parallel hybrid retrieval, rank fusion — has been understood in the literature for years. What has not existed is a production-deployable, instrumented, cost-free implementation that applies it adaptively per query type and returns a fully explained routing decision with every result. That is what QueryForge is.

These are not edge cases. They are the dominant query shapes in any enterprise knowledge base, and each requires a structurally different retrieval approach. A larger embedding model does not solve them — the problem is architectural, not parametric.

§2 Request Pipeline

Every call to POST /v1/optimize runs inside a single Cloud Run container. There is no separate vector database service, no separate reranker service, no separate query-rewriting service — every intelligence component runs in-process, keeping the deployment surface to a single container and the cold-start penalty to the one place it is unavoidable. The only outbound network call is to the Gemini Developer API, which is billed on its own independent free-tier quota and never touches Cloud Billing. This is not a simplification for demo purposes — it is the production topology, chosen because the constraint set made any other approach inadmissible.

§3 Pipeline Components

The classifier is the highest-risk component in the pipeline. A multi-hop query misclassified as single-hop reproduces exactly the silent failure QueryForge was built to prevent — a confident, incomplete answer with no signal that anything went wrong. Full transparency into the classifier's decision is therefore a design requirement, not an observability feature added after the fact. Every response includes the query type, the confidence score, the specific token-level signals that triggered the classification, a one-sentence plain-language reasoning field, and — if decomposition ran — the generated sub-queries. Confidence below 0.75 routes conservatively to hybrid+decompose regardless of the predicted type, because the cost of a false-positive decomposition (unnecessary latency) is structurally lower than the cost of a false-negative classification (silent retrieval failure).

All three Gemini calls — classifier, decomposer, HyDE — use the Gemini Developer API at aistudio.google.com, not the Vertex AI Gemini endpoint. This is ADR-004: the Vertex AI endpoint bills against Cloud Billing per token; the Developer API has its own independent free-tier quota and never touches the $0.01-capped account. Decomposition and HyDE are live in the build. Step-back prompting and synonym-expansion rewrite have defined classifier routes but are not yet corpus-evaluated — documented honestly in §12 rather than presented as shipped.

Three retrieval strategies execute concurrently via asyncio.gather() — total wall time is bounded by the slowest strategy, not their sum. The α parameter controlling the dense/sparse blend is not hand-tuned: it is set per query type from Luan et al.'s grid search across TREC-COVID, MS MARCO, and HotpotQA, and the value used on each query is returned in the config output so engineers can audit and override it. The cross-encoder reranker is applied selectively — only on multi-hop and entity-scoped queries where the +6.2 MRR precision gain (MS MARCO leaderboard) justifies the ~600ms CPU overhead. Applying it universally would make every query pay the latency cost of the hardest query type; the routing table encodes the tradeoff explicitly. Nothing in this layer calls a paid API — all model weights are bundled in the container image under ADR-003.

§4 Chunking Strategy

Uniform token-window chunking — the LlamaIndex and LangChain default — is the correct answer for uniform prose and the wrong answer for the heterogeneous document types that enterprise knowledge bases actually contain. A 512-token window that splits a procurement SOP mid-procedure returns a chunk that begins at step 4 with no context for steps 1–3. That chunk retrieves correctly on keyword matching and generates an answer that sounds complete and is procedurally incomplete. QueryForge routes chunking strategy per content type: section-aware splitting for policy and legal documents (preserving §-boundaries), step-aware splitting for runbooks and SOPs (preserving step integrity), QA-pair preservation for FAQ content, and row-group chunking with header repetition for tabular data. Chunking config is versioned as YAML in Cloud Storage; the chunk version is stored as metadata on every Firestore document and returned in retrieval results so the experiment grid in §6.2 can isolate chunking strategy as an independent variable.

§5 Pipeline Simulator

Five reference scenarios — one per failure mode from §1 — scripted against the real stage timings and routing decisions the build produces. This is the front-end experience an end user gets when they call /v1/optimize: pick a scenario, run it, and watch the classifier's decision, the retrieval strategy it selects, and the fused result explain themselves in real time.

// select a scenario and run simulation

§6 Design Validation

Three views on whether the routing delivers measurable improvement: a before/after on a concrete query that illustrates why rank position matters more than whether a document is retrieved at all, the experiment grid that produced the adaptive-α defaults, and the MLOps loop that ensures config recommendations stay calibrated as the corpus and query distribution evolve. The numbers are real — derived from evaluation against a HotpotQA-equivalent internal corpus — and the methodology is reproducible: the same grid search the config recommender runs at deploy time.

The query from the README example — "What approval is required for vendor contracts over $50K with non-standard payment terms?" — against a baseline single dense embedding versus QueryForge's decompose+hybrid+RRF routing.

Both documents the answer actually depends on are retrieved by the baseline too — but at ranks 4 and 5, past most top-k=3 cutoffs used in production RAG. QueryForge's decomposition retrieves each concept ("approval threshold," "non-standard terms," "approval authority") independently, so RRF surfaces all three at the top instead of diluting them into one averaged embedding.

The α defaults in §3.2 are not engineering intuition — they are the output of a grid search over strategy × α ∈ {0.40, 0.45, 0.50, 0.55, 0.70} × reranker on/off, evaluated against a HotpotQA-equivalent internal corpus on three metrics: Recall@10, MRR, and p50 latency. This is the same grid QueryForge's config recommender runs at deploy time for a new corpus. The results below are what the recommender is trained to replicate: route conservatively on latency, aggressively on recall, and never pay the reranker tax for query types where the precision gain does not justify it.

Dense-only retrieval — the default that most enterprise RAG systems ship at launch and never revisit — achieves Recall@10 of 0.61. QueryForge's full pipeline reaches 0.92: a 31-percentage-point improvement that represents the difference between a system that misses 39% of required evidence on multi-hop queries and one that misses 8%. The reranker adds 400ms of latency for a 7-point recall gain over decompose+hybrid alone — the routing table applies it only to multi-hop and entity-scoped queries, where that tradeoff is justified. For single-hop queries, the system routes to dense-only in under 700ms and never pays costs that the query type doesn't warrant.

§7 Google Cloud Architecture

The architecture is Google-native by deliberate constraint, not default. Every component was selected because it fits inside the Always-Free monthly allotment of a shared billing account — and because the Always-Free boundary is hard: anything outside it risks a bill that could propagate to three other applications this project must never affect. The service map below is the production topology. The same container, the same configuration, and the same cost envelope that runs the demo is what production scale-out looks like — the graduation path replaces Firestore Vector Search with Vertex AI Vector Search and adds the paid Gemini tier, but the application code does not change.

The Cloud Run service account is scoped to least privilege: roles/datastore.user (Firestore read/write), roles/storage.objectViewer (corpus bucket, read-only), roles/secretmanager.secretAccessor (Gemini API key retrieval), and roles/run.invoker for authenticated callers. No third-party vector database, SaaS tool, or external hosting provider is in the request path — the only outbound network call from the container is to the Gemini Developer API. This matters for enterprise deployability: a system whose data path stays within a single vendor boundary is a fundamentally different compliance conversation than one that routes through multiple external services. It is worth being direct about the one genuine trade-off: on the Gemini Developer API free tier, Google may use inputs and outputs to improve its models. Enabling billing on the API opts out of that data use. For a $0.01-capped portfolio project, that is an accepted limitation. For an enterprise deployment handling confidential documents, it is the first thing to change — and ADR-004 says so plainly rather than burying it.

§8 Cost & Adoption Case

Two cost models, both grounded in cited sources: the business cost of the retrieval gap QueryForge closes, and the running cost of the Google Cloud solution itself. Every figure below is sourced — where a number is a rough estimate rather than a primary figure, that's stated plainly rather than dressed up as precision.

A hard cap only means something if it enforces itself before a bill is generated. The first design for this used a reactive Cloud Billing Budget → Pub/Sub → Cloud Function kill switch. Building it surfaced a problem: Google's own billing data lags by at least 24 hours, which makes any billing-data-driven trigger structurally too slow to catch a $0.01 overspend before it happens — by the time it fires, the overspend already occurred.

§9 Deployment

Single-command deploy to Cloud Run, followed immediately by the budget cap — the cap is treated as part of the deployment, not an optional afterthought.

# build + deploy the container, capped at 3 instances, 1Gi memory
gcloud run deploy queryforge \
  --source . \
  --region us-central1 \
  --max-instances 3 \
  --memory 1Gi \
  --allow-unauthenticated \
  --set-secrets GEMINI_API_KEY=gemini-api-key:latest

# hard budget cap — scoped to THIS project only
# does not touch the shared billing account or the other 3 apps on it
gcloud billing budgets create \
  --billing-account=$BILLING_ACCOUNT_ID \
  --display-name="queryforge-hard-cap" \
  --budget-amount=0.01USD \
  --threshold-rule=percent=1.0 \
  --filter-projects=projects/$QUERYFORGE_PROJECT_ID

§10 Architecture Decision Records

Architecture Decision Records are the unit of engineering judgment in this document. Each ADR below records a decision that had real consequences: what was chosen, what was seriously considered and rejected, the reasoning that separated them, and the consequences that were accepted. They are not post-hoc justifications for decisions already made on intuition. They are the working record of a design process that started with a technically sound multi-vendor stack, encountered two hard constraints — 100% Google Cloud, $0.01 ceiling on a shared billing account — and had to rebuild every layer from first principles. ADR-006 is included with its revision history intact because the original decision was implemented, found to be structurally inadequate, and replaced. That revision is not a failure to document. It is the point of the format.

§11 Data Schema

Firestore holds three collections. The schema is not incidental: corpus_chunks stores chunk version and effective date as indexed fields, which is what makes temporal queries retrievable via BM25 date-field boost rather than requiring a separate metadata store. query_logs is the drift-detection input for the MLOps loop in §6.3 — every routing decision, alpha value, sub-query set, and latency measurement is logged here, and this log is what feeds the next experiment cycle when query distribution shifts. All three collections fit inside the Always-Free daily read/write quota at demo scale; the graduation path to production replaces the Firestore vector index with Vertex AI Vector Search (see ADR-002) but keeps the same log schema.

// One document per chunk. `embedding` is indexed for Firestore Vector Search.
{
  "doc_id":          "policy_travel_v3_chunk_014",
  "corpus_id":       "acme-hr-corpus",
  "content_type":    "policy_docs",
  "chunk_strategy":  "section-aware",
  "chunk_version":   "v3",
  "text":            "§4.2 Non-standard payment terms require...",
  "embedding":       [0.0123, -0.0871, ...],   // all-MiniLM-L6-v2, 384-dim
  "metadata": {
    "effective_date": "2026-02-01",
    "source_uri":     "gs://acme-corpus/policy_travel_v3.pdf"
  }
}

// One document per /v1/optimize call.
{
  "query_id":        "qlog_00482",
  "query_text":      "What approval is required for vendor contracts over $50K...",
  "classifier_type": "multi-hop-entity",
  "confidence":      0.91,
  "sub_queries":     ["vendor contract approval threshold $50K", "..."],
  "alpha":           0.40,
  "reranked":        true,
  "latency_ms":      2340,
  "timestamp":       "2026-07-09T14:02:11Z"
}

§12 MVP Scope & Build Boundaries

Everything below runs inside the Always-Free tier and the $0.01 cap. Nothing here required a paid tier to demonstrate.

§13 Limitations & Known Issues

These are real limitations, not caveats added for completeness. Each one has a mitigation where one exists, and a clear statement of what the consequence is where it does not. An enterprise system that cannot articulate its own failure modes is a more dangerous system than one that can. The graduation path to production for each limitation is documented in §12.

§14 Glossary