Generative AI Implementation
A practitioner's guide to RAG consulting in Australia: what breaks between a slick demo and a production retrieval-augmented generation system, and a pragmatic path from POC to production.
Quantum Associates — Quantum Associates
· 8 min read
The demo always works. A retrieval-augmented generation prototype that answers questions over your policy library or contract set can be stood up in an afternoon, and it will impress a steering committee. Then you try to put it in front of two thousand staff, or a regulated business line, and the wheels come off. This is the uncomfortable truth of rag consulting australia clients keep rediscovering: the hard part was never the demo. It was everything the demo quietly ignored.
The summary you can act on: a RAG proof-of-concept is roughly 10 per cent of the work. The other 90 per cent is retrieval quality, evaluation, freshness, access control, latency, cost and observability — and none of it shows up until real users and real data arrive. If your delivery plan treats RAG as “wire the LLM to the vector database”, you are budgeting for the easy 10 per cent and hoping the rest is free. It is not.
This piece is about RAG as a delivery capability, not a concept. If you want the conceptual grounding — when retrieval beats fine-tuning beats prompting — we have written that up separately in RAG vs fine-tuning vs prompting. Here I want to walk through what actually breaks on the road to production, and how we sequence the work.
A prototype is built on a friendly slice of data, tested with questions the builder already knows the answers to, and judged on vibes. Production is the opposite on every axis. The corpus is messier and larger, the questions are ones nobody anticipated, the users have varying permissions, and “close enough” answers become audit findings or customer complaints.
The gap is not a single flaw you can patch. It is a set of engineering disciplines that the demo let you skip. Getting a RAG system into production means confronting each one deliberately.
The generation model gets the glory, but in a RAG system the retriever decides whether the answer is even possible. If the right passage never makes it into the model’s context window, no amount of prompt engineering will save you. Most disappointing RAG systems are not suffering from a weak model — they are suffering from weak retrieval.
Retrieval quality comes down to unglamorous decisions:
None of this is exotic. It is just work that a two-day POC has no reason to do, and that a production system cannot survive without.
The single biggest maturity marker in a RAG programme is whether it has an evaluation harness. Without one, every change is a guess and every “it seems better now” is folklore. With one, you can move fast because you can see regressions.
A workable evaluation approach has a few layers. Build a representative set of real questions with known-good answers, drawn from actual users, not the builder’s imagination. Measure retrieval separately from generation — did we fetch the right passages, and then did we answer faithfully from them? Track faithfulness (is the answer grounded in the retrieved sources, or invented?) as distinctly from relevance. And treat any “LLM-as-judge” scoring as a useful instrument that itself needs validating against human judgement, not as ground truth.
This is the discipline that separates a system you can safely change from one everyone is afraid to touch. If a vendor cannot show you how they will measure quality, they are asking you to fly blind.
Your knowledge base is not a snapshot; it changes daily. Policies get superseded, prices move, contracts get amended. A RAG system that indexed the corpus once and never again will confidently cite last quarter’s rate card. In regulated settings, answering from a withdrawn policy is not a quirk — it is a compliance event.
Production RAG needs an ingestion pipeline, not a one-off import: a way to detect changed source documents, re-chunk and re-embed only what changed, retire superseded content so it cannot be retrieved, and do all of it on a schedule that matches how fast your business actually moves. Deciding that cadence — real-time, hourly, nightly — is a business conversation about the cost of a stale answer, and it belongs in the design phase, not as a surprise after go-live.
Here is the failure mode that keeps executives awake once they understand it. A naive RAG system retrieves from the whole corpus regardless of who is asking. So an HR document about an executive’s remuneration, a legal matter, or another customer’s records can surface to a user who should never see it — and the model will present it helpfully and fluently. The system becomes a very efficient engine for leaking data across permission boundaries.
The fix is that retrieval must respect the user’s identity and entitlements, not just the application’s. In practice that means filtering the candidate set by the asking user’s permissions before anything reaches the model, carrying document-level access metadata through the whole pipeline, and never relying on the prompt to instruct the model to “not reveal” restricted content — a prompt instruction is not a security control. Getting this right usually means integrating with your existing identity and entitlement model, which is precisely the kind of enterprise plumbing a POC skips. It is also the single most important thing to get right before you widen the audience.
Three practical realities arrive with real traffic.
Latency. Retrieval, re-ranking and generation each add time, and users abandon slow tools. There are levers — caching frequent queries, tightening how many passages you retrieve, choosing a model sized to the task rather than the largest available — but they are trade-offs against quality and cost that need to be made on purpose.
Cost. RAG cost scales with usage in ways a fixed-price pilot hides. Every query consumes tokens for the retrieved context plus the answer; a chatty system over a large corpus can be startlingly expensive at scale. Model choice and how much context you stuff into each call are the dominant drivers. Model buy-versus-build decisions have a direct bearing here, and we have written about that trade-off in build vs buy vs fine-tune.
Observability. In production you must be able to answer “why did it say that?” — which sources were retrieved, what the user asked, what the model returned, and where it went wrong. Logging the full chain (with appropriate privacy controls) is what lets you debug complaints, catch quality drift, and satisfy an auditor. A RAG system you cannot inspect is one you cannot govern.
For Australian organisations, two questions matter more than the architecture diagram: where does the vector store live, and where does the model run? Your source documents — often commercially sensitive or containing personal information — get embedded and stored in the vector database, and the retrieved passages are sent to the model at query time. Both are points where data can leave the jurisdiction or the tenancy you assumed it stayed in.
The governance work is to be deliberate about hosting region for the vector store, the deployment region and tenancy of the model (several major providers now offer Australian or in-tenancy options, but you have to choose them), and the contractual and privacy posture around anything sent to a third-party API. For entities handling personal information this ties directly to Privacy Act obligations, and for regulated sectors it intersects with the expectations around outsourcing and data. Treat residency as a design constraint from day one, because retrofitting it after you have embedded the whole corpus is expensive and disruptive.
The way we sequence a RAG engagement reflects everything above:
This is the same discipline we bring across our generative AI services, and it is why a scoped, evaluated generative AI pilot is a far safer first step than a big-bang build. The pilot exists precisely to surface the 90 per cent before you have spent the 90 per cent.
RAG is one of the highest-value patterns in enterprise AI right now — and one of the easiest to get dangerously wrong, because the demo hides the risk. If you are weighing a RAG initiative and want an honest read on scope, residency and what production will actually demand, get in touch. We would rather tell you where it breaks before you build it than after.
Related insights
Generative AI Implementation
A decision framework for when retrieval-augmented generation earns its keep, when fine-tuning is the right answer, and when prompting alone is enough. With the cost-and-latency reality of each in 2026.
Generative AI Implementation
When to use an off-the-shelf AI product, when to build your own integration, when to fine-tune — with the trade-offs that matter at AU mid-market scale.
Next step
30 minutes, no pitch, no deck — just a working conversation about how this applies to your situation.