Can we combine RAG and fine-tuning?

Yes. Start RAG-only; add fine-tuning when evals plateau.

Master RAG and eval gates before multi-step agent autonomy.

How does pricing interact with the choice?

RAG adds retrieval cost; fine-tuning adds training and drift monitoring.

RAG vs Fine-Tuning vs Prompts: A PM Decision Framework

Atiq Israk has shipped AI on AssetIQ (RFID + inventory truth, 264% revenue growth) and enterprise workflows at Toyota/Navana ($40M saved). This framework helps PMs choose RAG, fine-tuning, or prompts without defaulting to the loudest vendor pitch.

Key Takeaways

Start with prompts + RAG on canonical data; fine-tune only when behavior must be stable and you have hundreds of labeled examples.

The decision hinges on data you control, change frequency, and eval pass rates—not model marketing.

Most B2B operator products win with RAG + tight scope before any custom training budget.

What Are the Three Options?

Prompt engineering shapes instructions and output format in the model call. RAG (retrieval) injects fresh documents at query time. Fine-tuning adapts model weights on your labeled dataset. Each solves a different failure mode.

When Should PMs Use Prompts Only?

Use prompts alone when:

The task is format transformation (summarize, classify intent) on text already in the request.
You can eval quality with <50 examples and iterate daily.
Ground truth changes weekly—no time to retrain.

Navbot's first intent classifier started here before retrieval-backed answers scaled.

Developer reviewing ML architecture options — Most B2B wins start with prompts and retrieval—not custom training.

When Should PMs Use RAG?

Use RAG when answers must reflect your inventory, policies, or menus—and those sources update often. AssetIQ needed live SKU and location truth; marketing copy in a fine-tuned model would rot immediately. RAG plus eval gates on retrieval quality drove 264% revenue growth because operators trusted reconciliation.

Pair with context engineering: canonical sources, freshness SLAs, exclusion lists.

Signal	Prefer	Example
Truth lives in docs/DB	RAG	AssetIQ SKU lookup
Stable tone/format, small domain	Prompts → maybe fine-tune	Kaizen brand voice wedge
Thousands of labeled pairs, stable task	Fine-tune	Repeat classification at scale
Can't maintain clean corpus	Fix data ops first	Any RAG project

When Is Fine-Tuning Worth It?

Fine-tune when:

You have 500+ high-quality labeled examples (not scraped noise).
The task definition is stable for 6+ months.
Prompt + RAG cannot hit eval threshold at acceptable latency/cost.
Legal/compliance requires a fixed behavior envelope.

Most teams fine-tune too early. Run the eval suite first; if RAG retrieval fails, fix context before training.

Whiteboard with decision flowchart — Four questions in one meeting beat weeks of model debate.

How Do You Decide in One Meeting?

Ask four questions:

Where does truth live—and do we control it?
How often does truth change?
What eval pass rate do we need to ship?
What is cost per successful task at pilot volume?

Document the answer in the PRD/eval doc. This is Build the Machine: smallest approach that beats manual on the metric.

Explore: AI product management · case studies

Atiq Israk

Atiq Israk