RAG Pipeline LLM API Cost Estimator
Estimate monthly LLM costs for a Retrieval-Augmented Generation (RAG) pipeline. RAG pipelines have higher input token counts due to retrieved context.
Recommended Setup
Model
Claude Haiku 4.5
Anthropic
Monthly tokens
230M
200M in / 30M out
Estimated monthly cost
$350
Cost Comparison: All Cloud Models
Based on 200M input + 30M output tokens/month
| Model | Provider | Monthly cost |
|---|---|---|
| gpt-oss-20b | OpenAI | $9.90 |
| Qwen3.7 Flash | Qwen (Alibaba) | $9.90 |
| Llama 3.1 8B (Groq) | Groq | $12.40 |
| Gemma 3 4B | $13.00 | |
| gpt-oss-120b | OpenAI | $13.10 |
| Gemma 3 12B | $14.50 | |
| Doubao Seed 2.0 Mini | Doubao (ByteDance) | $14.70 |
| Gemma 3n 4B | $15.60 | |
| Qwen3 30B A3B Instruct 2507 | Qwen (Alibaba) | $15.70 |
| Qwen3.5-Flash | Qwen (Alibaba) | $21.80 |
| GPT-5 Nano | OpenAI | $22.00 |
| Qwen3 Coder 30B A3B Instruct | Qwen (Alibaba) | $22.40 |
| DeepSeek V4 Flash | DeepSeek | $23.10 |
| GLM-4.7 Flash | Zhipu AI (GLM) | $24.00 |
| Gemma 4 26B A4B | $24.20 | |
| Qwen3.5-9B | Qwen (Alibaba) | $24.50 |
| gpt-oss-safeguard-20b | OpenAI | $25.00 |
| Qwen2.5 7B Instruct | Qwen (Alibaba) | $26.00 |
| Gemma 3 27B | $29.50 | |
| Gemma 4 31B | $30.20 | |
| Doubao Pro 32K | Doubao (ByteDance) | $30.40 |
| Hunyuan TurboS | Hunyuan (Tencent) | $30.40 |
| Qwen3 14B | Qwen (Alibaba) | $31.20 |
| GPT-4.1 Nano | OpenAI | $32.00 |
| Gemini 2.5 Flash-Lite | $32.00 | |
| Llama 4 Scout (Groq) | Groq | $32.20 |
| Qwen3 VL 32B Instruct | Qwen (Alibaba) | $32.60 |
| Qwen3 235B A22B Instruct 2507 | Qwen (Alibaba) | $34.50 |
| DeepSeek V4 Flash 0731 | DeepSeek | $36.40 |
| Qwen3.5-Flash | Qwen (Alibaba) | $37.50 |
| Qwen3 VL 8B Instruct | Qwen (Alibaba) | $37.50 |
| Qwen3 30B A3B | Qwen (Alibaba) | $39.00 |
| Qwen3 VL 30B A3B Instruct | Qwen (Alibaba) | $41.60 |
| Hunyuan T1 | Hunyuan (Tencent) | $44.80 |
| GPT-4o-mini | OpenAI | $48.00 |
| GPT-OSS 120B (Groq) | Groq | $48.00 |
| Qwen3 Coder Next | Qwen (Alibaba) | $48.00 |
| GLM 4.5 Air | Zhipu AI (GLM) | $51.50 |
| Qwen3 Next 80B A3B Instruct | Qwen (Alibaba) | $53.00 |
| Qwen3.6 35B A3B | Qwen (Alibaba) | $58.00 |
| DeepSeek V3.2 | DeepSeek | $63.40 |
| Qwen3 Next 80B A3B Thinking | Qwen (Alibaba) | $66.00 |
| Qwen3 Coder Flash | Qwen (Alibaba) | $69.40 |
| Qwen3.6 Flash | Qwen (Alibaba) | $71.90 |
| Qwen3.5-Plus | Qwen (Alibaba) | $75.40 |
| Qwen3 32B (Groq) | Groq | $75.70 |
| GPT-5.6 Luna | OpenAI | $76.00 |
| GPT-5.6 Luna Pro | OpenAI | $76.00 |
| GPT-5.4 Nano | OpenAI | $77.50 |
| DeepSeek V3 0324 | DeepSeek | $80.00 |
| DeepSeek V3 | DeepSeek | $82.90 |
| DeepSeek V3.1 Terminus | DeepSeek | $84.00 |
| Qwen2.5 72B Instruct | Qwen (Alibaba) | $84.00 |
| Qwen3.5-27B | Qwen (Alibaba) | $86.80 |
| DeepSeek V3 (Mar 2025) | DeepSeek | $87.00 |
| GLM 4.6V | Zhipu AI (GLM) | $87.00 |
| Qwen3.5-35B-A3B | Qwen (Alibaba) | $87.50 |
| Qwen3 Coder 480B A35B | Qwen (Alibaba) | $90.00 |
| Gemini 3.1 Flash-Lite | $95.00 | |
| Nano Banana 2 Lite (Gemini 3.1 Flash Lite Image) | $95.00 | |
| Qwen3.5 Plus 2026-02-15 | Qwen (Alibaba) | $98.80 |
| Qwen3 VL 235B A22B Instruct | Qwen (Alibaba) | $99.00 |
| Qwen3 VL 8B Thinking | Qwen (Alibaba) | $99.00 |
| Qwen3.7 Plus | Qwen (Alibaba) | $102 |
| GPT-5 Mini | OpenAI | $110 |
| GPT-5.1-Codex-Mini | OpenAI | $110 |
| Qwen3 VL 30B A3B Thinking | Qwen (Alibaba) | $112 |
| Qwen3 30B A3B Thinking 2507 | Qwen (Alibaba) | $112 |
| Qwen3.5-122B-A10B | Qwen (Alibaba) | $114 |
| Qwen3 235B A22B Thinking 2507 | Qwen (Alibaba) | $115 |
| Qwen3.6 Plus | Qwen (Alibaba) | $125 |
| GPT-4.1 Mini | OpenAI | $128 |
| Gemini 3.7 Flash | $132 | |
| GLM 4.7 | Zhipu AI (GLM) | $133 |
| Gemini 2.5 Flash | $135 | |
| Gemini 3.5 Flash Lite | $135 | |
| Nano Banana (Gemini 2.5 Flash Image) | $135 | |
| Llama 3.3 70B (Groq) | Groq | $142 |
| Qwen3-Max | Qwen (Alibaba) | $145 |
| DeepSeek V4 Pro | DeepSeek | $151 |
| DeepSeek V3.1 | DeepSeek | $160 |
| Qwen3.6 27B | Qwen (Alibaba) | $160 |
| GLM 4.6 | Zhipu AI (GLM) | $160 |
| Qwen2.5 Coder 32B Instruct | Qwen (Alibaba) | $162 |
| Qwen3.8 27B | Qwen (Alibaba) | $163 |
| R1 0528 | DeepSeek | $165 |
| Doubao Seed 2.0 Pro | Doubao (ByteDance) | $165 |
| GLM 4.5V | Zhipu AI (GLM) | $174 |
| Kimi K2 | Kimi (Moonshot AI) | $183 |
| Kimi K2.5 (Together) | Together AI | $184 |
| R1 Distill Llama 70B | DeepSeek | $184 |
| GLM 4.5 | Zhipu AI (GLM) | $186 |
| Nano Banana 2 (Gemini 3.1 Flash Image) | $190 | |
| Qwen2.5 VL 72B Instruct | Qwen (Alibaba) | $190 |
| GPT Audio Mini | OpenAI | $192 |
| Kimi K2 Thinking | Kimi (Moonshot AI) | $195 |
| Kimi K2 0905 | Kimi (Moonshot AI) | $195 |
| Qwen3 VL 235B A22B Thinking | Qwen (Alibaba) | $200 |
| Llama 3.3 70B (Together) | Together AI | $202 |
| Qwen3.5 397B A17B | Qwen (Alibaba) | $208 |
| Kimi K2.5 | Kimi (Moonshot AI) | $210 |
| DeepSeek R1 | DeepSeek | $215 |
| Qwen3 Coder Plus | Qwen (Alibaba) | $228 |
| Qwen3.5 397B (Together) | Together AI | $228 |
| Gemini 3.6 Flash | $263 | |
| Qwen3 Max Thinking | Qwen (Alibaba) | $273 |
| Claude 3.5 Haiku | Anthropic | $280 |
| GPT-5.4 Mini | OpenAI | $285 |
| GLM-5 | Zhipu AI (GLM) | $296 |
| Kimi K2.7 Code | Kimi (Moonshot AI) | $310 |
| Kimi K2.6 | Kimi (Moonshot AI) | $310 |
| DeepSeek V4 Pro 0813 | DeepSeek | $325 |
| Claude Haiku 4.5★ recommended | Anthropic | $350 |
| o4-mini | OpenAI | $352 |
| o3 Mini | OpenAI | $352 |
| o4 Mini High | OpenAI | $352 |
| o3 Mini High | OpenAI | $352 |
| GLM-5-Turbo | Zhipu AI (GLM) | $360 |
| GLM 5V Turbo | Zhipu AI (GLM) | $360 |
| GLM-5.1 | Zhipu AI (GLM) | $412 |
| GLM 5.3 | Zhipu AI (GLM) | $412 |
| Qwen3.7 Max | Qwen (Alibaba) | $429 |
| GPT-5 | OpenAI | $550 |
| GPT-5 Codex | OpenAI | $550 |
| GPT-5.1-Codex-Max | OpenAI | $550 |
| GPT-5.1 | OpenAI | $550 |
| GPT-5.1-Codex | OpenAI | $550 |
| Gemini 2.5 Pro | $550 | |
| Gemini 2.5 Pro Preview 05-06 | $550 | |
| Moonshot V1 (128K) | Kimi (Moonshot AI) | $550 |
| DeepSeek V4 Pro (Together) | Together AI | $552 |
| GPT-5 Image Mini | OpenAI | $560 |
| Gemini 3.5 Flash | $570 | |
| Qwen3.8 2.4T A95B | Qwen (Alibaba) | $580 |
| Qwen3.8 Max | Qwen (Alibaba) | $580 |
| Kimi K2.7 Code HighSpeed | Kimi (Moonshot AI) | $620 |
| GPT-4.1 | OpenAI | $640 |
| o3 | OpenAI | $640 |
| o4 Mini Deep Research | OpenAI | $640 |
| GPT-5.6 Sol Pro | OpenAI | $700 |
| Claude Sonnet 5 | Anthropic | $700 |
| GPT-5.6 Terra | OpenAI | $760 |
| GPT-5.6 Terra Pro | OpenAI | $760 |
| Gemini 3.1 Pro Preview | $760 | |
| Nano Banana Pro (Gemini 3 Pro Image) | $760 | |
| Gemini 3.1 Pro Preview Custom Tools | $760 | |
| GPT-5.3 Chat | OpenAI | $770 |
| GPT-5.3-Codex | OpenAI | $770 |
| GPT-5.2-Codex | OpenAI | $770 |
| GPT-5.2 Chat | OpenAI | $770 |
| GPT-5.2 | OpenAI | $770 |
| GPT-4o | OpenAI | $800 |
| GPT Audio | OpenAI | $800 |
| GPT-5.4 | OpenAI | $950 |
| Claude Sonnet 4.6 | Anthropic | $1,050 |
| Claude Sonnet 4.5 | Anthropic | $1,050 |
| Claude Sonnet 4 | Anthropic | $1,050 |
| Kimi K3 | Kimi (Moonshot AI) | $1,050 |
| Claude Opus 4.7 | Anthropic | $1,750 |
| Claude Opus 4.6 | Anthropic | $1,750 |
| Claude Opus 4.8 | Anthropic | $1,750 |
| Claude Opus 4.5 | Anthropic | $1,750 |
| Claude Opus 5 | Anthropic | $1,750 |
| GPT-5.6 Sol | OpenAI | $1,900 |
| GPT-5.5 | OpenAI | $1,900 |
| GPT-5.4 Image 2 | OpenAI | $2,050 |
| GPT-5 Image | OpenAI | $2,300 |
| GLM-5.2 | Zhipu AI (GLM) | $2,440 |
| o3 Deep Research | OpenAI | $3,200 |
| Claude Opus 4.8 (Fast) | Anthropic | $3,500 |
| Claude Opus 5 (Fast) | Anthropic | $3,500 |
| Claude Fable 5 | Anthropic | $3,500 |
| o1 | OpenAI | $4,800 |
| Claude Opus 4.1 | Anthropic | $5,250 |
| Claude Opus 4 | Anthropic | $5,250 |
| o3 Pro | OpenAI | $6,400 |
| GPT-5 Pro | OpenAI | $6,600 |
| GPT-5.2 Pro | OpenAI | $9,240 |
| Claude Opus 4.7 (Fast) | Anthropic | $10,500 |
| Claude Opus 4.6 (Fast) | Anthropic | $10,500 |
| GPT-5.5 Pro | OpenAI | $11,400 |
| GPT-5.4 Pro | OpenAI | $11,400 |
| o1-pro | OpenAI | $48,000 |
Frequently Asked Questions
Why are RAG pipeline token costs higher than regular chatbots?
RAG injects retrieved document chunks into every prompt. Each retrieval adds 500–2,000 tokens of context. A moderate RAG system serving 5,000 queries/day can consume 100–500M input tokens per month.
Which LLM is best for RAG pipelines?
Claude Haiku 4.5 ($1/1M input) and GPT-5.4 Nano ($0.20/1M) are popular choices. For RAG with very long context windows, Gemini 3.5 Flash ($1.50/1M) supports 1M tokens per request and has excellent price/performance. For budget-focused RAG, Gemini 2.5 Flash-Lite ($0.10/1M) is the cheapest option with 1M context.
Should I self-host the LLM for my RAG pipeline?
If your RAG pipeline consumes 500M+ tokens/month, a self-hosted A100 or two RTX 4090s may become cost-competitive. Use this calculator to find your break-even point.