RAG Pipeline LLM API Cost Estimator
Estimate monthly LLM costs for a Retrieval-Augmented Generation (RAG) pipeline. RAG pipelines have higher input token counts due to retrieved context.
Recommended Setup
Model
Claude Haiku 4.5
Anthropic
Monthly tokens
230M
200M in / 30M out
Estimated monthly cost
$350
Cost Comparison: All Cloud Models
Based on 200M input + 30M output tokens/month
| Model | Provider | Monthly cost |
|---|---|---|
| gpt-oss-20b | OpenAI | $6.70 |
| Gemma 4 26B A4B | $8.40 | |
| GLM 5.3 Flash | Zhipu AI (GLM) | $8.70 |
| Qwen3.7 Flash | Qwen (Alibaba) | $9.90 |
| Llama 3.1 8B (Groq) | Groq | $12.40 |
| Gemma 3 4B | $13.00 | |
| Gemma 3 12B | $14.50 | |
| Doubao Seed 2.0 Mini | Doubao (ByteDance) | $14.70 |
| Gemma 3n 4B | $15.60 | |
| Qwen3.5-Flash | Qwen (Alibaba) | $21.80 |
| GPT-5 Nano | OpenAI | $22.00 |
| DeepSeek V4 Flash | DeepSeek | $23.40 |
| GLM-4.7 Flash | Zhipu AI (GLM) | $24.00 |
| gpt-oss-safeguard-20b | OpenAI | $25.00 |
| DeepSeek V4.1 Flash | DeepSeek | $25.00 |
| Qwen2.5 7B Instruct | Qwen (Alibaba) | $26.00 |
| Gemma 3 27B | $26.50 | |
| Qwen3 30B A3B Instruct 2507 | Qwen (Alibaba) | $29.00 |
| Doubao Pro 32K | Doubao (ByteDance) | $30.40 |
| Hunyuan TurboS | Hunyuan (Tencent) | $30.40 |
| DeepSeek V4 Flash 0731 | DeepSeek | $31.20 |
| Qwen3 14B | Qwen (Alibaba) | $31.20 |
| GPT-4.1 Nano | OpenAI | $32.00 |
| Gemini 2.5 Flash-Lite | $32.00 | |
| Llama 4 Scout (Groq) | Groq | $32.20 |
| Qwen3 VL 32B Instruct | Qwen (Alibaba) | $32.60 |
| Qwen3 235B A22B Instruct 2507 | Qwen (Alibaba) | $34.50 |
| GPT-6 Luna Pro | OpenAI | $35.00 |
| GPT-6 Luna | OpenAI | $35.00 |
| Claude Haiku 5.5 | Anthropic | $35.00 |
| Qwen3.5-Flash | Qwen (Alibaba) | $37.50 |
| Qwen3 30B A3B | Qwen (Alibaba) | $39.00 |
| Qwen3.5-9B | Qwen (Alibaba) | $41.50 |
| Qwen3.8 Omni Flash | Qwen (Alibaba) | $44.10 |
| Qwen3.8 Flash | Qwen (Alibaba) | $44.10 |
| Hunyuan T1 | Hunyuan (Tencent) | $44.80 |
| Qwen3.6 35B A3B | Qwen (Alibaba) | $47.00 |
| gpt-oss-120b | OpenAI | $48.00 |
| GPT-4o-mini | OpenAI | $48.00 |
| GPT-OSS 120B (Groq) | Groq | $48.00 |
| Qwen3 Coder Next | Qwen (Alibaba) | $48.00 |
| Qwen3 VL 30B A3B Instruct | Qwen (Alibaba) | $48.00 |
| Qwen3 Coder 30B A3B Instruct | Qwen (Alibaba) | $48.00 |
| Qwen3 Next 80B A3B Instruct | Qwen (Alibaba) | $51.00 |
| GLM 4.5 Air | Zhipu AI (GLM) | $51.50 |
| Gemma 4 31B | $52.00 | |
| Qwen3.5-35B-A3B | Qwen (Alibaba) | $58.00 |
| DeepSeek V3.2 | DeepSeek | $63.40 |
| Qwen3 Next 80B A3B Thinking | Qwen (Alibaba) | $66.00 |
| Qwen3 Coder Flash | Qwen (Alibaba) | $69.40 |
| Qwen3.6 Flash | Qwen (Alibaba) | $71.90 |
| Qwen3 VL 8B Instruct | Qwen (Alibaba) | $72.50 |
| Qwen3.5-Plus | Qwen (Alibaba) | $75.40 |
| Qwen3 32B (Groq) | Groq | $75.70 |
| GPT-5.6 Luna | OpenAI | $76.00 |
| GPT-5.6 Luna Pro | OpenAI | $76.00 |
| GPT-5.4 Nano | OpenAI | $77.50 |
| DeepSeek V3.1 | DeepSeek | $78.50 |
| DeepSeek V3 | DeepSeek | $82.90 |
| DeepSeek V3.1 Terminus | DeepSeek | $84.00 |
| Qwen2.5 72B Instruct | Qwen (Alibaba) | $84.00 |
| Qwen3.8 27B | Qwen (Alibaba) | $86.40 |
| DeepSeek V3 (Mar 2025) | DeepSeek | $87.00 |
| GLM 4.6V | Zhipu AI (GLM) | $87.00 |
| Qwen3 VL 30B A3B Thinking | Qwen (Alibaba) | $88.00 |
| Qwen3 Coder 480B A35B | Qwen (Alibaba) | $90.00 |
| DeepSeek V3 0324 | DeepSeek | $92.20 |
| Gemini 3.1 Flash-Lite | $95.00 | |
| Nano Banana 2 Lite (Gemini 3.1 Flash Lite Image) | $95.00 | |
| DeepSeek V4.1 Flash 1017 | DeepSeek | $96.00 |
| Qwen3.5 Plus 2026-02-15 | Qwen (Alibaba) | $98.80 |
| Qwen3 VL 235B A22B Instruct | Qwen (Alibaba) | $99.00 |
| Qwen3 VL 8B Thinking | Qwen (Alibaba) | $99.00 |
| Qwen3.7 Plus | Qwen (Alibaba) | $102 |
| GPT-5 Mini | OpenAI | $110 |
| GPT-5.1-Codex-Mini | OpenAI | $110 |
| GLM 5.3 FlashX | Zhipu AI (GLM) | $112 |
| Qwen3 30B A3B Thinking 2507 | Qwen (Alibaba) | $112 |
| Qwen3.5-122B-A10B | Qwen (Alibaba) | $114 |
| GLM 5.3 | Zhipu AI (GLM) | $124 |
| Qwen3.6 Plus | Qwen (Alibaba) | $125 |
| GPT-4.1 Mini | OpenAI | $128 |
| Qwen3.5-27B | Qwen (Alibaba) | $130 |
| Gemini 2.5 Flash | $135 | |
| Gemini 3.5 Flash Lite | $135 | |
| Nano Banana (Gemini 2.5 Flash Image) | $135 | |
| Llama 3.3 70B (Groq) | Groq | $142 |
| DeepSeek V4 Pro 0813 | DeepSeek | $144 |
| Qwen3-Max | Qwen (Alibaba) | $145 |
| GLM 4.6 | Zhipu AI (GLM) | $160 |
| Qwen2.5 Coder 32B Instruct | Qwen (Alibaba) | $162 |
| R1 0528 | DeepSeek | $165 |
| Doubao Seed 2.0 Pro | Doubao (ByteDance) | $165 |
| Qwen3.6 27B | Qwen (Alibaba) | $171 |
| GLM 4.5V | Zhipu AI (GLM) | $174 |
| Kimi K2.5 | Kimi (Moonshot AI) | $175 |
| Kimi K2 | Kimi (Moonshot AI) | $183 |
| Kimi K2.5 (Together) | Together AI | $184 |
| R1 Distill Llama 70B | DeepSeek | $184 |
| GLM 4.7 | Zhipu AI (GLM) | $186 |
| GLM 4.5 | Zhipu AI (GLM) | $186 |
| Nano Banana 2 (Gemini 3.1 Flash Image) | $190 | |
| Qwen2.5 VL 72B Instruct | Qwen (Alibaba) | $190 |
| GPT Audio Mini | OpenAI | $192 |
| Qwen3 235B A22B Thinking 2507 | Qwen (Alibaba) | $195 |
| Kimi K2 Thinking | Kimi (Moonshot AI) | $195 |
| Kimi K2 0905 | Kimi (Moonshot AI) | $195 |
| Qwen3 VL 235B A22B Thinking | Qwen (Alibaba) | $200 |
| Llama 3.3 70B (Together) | Together AI | $202 |
| Qwen3.5 397B A17B | Qwen (Alibaba) | $208 |
| DeepSeek R1 | DeepSeek | $215 |
| Qwen3 Coder Plus | Qwen (Alibaba) | $228 |
| Qwen3.5 397B (Together) | Together AI | $228 |
| Gemini 3.6 Flash | $263 | |
| Gemini 3.7 Flash | $263 | |
| Qwen3 Max Thinking | Qwen (Alibaba) | $273 |
| Claude 3.5 Haiku | Anthropic | $280 |
| GPT-5.4 Mini | OpenAI | $285 |
| Gemini 3.8 Flash | $290 | |
| GLM-5 | Zhipu AI (GLM) | $296 |
| Kimi K2.7 Code | Kimi (Moonshot AI) | $310 |
| Kimi K2.6 | Kimi (Moonshot AI) | $310 |
| GLM-5.2 | Zhipu AI (GLM) | $337 |
| GPT-6.1 Terra Pro | OpenAI | $350 |
| GPT-6.1 Terra | OpenAI | $350 |
| Claude Haiku 4.5★ recommended | Anthropic | $350 |
| o4-mini | OpenAI | $352 |
| o3 Mini | OpenAI | $352 |
| o4 Mini High | OpenAI | $352 |
| o3 Mini High | OpenAI | $352 |
| GLM-5-Turbo | Zhipu AI (GLM) | $360 |
| GLM 5V Turbo | Zhipu AI (GLM) | $360 |
| GLM-5.1 | Zhipu AI (GLM) | $412 |
| DeepSeek V4 Pro | DeepSeek | $416 |
| Qwen3.7 Max | Qwen (Alibaba) | $429 |
| Nano Banana 2.1 | $525 | |
| GPT-5 | OpenAI | $550 |
| GPT-5 Codex | OpenAI | $550 |
| GPT-5.1-Codex-Max | OpenAI | $550 |
| GPT-5.1 | OpenAI | $550 |
| GPT-5.1-Codex | OpenAI | $550 |
| Gemini 2.5 Pro | $550 | |
| Gemini 2.5 Pro Preview 05-06 | $550 | |
| Moonshot V1 (128K) | Kimi (Moonshot AI) | $550 |
| DeepSeek V4 Pro (Together) | Together AI | $552 |
| GPT-5 Image Mini | OpenAI | $560 |
| Gemini 3.5 Flash | $570 | |
| Qwen3.8 2.4T A95B | Qwen (Alibaba) | $580 |
| Qwen3.8 Max | Qwen (Alibaba) | $580 |
| Qwen3.8 Max (0902) | Qwen (Alibaba) | $580 |
| Kimi K2.7 Code HighSpeed | Kimi (Moonshot AI) | $620 |
| GPT-4.1 | OpenAI | $640 |
| o3 | OpenAI | $640 |
| o4 Mini Deep Research | OpenAI | $640 |
| GPT-6.1 Sol Pro | OpenAI | $700 |
| GPT-6.1 Sol | OpenAI | $700 |
| GPT-6 Sol Pro | OpenAI | $700 |
| GPT-6 Sol | OpenAI | $700 |
| Claude Sonnet 5 | Anthropic | $700 |
| Claude Sonnet 5.5 | Anthropic | $700 |
| Gemini 4 Argon | $700 | |
| Kimi K3.1 Pro | Kimi (Moonshot AI) | $700 |
| GPT-5.6 Terra | OpenAI | $760 |
| GPT-5.6 Terra Pro | OpenAI | $760 |
| Gemini 3.1 Pro Preview | $760 | |
| Nano Banana Pro (Gemini 3 Pro Image) | $760 | |
| Gemini 3.1 Pro Preview Custom Tools | $760 | |
| GPT-5.3 Chat | OpenAI | $770 |
| GPT-5.3-Codex | OpenAI | $770 |
| GPT-5.2-Codex | OpenAI | $770 |
| GPT-5.2 Chat | OpenAI | $770 |
| GPT-5.2 | OpenAI | $770 |
| GPT-4o | OpenAI | $800 |
| GPT Audio | OpenAI | $800 |
| GLM 5.3 Prime | Zhipu AI (GLM) | $824 |
| GPT-5.4 | OpenAI | $950 |
| Claude Sonnet 4.6 | Anthropic | $1,050 |
| Claude Sonnet 4.5 | Anthropic | $1,050 |
| Claude Sonnet 4 | Anthropic | $1,050 |
| Kimi K3 | Kimi (Moonshot AI) | $1,050 |
| Qwen3.8 Max Prime | Qwen (Alibaba) | $1,160 |
| GPT-5.6 Sol Pro | OpenAI | $1,400 |
| Claude Opus 5.5 | Anthropic | $1,400 |
| Claude Opus 4.7 | Anthropic | $1,750 |
| Claude Opus 4.6 | Anthropic | $1,750 |
| Claude Opus 4.8 | Anthropic | $1,750 |
| Claude Opus 4.5 | Anthropic | $1,750 |
| Claude Opus 5 | Anthropic | $1,750 |
| GPT-5.6 Sol | OpenAI | $1,900 |
| GPT-5.5 | OpenAI | $1,900 |
| GPT-5.4 Image 2 | OpenAI | $2,050 |
| GPT-5 Image | OpenAI | $2,300 |
| o3 Deep Research | OpenAI | $3,200 |
| GPT-6 Astra | OpenAI | $3,500 |
| GPT-6 Astra Pro | OpenAI | $3,500 |
| Claude Opus 4.8 (Fast) | Anthropic | $3,500 |
| Claude Opus 5 (Fast) | Anthropic | $3,500 |
| Claude Fable 5 | Anthropic | $3,500 |
| Claude Fable 5.1 | Anthropic | $3,500 |
| o1 | OpenAI | $4,800 |
| Claude Opus 4.1 | Anthropic | $5,250 |
| Claude Opus 4 | Anthropic | $5,250 |
| o3 Pro | OpenAI | $6,400 |
| GPT-5 Pro | OpenAI | $6,600 |
| GPT-5.2 Pro | OpenAI | $9,240 |
| Claude Opus 4.7 (Fast) | Anthropic | $10,500 |
| Claude Opus 4.6 (Fast) | Anthropic | $10,500 |
| GPT-5.5 Pro | OpenAI | $11,400 |
| GPT-5.4 Pro | OpenAI | $11,400 |
| o1-pro | OpenAI | $48,000 |
Frequently Asked Questions
Why are RAG pipeline token costs higher than regular chatbots?
RAG injects retrieved document chunks into every prompt. Each retrieval adds 500–2,000 tokens of context. A moderate RAG system serving 5,000 queries/day can consume 100–500M input tokens per month.
Which LLM is best for RAG pipelines?
Compare current models by context window, retrieval quality, and total input and output cost. Test representative documents before choosing a model.
Should I self-host the LLM for my RAG pipeline?
If your RAG pipeline consumes 500M+ tokens/month, a self-hosted A100 or two RTX 4090s may become cost-competitive. Use this calculator to find your break-even point.