# LLM Inference Calculator API

> Estimate whether an LLM fits on given hardware, how fast it serves, what it costs, and which hardware meets a target. One GET request returns the whole answer.

## How to use

- AI agents: build the request yourself and send it with curl (or any HTTP GET) from a shell, for example curl "https://inference.emirtuncer.com/api/calc?gpu=b200&count=8&replicas=7&model=glm-5.3&users=1024&format=md". Web-reading tools often refuse URLs they built or silently drop everything after "?"; if /api/calc answers "no parameters received", that is what happened.
- Recommended: put the parameters in the PATH. Some browsing tools drop query strings from URLs they build, but keep paths. https://inference.emirtuncer.com/api/calc/g=b200/n=8/m=glm-5.3/u=1024.md returns the answer as Markdown; drop the .md for JSON, or use .txt for plain text.
- Path form: one key=value segment per parameter after /api/calc/, e.g. /api/calc/g=b200/n=8/r=7/m=glm-5.3/u=1024. A trailing .md, .txt or .json on the last segment (or a format=md segment) picks the format; other dots, as in glm-5.3, are part of the value. A "/" inside a value must be percent-encoded as %2F (meta-llama%2FLlama-3.1-8B-Instruct). Comma lists (g=h100-sxm,h200-sxm) and long names (gpu=b200/count=8) work as in the query form. A segment without "=" is reported in input.ignored.
- The query form is equivalent: https://inference.emirtuncer.com/api/calc?g=b200&n=8&m=glm-5.3&u=1024&format=md. If both are given, path parameters win.
- The same parameters on https://inference.emirtuncer.com/p/<parameters> (or https://inference.emirtuncer.com/?<parameters>) are a human page that also contains the full report as plain HTML text, so it can be shared and read without JavaScript.
- Read verdict.headline first: it answers the question in one sentence. verdict.status is ok, warn or fail.
- To try an alternative, follow suggestions[].api instead of building a new query.
- If your browsing tool can only open links it has seen, open any report page and follow its Explore links (other GPUs, loads, formats, rankings, solver).
- GPU and model names are fuzzy-matched; check input.corrected, input.assumed and input.ignored to see what was actually computed.
- Long parameter names work as well as the short keys (for example gpu, model, quant, users, context_length).
- A bare call means 1 GPU and 1 user. When e and q are both omitted: vllm with fp8 where the board runs it (most NVIDIA, AMD Instinct/RDNA3+ and Intel boards); otherwise the first of vllm+fp16, vllm+awq4, llamacpp+q4_k_m, ollama+q4_k_m that runs and fits; Apple always gets llamacpp with q4_k_m. If only one of e and q is given, the other is chosen to match. The choice is reported in input.assumed. In a compare across vendors a single engine and quant is chosen for all boards.

## Modes

- `estimator`: One configuration: fit, speed, latency, cost and the bottleneck.
- `agents`: Coding-agent tasks on one configuration: task time, developers supported, cost per developer.
- `solver`: Given a model, users (u) and a speed floor (s), the cheapest hardware that meets it.
- `compare`: Give one parameter as a comma list (up to 4 values); the first is the baseline.
- `rankings`: Every GPU in the catalog for one scenario, sorted by speed, cost or tokens per dollar.

## Parameters

- `t` (also `mode`): Mode. A comma list in any config parameter implies compare. Values: estimator, agents, solver, compare, rankings.
- `g` (also `gpu`, `gpus`, `gpu_id`): GPU id or name, fuzzy-matched ("4090", "H100", "m4 max"). Default h100-sxm. See /api/gpus.
- `n` (also `count`, `gpu_count`, `num_gpus`): GPUs per tensor-parallel group, 1-64. Default 1.
- `x` (also `interconnect`): Interconnect. Default nvlink, capped to what the board supports. Values: pcie4, pcie5, nvlink, nvlink5, infinityFabric.
- `r` (also `replicas`): Independent model replicas, 1-64. Default 1.
- `m` (also `model`, `model_id`): Model id or name, fuzzy-matched. Default llama-3.3-70b. See /api/models/q=<text>.
- `q` (also `quant`, `quantization`): Weight quantization. Default fp8 on hardware that runs it, otherwise chosen to suit the board. Values: fp16, fp8, int8, awq4, gptq4, q8_0, q6_k, q5_k_m, q4_k_m, q3_k_m.
- `k` (also `kv`, `kv_cache`): KV cache dtype. Default fp16. Values: fp16, fp8, int8.
- `c` (also `context`, `context_length`): Context length per request in tokens (prompt + output). Default 8192.
- `p` (also `prompt`, `prompt_tokens`): Prompt tokens, used for time-to-first-token. Default 2048.
- `u` (also `users`, `concurrency`): Concurrent requests in flight. Default 1.
- `e` (also `engine`): Serving engine. Default vllm on hardware that runs it, otherwise chosen to suit the board. Values: vllm, sglang, tensorrt, llamacpp, ollama.
- `f` (also `efficiency`): Measured memory-bandwidth utilization 0.01-1, overrides the engine profile.
- `h` (also `hourly`): Hourly USD per GPU, overrides the catalog price.
- `d` (also `hours`): Hours per day the deployment serves, 0.1-24. Default 24.
- `s` (also `min_tps`): Per-user speed floor in tok/s (solver). Default 25.
- `format`: Response format for /api/calc. Default json. Values: json, md, txt.
- `sort`: Rankings order. Default speed. Values: speed, cost, tpd.
- `limit`: Rows for solver/rankings, 1-50. Default 10.
- `ttft`: Solver: maximum time to first token, seconds.
- `budget`: Solver: maximum USD per hour for the whole deployment.
- `task`: Agents: benchmark one coding-agent task only. Values: completion, question, edit, tests, review, refactor, feature.

## Response

- `verdict`: status, one-sentence headline, bottleneck, key_numbers with units in their names, warnings, confidence
- `suggestions`: alternatives with measured effect and ready links (url for humans, api for agents)
- `input`: resolved values, assumed defaults, fuzzy corrections and ignored values
- `result`: full mode-specific detail
- `links`: canonical page, JSON API, Markdown and this guide
- `disclaimer`: these are analytical estimates, not benchmarks

## Examples

- Does Llama 3.3 70B serve 256 users on 8× B200 with 2 replicas?: https://inference.emirtuncer.com/api/calc/g=b200/n=8/r=2/m=llama-3.3-70b/u=256.md
- Does Llama 3.3 70B FP8 serve 16 users on 2× H100?: https://inference.emirtuncer.com/api/calc/t=estimator/gpu=h100-sxm/count=2/model=llama-3.3-70b/quant=fp8/users=16.md
- Will Qwen3 8B run on my RTX 4090 with llama.cpp?: https://inference.emirtuncer.com/api/calc/gpu=4090/model=qwen3-8b/quant=q4_k_m/engine=llamacpp.md
- Does Llama 3.3 70B run on an M4 Max Mac (engine and format chosen for Apple)?: https://inference.emirtuncer.com/api/calc/gpu=m4-max-128/model=llama-3.3-70b.md
- How many developers can a coding agent on one H100 serve?: https://inference.emirtuncer.com/api/calc/mode=agents/gpu=h100-sxm/model=qwen3-32b/quant=fp8/users=8.md
- Cheapest hardware for 200 users at 25 tok/s each: https://inference.emirtuncer.com/api/calc/mode=solver/model=llama-3.3-70b/quant=fp8/users=200/min_tps=25.md
- 2× H100 vs 2× H200 vs 2× MI300X for the same workload: https://inference.emirtuncer.com/api/calc/gpu=h100-sxm,h200-sxm,mi300x/count=2/model=llama-3.3-70b/quant=fp8/users=16.md
- Best tokens per dollar for an 8B model: https://inference.emirtuncer.com/api/calc/mode=rankings/model=llama-3.1-8b/quant=fp8/users=4/sort=tpd.md

Same request as a query string: https://inference.emirtuncer.com/api/calc?g=b200&n=8&r=2&m=llama-3.3-70b&u=256

## Catalogs

- GPUs: https://inference.emirtuncer.com/api/gpus
- Models: https://inference.emirtuncer.com/api/models/q=llama (search text goes in q=; ?q=llama also works)