{"name":"LLM Inference Calculator API","summary":"Estimate whether an LLM fits on given hardware, how fast it serves, what it costs, and which hardware meets a target. One GET request returns the whole answer.","how_to_use":["AI agents: build the request yourself and send it with curl (or any HTTP GET) from a shell, for example curl \"https://inference.emirtuncer.com/api/calc?gpu=b200&count=8&replicas=7&model=glm-5.3&users=1024&format=md\". Web-reading tools often refuse URLs they built or silently drop everything after \"?\"; if /api/calc answers \"no parameters received\", that is what happened.","Recommended: put the parameters in the PATH. Some browsing tools drop query strings from URLs they build, but keep paths. https://inference.emirtuncer.com/api/calc/g=b200/n=8/m=glm-5.3/u=1024.md returns the answer as Markdown; drop the .md for JSON, or use .txt for plain text.","Path form: one key=value segment per parameter after /api/calc/, e.g. /api/calc/g=b200/n=8/r=7/m=glm-5.3/u=1024. A trailing .md, .txt or .json on the last segment (or a format=md segment) picks the format; other dots, as in glm-5.3, are part of the value. A \"/\" inside a value must be percent-encoded as %2F (meta-llama%2FLlama-3.1-8B-Instruct). Comma lists (g=h100-sxm,h200-sxm) and long names (gpu=b200/count=8) work as in the query form. A segment without \"=\" is reported in input.ignored.","The query form is equivalent: https://inference.emirtuncer.com/api/calc?g=b200&n=8&m=glm-5.3&u=1024&format=md. If both are given, path parameters win.","The same parameters on https://inference.emirtuncer.com/p/<parameters> (or https://inference.emirtuncer.com/?<parameters>) are a human page that also contains the full report as plain HTML text, so it can be shared and read without JavaScript.","Read verdict.headline first: it answers the question in one sentence. verdict.status is ok, warn or fail.","To try an alternative, follow suggestions[].api instead of building a new query.","If your browsing tool can only open links it has seen, open any report page and follow its Explore links (other GPUs, loads, formats, rankings, solver).","GPU and model names are fuzzy-matched; check input.corrected, input.assumed and input.ignored to see what was actually computed.","Long parameter names work as well as the short keys (for example gpu, model, quant, users, context_length).","A bare call means 1 GPU and 1 user. When e and q are both omitted: vllm with fp8 where the board runs it (most NVIDIA, AMD Instinct/RDNA3+ and Intel boards); otherwise the first of vllm+fp16, vllm+awq4, llamacpp+q4_k_m, ollama+q4_k_m that runs and fits; Apple always gets llamacpp with q4_k_m. If only one of e and q is given, the other is chosen to match. The choice is reported in input.assumed. In a compare across vendors a single engine and quant is chosen for all boards."],"endpoints":{"calc":"https://inference.emirtuncer.com/api/calc","page":"https://inference.emirtuncer.com/","gpus":"https://inference.emirtuncer.com/api/gpus","models":"https://inference.emirtuncer.com/api/models/q=llama","guide_text":"https://inference.emirtuncer.com/llms.txt"},"modes":{"estimator":"One configuration: fit, speed, latency, cost and the bottleneck.","agents":"Coding-agent tasks on one configuration: task time, developers supported, cost per developer.","solver":"Given a model, users (u) and a speed floor (s), the cheapest hardware that meets it.","compare":"Give one parameter as a comma list (up to 4 values); the first is the baseline.","rankings":"Every GPU in the catalog for one scenario, sorted by speed, cost or tokens per dollar."},"parameters":[{"key":"t","aliases":["mode"],"description":"Mode. A comma list in any config parameter implies compare.","values":["estimator","agents","solver","compare","rankings"]},{"key":"g","aliases":["gpu","gpus","gpu_id"],"description":"GPU id or name, fuzzy-matched (\"4090\", \"H100\", \"m4 max\"). Default h100-sxm. See /api/gpus."},{"key":"n","aliases":["count","gpu_count","num_gpus"],"description":"GPUs per tensor-parallel group, 1-64. Default 1."},{"key":"x","aliases":["interconnect"],"description":"Interconnect. Default nvlink, capped to what the board supports.","values":["pcie4","pcie5","nvlink","nvlink5","infinityFabric"]},{"key":"r","aliases":["replicas"],"description":"Independent model replicas, 1-64. Default 1."},{"key":"m","aliases":["model","model_id"],"description":"Model id or name, fuzzy-matched. Default llama-3.3-70b. See /api/models/q=<text>."},{"key":"q","aliases":["quant","quantization"],"description":"Weight quantization. Default fp8 on hardware that runs it, otherwise chosen to suit the board.","values":["fp16","fp8","int8","awq4","gptq4","q8_0","q6_k","q5_k_m","q4_k_m","q3_k_m"]},{"key":"k","aliases":["kv","kv_cache"],"description":"KV cache dtype. Default fp16.","values":["fp16","fp8","int8"]},{"key":"c","aliases":["context","context_length"],"description":"Context length per request in tokens (prompt + output). Default 8192."},{"key":"p","aliases":["prompt","prompt_tokens"],"description":"Prompt tokens, used for time-to-first-token. Default 2048."},{"key":"u","aliases":["users","concurrency"],"description":"Concurrent requests in flight. Default 1."},{"key":"e","aliases":["engine"],"description":"Serving engine. Default vllm on hardware that runs it, otherwise chosen to suit the board.","values":["vllm","sglang","tensorrt","llamacpp","ollama"]},{"key":"f","aliases":["efficiency"],"description":"Measured memory-bandwidth utilization 0.01-1, overrides the engine profile."},{"key":"h","aliases":["hourly"],"description":"Hourly USD per GPU, overrides the catalog price."},{"key":"d","aliases":["hours"],"description":"Hours per day the deployment serves, 0.1-24. Default 24."},{"key":"s","aliases":["min_tps"],"description":"Per-user speed floor in tok/s (solver). Default 25."},{"key":"format","aliases":[],"description":"Response format for /api/calc. Default json.","values":["json","md","txt"]},{"key":"sort","aliases":[],"description":"Rankings order. Default speed.","values":["speed","cost","tpd"]},{"key":"limit","aliases":[],"description":"Rows for solver/rankings, 1-50. Default 10."},{"key":"ttft","aliases":[],"description":"Solver: maximum time to first token, seconds."},{"key":"budget","aliases":[],"description":"Solver: maximum USD per hour for the whole deployment."},{"key":"task","aliases":[],"description":"Agents: benchmark one coding-agent task only.","values":["completion","question","edit","tests","review","refactor","feature"]}],"defaults":{"gpuId":"h100-sxm","gpuCount":1,"interconnect":"nvlink","replicas":1,"modelId":"llama-3.3-70b","quant":"fp8","kvDtype":"fp16","contextLength":8192,"promptTokens":2048,"concurrency":1,"engine":"vllm","hoursPerDay":24,"minTps":25,"engineAndQuant":"When e and q are both omitted: vllm with fp8 where the board runs it (most NVIDIA, AMD Instinct/RDNA3+ and Intel boards); otherwise the first of vllm+fp16, vllm+awq4, llamacpp+q4_k_m, ollama+q4_k_m that runs and fits; Apple always gets llamacpp with q4_k_m. If only one of e and q is given, the other is chosen to match. The choice is reported in input.assumed. In a compare across vendors a single engine and quant is chosen for all boards."},"examples":[{"description":"Does Llama 3.3 70B serve 256 users on 8× B200 with 2 replicas?","page":"https://inference.emirtuncer.com/p/g=b200/n=8/r=2/m=llama-3.3-70b/u=256","api":"https://inference.emirtuncer.com/api/calc/g=b200/n=8/r=2/m=llama-3.3-70b/u=256","markdown":"https://inference.emirtuncer.com/api/calc/g=b200/n=8/r=2/m=llama-3.3-70b/u=256.md","query":"https://inference.emirtuncer.com/api/calc?g=b200&n=8&r=2&m=llama-3.3-70b&u=256"},{"description":"Does Llama 3.3 70B FP8 serve 16 users on 2× H100?","page":"https://inference.emirtuncer.com/p/t=estimator/gpu=h100-sxm/count=2/model=llama-3.3-70b/quant=fp8/users=16","api":"https://inference.emirtuncer.com/api/calc/t=estimator/gpu=h100-sxm/count=2/model=llama-3.3-70b/quant=fp8/users=16","markdown":"https://inference.emirtuncer.com/api/calc/t=estimator/gpu=h100-sxm/count=2/model=llama-3.3-70b/quant=fp8/users=16.md","query":"https://inference.emirtuncer.com/api/calc?t=estimator&gpu=h100-sxm&count=2&model=llama-3.3-70b&quant=fp8&users=16"},{"description":"Will Qwen3 8B run on my RTX 4090 with llama.cpp?","page":"https://inference.emirtuncer.com/p/gpu=4090/model=qwen3-8b/quant=q4_k_m/engine=llamacpp","api":"https://inference.emirtuncer.com/api/calc/gpu=4090/model=qwen3-8b/quant=q4_k_m/engine=llamacpp","markdown":"https://inference.emirtuncer.com/api/calc/gpu=4090/model=qwen3-8b/quant=q4_k_m/engine=llamacpp.md","query":"https://inference.emirtuncer.com/api/calc?gpu=4090&model=qwen3-8b&quant=q4_k_m&engine=llamacpp"},{"description":"Does Llama 3.3 70B run on an M4 Max Mac (engine and format chosen for Apple)?","page":"https://inference.emirtuncer.com/p/gpu=m4-max-128/model=llama-3.3-70b","api":"https://inference.emirtuncer.com/api/calc/gpu=m4-max-128/model=llama-3.3-70b","markdown":"https://inference.emirtuncer.com/api/calc/gpu=m4-max-128/model=llama-3.3-70b.md","query":"https://inference.emirtuncer.com/api/calc?gpu=m4-max-128&model=llama-3.3-70b"},{"description":"How many developers can a coding agent on one H100 serve?","page":"https://inference.emirtuncer.com/p/mode=agents/gpu=h100-sxm/model=qwen3-32b/quant=fp8/users=8","api":"https://inference.emirtuncer.com/api/calc/mode=agents/gpu=h100-sxm/model=qwen3-32b/quant=fp8/users=8","markdown":"https://inference.emirtuncer.com/api/calc/mode=agents/gpu=h100-sxm/model=qwen3-32b/quant=fp8/users=8.md","query":"https://inference.emirtuncer.com/api/calc?mode=agents&gpu=h100-sxm&model=qwen3-32b&quant=fp8&users=8"},{"description":"Cheapest hardware for 200 users at 25 tok/s each","page":"https://inference.emirtuncer.com/p/mode=solver/model=llama-3.3-70b/quant=fp8/users=200/min_tps=25","api":"https://inference.emirtuncer.com/api/calc/mode=solver/model=llama-3.3-70b/quant=fp8/users=200/min_tps=25","markdown":"https://inference.emirtuncer.com/api/calc/mode=solver/model=llama-3.3-70b/quant=fp8/users=200/min_tps=25.md","query":"https://inference.emirtuncer.com/api/calc?mode=solver&model=llama-3.3-70b&quant=fp8&users=200&min_tps=25"},{"description":"2× H100 vs 2× H200 vs 2× MI300X for the same workload","page":"https://inference.emirtuncer.com/p/gpu=h100-sxm,h200-sxm,mi300x/count=2/model=llama-3.3-70b/quant=fp8/users=16","api":"https://inference.emirtuncer.com/api/calc/gpu=h100-sxm,h200-sxm,mi300x/count=2/model=llama-3.3-70b/quant=fp8/users=16","markdown":"https://inference.emirtuncer.com/api/calc/gpu=h100-sxm,h200-sxm,mi300x/count=2/model=llama-3.3-70b/quant=fp8/users=16.md","query":"https://inference.emirtuncer.com/api/calc?gpu=h100-sxm%2Ch200-sxm%2Cmi300x&count=2&model=llama-3.3-70b&quant=fp8&users=16"},{"description":"Best tokens per dollar for an 8B model","page":"https://inference.emirtuncer.com/p/mode=rankings/model=llama-3.1-8b/quant=fp8/users=4/sort=tpd","api":"https://inference.emirtuncer.com/api/calc/mode=rankings/model=llama-3.1-8b/quant=fp8/users=4/sort=tpd","markdown":"https://inference.emirtuncer.com/api/calc/mode=rankings/model=llama-3.1-8b/quant=fp8/users=4/sort=tpd.md","query":"https://inference.emirtuncer.com/api/calc?mode=rankings&model=llama-3.1-8b&quant=fp8&users=4&sort=tpd"}],"response":{"verdict":"status, one-sentence headline, bottleneck, key_numbers with units in their names, warnings, confidence","suggestions":"alternatives with measured effect and ready links (url for humans, api for agents)","input":"resolved values, assumed defaults, fuzzy corrections and ignored values","result":"full mode-specific detail","links":"canonical page, JSON API, Markdown and this guide","disclaimer":"these are analytical estimates, not benchmarks"}}