Self-host or API: how to actually decide which way to run an AI model

Every time a serious open-weight model ships, the same argument breaks out underneath it: should you run this yourself, or just pay per token through an API? DeepSeek's own release notes this week for V4-Flash-0731 quietly settle half of that argument for anyone who reads the fine print — they publish exactly what self-hosting the model would take, in hardware terms, right next to the API price. Most releases don't hand you both numbers side by side like that, which is exactly why the decision usually gets made on vibes instead of arithmetic. It doesn't have to.

Name the constraint before you compare prices

Cost is what everyone wants to compare first, and it's the wrong place to start. One constraint overrides the entire cost conversation, and checking it first saves you from running a spreadsheet you didn't need. Everything else genuinely is a cost-and-convenience tradeoff — this one isn't.

The branch worth dwelling on is the third one, because it's the only one where the answer genuinely depends on arithmetic rather than a hard rule. "High, steady, predictable" is doing three separate jobs in that sentence, and dropping any one of them breaks the logic. High matters because hardware has to clear a usage floor before it's cheaper than paying per token — a GPU node sitting mostly idle is a worse deal than the API, not a better one. Steady matters because self-hosted capacity is sized for a load level, and traffic that spikes well above your provisioned hardware either falls over or forces you back to an API for overflow anyway, at which point you're paying for both. Predictable matters because the whole comparison in step five assumes you can forecast next month's volume well enough to size against it; a business whose AI usage swings 5x month to month is optimizing for the wrong variable if it's optimizing for per-token cost at all.

The number nobody runs

If you land on the "run the numbers" branch, here's what an actual comparison looks like using this week's release as the worked example — not because the exact figures transfer to every model, but because the shape of the comparison does. DeepSeek-V4-Flash-0731's API runs $0.14 per million input tokens and $0.28 per million output tokens. Self-hosting the same model takes roughly 110GB of memory at 3-bit quantization, or one 4×GB300 node at full precision, by DeepSeek's own published spec.

Notice what that ledger deliberately doesn't do: it doesn't convert the hardware requirement into a dollar figure, because DeepSeek doesn't publish one and neither should anyone writing about it secondhand. Cloud GPU rental rates and outright hardware purchase prices both move constantly and vary by provider, region, and whether you're buying spot or reserved capacity. The honest version of this comparison has one priced side and one side you have to price yourself — which is the entire reason step four of the procedure below exists as a separate step instead of a number this guide hands you.

The six-step decision

Step five is where most of these comparisons quietly fall apart, and it's worth dwelling on why. A hardware purchase is a one-time number; an API bill is a recurring one. Setting them side by side without converting the hardware cost into a monthly, amortized figure makes self-hosting look artificially cheap — you're comparing a single upfront payment to only one month of the alternative. Amortize over the hardware's realistic useful life, usually two to four years for GPU infrastructure before it's meaningfully behind the frontier, and the comparison becomes honest.

One more wrinkle worth naming: this decision isn't really a one-time fork, because the model landscape underneath it keeps moving. [Matching a job to a model tier](#/article/which-ai-for-which-job) assumes you're choosing between API-hosted options; open weights add a genuine fourth branch to that framework, not just a cheaper version of the same choice. A team that self-hosts locks in a specific model's capability level until it re-runs this procedure, while a team on an API rides each vendor's improvements automatically — DeepSeek's own V4-Flash retraining this week is exactly the kind of free upgrade an API user gets for nothing and a self-hosting team has to redeploy for manually. That's a real cost of self-hosting that step four's hardware quote doesn't capture, and it belongs in the decision alongside the dollar figures.

Two honest limits before you run this yourself. First, the worked example above uses one model's numbers from one week — API prices and hardware costs both move, sometimes sharply, so re-check the actual figures rather than reusing these. Second, this framework assumes you can get a real hardware quote and real usage data; if either one is a guess, the output of step five is a guess too, dressed up as arithmetic. The six steps don't remove uncertainty from the decision. They make sure the uncertainty that's left is the kind you chose, rather than the kind you didn't notice.