Local AI for content: what runs on your box, what needs the cloud

In July I bought a 128 GB GB10 box to move content production off the cloud bill. Two months later, about half of that plan has held. This is the honest split: which jobs now run on my own hardware, which ones went back, and the numbers behind each call - my own where I measured them, the vendor's where I did not.

Published September 22, 2026 · By Dmitry Tolstoi · 9 min read

The rule that decides it: overnight or at your desk

In short: local hardware wins on work that can run while you sleep, and loses on anything you sit and wait for.

Most guides to this open on privacy or price. Both were poor predictors of what I actually kept local. The variable that got every call right was simpler: am I waiting on the result?

A job that runs in a queue overnight has no speed requirement, so slow hardware is free time. A job that sits between me and my next decision is different, because there the cost is my attention. A box that takes forty minutes to do what an API does in nine seconds is not cheaper. It is only cheaper on the invoice.

Audio is the easy win: take transcription and voice locally

In short: a used 8 GB gaming card beats real time on transcription, and voice cloning at home costs nothing per line if you collect it in the morning.

The faster-whisper README publishes a benchmark that is easy to reproduce: 13 minutes of audio, Large-v2, fp16, beam size 5, on an NVIDIA RTX 3070 Ti 8GB. It finishes in 1m03s using 4525 MB of VRAM, against 2m23s for the reference implementation. That is roughly twelve times faster than real time, on a card you can buy second hand. OpenAI lists Whisper transcription at $0.006 per minute, so a hundred hours of footage a month is about $36 rented, which buys no hardware. The reason to move it is the loop: when a re-run costs nothing, the ten minutes where the mic clipped get a second pass instead of a shrug.

Voice is the same shape with worse timing. We run Chatterbox locally for narration, and my own measurement on 27 July was 8.4 seconds of speech produced in 112 seconds, so thirteen times real time. That is on the CPU and staying there, because the package pins torch 2.6.0 while CUDA wheels for ARM64 start at 2.9. The upside is that the GPU stays free, so narration and rendering share a night without fighting. For comparison, ElevenLabs lists Creator at $22 a month for 121k credits at roughly one credit per character, which is around a hundred scripts.

Text: the local model covers the boring half

In short: tagging, cleanup and first drafts run fine on a 16 GB machine, while the work you publish under your name still goes to a frontier model.

OpenAI's own guide to running gpt-oss with Ollama gives the entry price plainly: gpt-oss-20b wants at least 16 GB of VRAM or unified memory, and gpt-oss-120b at least 60 GB. The first is a decent laptop, the second a deliberate purchase. The ceiling is just as plain - Moonshot's Kimi K3 model card lists 2.8T total parameters with 104B activated, which is open weights and still nothing you host next to your desk.

So the local model gets the volume work: cleaning transcripts, tagging clips, naming files, summarising research before I read it. The cloud gets the judgement. That division is why our API spend on text sits in the tens of dollars a month rather than the hundreds. It is the same discipline I use when running several agents at once: the cheap model sorts, the expensive one decides.

Images: the licence bites before the hardware does

In short: the most popular local image model forbids commercial use of the model itself, even while handing you the pictures free and clear.

This is the part of local image generation that gets skipped. The FLUX.1 [dev] licence grants use of the model "solely for your Non-Commercial Purposes", and then says it outright: "If you want to use a FLUX.1 [dev] Model or a Derivative for any purpose that is not expressly authorized under this License, such as for a commercial activity, you must request a license from Company."

The same document also says "We claim no ownership rights in and to the Outputs." Both sentences are true, and people quote only the second one. The pictures are yours. Running that model inside a business is the part that needs a licence from Black Forest Labs. Before a generated frame lands on a page that has to look paid-for, open the licence file for the exact weights you downloaded. Apache-2.0 and OpenRAIL-M alternatives exist and cost the same to run.

Video: the one job where local still loses on the clock

In short: my box needs roughly four hours for a fourteen-second lip-synced clip, and the cloud does a comparable eight seconds for under a dollar.

My numbers, measured overnight on 27 July: InfiniteTalk on a Wan2.1 14B image-to-video base, bf16, 480x832. One diffusion step took about 105 seconds. At 30 steps that is roughly 52 minutes per sliding window, and a window is 81 frames at 25 fps, so about 3.2 seconds of finished video. A fourteen-second clip lands near four hours.

That is not an unlucky setup. The Wan2.2-T2V-A14B model card states that single-GPU inference needs "at least 80GB VRAM". The smaller TI2V-5B is advertised as able to "generate a 5-second 720P video in under 9 minutes on a single consumer-grade GPU". Nine minutes for five seconds is the good case, published by the people who built it.

Google's Gemini API price list puts Veo 3.1 at $0.40 per second for Standard, $0.10 per second for Fast at 720p, and $0.05 for Lite, audio included. An eight-second Fast clip is eighty cents and arrives while the kettle boils. Fast local video does exist: Lightricks says its distilled LTX-Video "generates HD videos in 10 seconds, with low-res preview after just 3 seconds (on H100)". Note the hardware named in that sentence.

Local video still earns two spots: b-roll for next week, queued overnight where nobody is waiting, and work no API will do for you, which for us means re-lipsyncing our own footage rather than generating a face. Everything with a publishing date goes to the cloud, because dated work is what keeps a posting schedule alive.

Memory bandwidth is the number that decides your box

In short: capacity tells you which models fit, bandwidth tells you how long each token and each frame makes you wait.

My ASUS Ascent GX10 carries the same GB10 chip as NVIDIA's DGX Spark. NVIDIA's own spec page lists 128 GB of coherent unified memory, 273 GB/s of memory bandwidth, up to 1 PFLOP FP4, a 240 W supply, and inference on models up to 200 billion parameters. The capacity claim is honest; very large models do fit. The bandwidth is what you feel every day, and in the Hacker News thread on the Spark review a commenter posting as threeducks put the comparison on the table:

More precisely, the RTX 5090 has a memory bandwidth of 1792 GB/s, while the DGX Spark only has 273 GB/s, which is about 1/6.5. For inference, the DGX Spark does not look like a good choice, as there are cheaper alternatives with better performance.

Both things are true at once. Big models fit, and every token has to be read out of comparatively slow memory. Pick on the shape of your problem. If the complaint is "the model does not fit", buy capacity. If it is "this is too slow", buy bandwidth, where a gaming card will embarrass a unified-memory box at a third of the price.

One trap is specific to unified memory. In August our image generation died with an out-of-memory kill at 8 GB, on a machine with 121 GB in it. A forgotten inference server was holding 70.6 GB and had been idle for 23 hours; the image model needed 54 GB and there were 47 GB left. In the process list the culprit looked like a 100 MB process, because the memory it held was GPU memory.

What a local box costs after the invoice

In short: electricity is the cheap part, and the expensive part is the week you spend making packages build on your architecture.

I paid $3,999 for the 1 TB ASUS Ascent GX10 in July 2026. Retail on GB10 machines has moved since, and ASUS's own US store currently lists the same model at "Starting at $6,999.00", so price it yourself rather than trusting a figure in an article. On power, the supply is rated 240 W and the chip's TDP is 140 W, and the US residential average for June 2026 was 18.34 cents per kWh. My own arithmetic: a box drawing 200 W for eight hours a night costs about $8.80 a month.

The install tax is the real bill, and nobody quotes it. On an ARM64 machine with a new GPU generation, several packages our video stack expected had no wheels at all: insightface, audio-separator, taichi, the GPU build of onnxruntime, one video decoding library. Neither sage nor flash attention was compiled for the platform, which left the plain PyTorch attention path and no easy speed lever. Budget a week, not an evening.

The split I run now, job by job

In short: five jobs live on the box and three stayed rented, and the line between them is whether a person is waiting.

• Transcription and captions: local, every time. Cheap hardware, faster than real time, free re-runs.
• Scripted narration: local, queued at night. Thirteen times real time is fine when you collect it in the morning.
• Transcript cleanup, tagging, first-pass summaries: local. High volume, low stakes, no audience.
• Re-lipsyncing our own footage: local, because no vendor will process a face on request and nobody is waiting on it.
• B-roll for the coming week: local, overnight, with the queue kept full so the hardware is never idle.
• Video with a publishing date on it: cloud. Eighty cents beats four hours.
• Anything published under my name: cloud model first, then my edit. The local model is not the bottleneck here, taste is.
• Research and back-and-forth during the working day: cloud, because I am the one waiting.

If you are weighing the purchase: rent for a month, log which jobs you ran and how long you sat watching them, then buy for the shape of that log. I bought on a guess and got half of it right, which is a cheerful way of saying the other half is a very expensive transcription machine.

Where this comes from

Vendor figures are linked where cited and were checked the week this was written. They come from NVIDIA for the GB10 specification, Google for Veo pricing, and OpenAI for Whisper pricing and the gpt-oss memory guidance. The rest are ElevenLabs for plan credits, Black Forest Labs for the FLUX licence text, and the Wan and LTX-Video repositories for their own claims. The electricity rate is the EIA's published US residential average for June 2026; the monthly power figure is my own arithmetic on it, assumptions stated beside it. The Hacker News quote is verbatim and attributed to the handle that wrote it. The three timings that are mine - 105 seconds per diffusion step, 8.4 seconds of speech in 112 seconds, and the 70.6 GB memory collision - are logged measurements from one box, at the settings named beside each. Your hardware will disagree, which is the point of measuring instead of quoting. Nothing here is a promise about your results.