Having spent some time playing with Claude Code, getting it to port an old Java Applet game to Godot (spoilers: it worked really well, but I had to spend a little on extra tokens), I thought I'd revisit local LLMs, but this time via a terminal-based system I could kick off within a source project.
First up, there's which 'model server' to use - Ollama or LM Studio. There's pros and cons to both, but the crucial aspect it seems is it's ability to read and write local files. The number of times I typed "create a hello world program in C, saving it to hello.c", only for it to display it on screen, embedded in a json response.
Model selection is crucial, too. Obviously, I wanted a code-focused model, but there's different vendors, different versions, sizes, and quant variants; with different providers offering the same model in a slightly different way. Simply selecting the biggest one your GPU can handle isn't enough either, as there's a tradeoff between size and performance - I found this when my hello world prompt took 10 minutes on a max-size model.
Again, the ability to read and write files affects model choice too - Qwen models seem to fare better than Llama in this regard.
Then there's the terminal prompter software choice - Pi, Continue, even Claude can be pointed at a local model. Pi seems the cleanest.
So, once the file read/write hurdle is resolved, I wanted to see how a local model would cope with the same Applet->Godot task.
Oh boy. It had no clue. So, to be kind, I asked it to make a Javascript version instead.
Claude coped well (apart from taking around an hour to complete), successfully porting the single-file, 4,500 line game to a sensible Godot project. My first attempts with a local model had it either choking on the size of the Java file, or hallucinating its merry way to an entirely different game.
"Break the task down into smaller chunks" came the advice, ironically from Claude. "Get it to analyse the codebase, produce a plan, then get it to progress through the plan piece by piece". I followed its advice, getting the model to produce a framework class, then flesh out input handling, enemy logic, rendering, audio, bit by bit.
I think that took about 8 hours.
Finally, after all the bugs and errors were squashed, we had a running game. Well, you had a yellow triangle you could move left and right. That's it.
Out of frustration, I complained to Claude. It suggested breaking things down further, but if I've got to micro-manage things to that degree I might as well just write the thing myself.
Claude agreed. In it's own way, it confessed it's still all a bit shit really.
So there you go. Using a local model is alright for "what does this error message mean?" and maybe some lightweight, high-level tasks, but it's hopeless for anything meaty. I guess the Tech Bros are safe for a while yet.