The browser-based, cloud-based solution is full of compromises.
Beyond the Cloud
If you ask a software developer today about how they use AI, nine out of ten people will do it: “I subscribe to ChatGPT or Claude and copy the code into a browser window.” Undoubtedly, cloud-based large language models (LLMs) have thrown a lot at productivity, but as I delved deeper into day-to-day AI-assisted programming, I realized that the browser-based, cloud-based solution is full of compromises.
As a pragmatic system builder, I have always strived for independence and control over data. That’s why I decided to cut myself off from the cloud on my developer workstation and switch to running local, local AI models. Let’s see why it’s worth getting started and what it looks like in practice!
The illusion: “Local AI takes millions of supercomputers”
The biggest misconception about local models is the need for hardware. Many people believe that dedicated data centers or multi-million-dollar graphics cards with tens of gigabytes of VRAM are needed.
When the prometheus I am working on my Pop!_OS workstation named Pop!_OS, a reliable but now absolutely mid-range NVIDIA GeForce GTX 1660 SUPER (6GB VRAM) card crashes in the chassis next to the i5 processor and 16GB RAM. And yet, on this machine, I run AI models sharpened for programming at lightning speed without any problems. The secret is Quantization The developers cleverly “compress” these models, keeping them surprisingly smart and fast even with smaller memory.
What does my local AI stack look like?
No need to cast spells with complicated terminal commands. The soul of my system is Ollama, an ingenious open source framework that makes it easy to download and run models on Linux.
The workflow is the VS Code is done in a code editor, where the Continue I installed an open source add-on called. This plugin connects directly to the Ollama server running in the background. And under the hood is currently the qwen2.5-coder:7b I'm using a model. It's a code-trained model with 7 billion parameters that fits into my 6GB VRAM and writes PHP, JS, or CSS code with amazing precision.
Why is local AI better than ChatGPT?
1. Zero Waiting and Continuous “Flow” If you need to copy the code into the browser, it immediately breaks the programmer focus (flow). On the other hand, the AI integrated into VS Code, which runs locally, works under my hands. I select a function, press a hotkey, and the AI instantly changes the code locally. No network latency, no “Server is currently overloaded” error message.
2. The Context of Mr. (No More Copy-Paste) When working on a premium WooCommerce webshop or the OptiKit ecosystem, the code never consists of a single file. Cloud AI is very difficult to explain the entire project structure. The Continue plugin, on the other hand, sees (almost) my entire workspace. If I ask the Ollaman runner qwen2.5-coder“write this function so that it fits our database class” will know what I’m talking about because it will directly “see” the other open files on my machine.
3. Privacy and Business Secrets This is the most important argument. As a senior developer, I often work on customer-specific systems, API keys, or sensitive database queries. Under no circumstances can I send my clients’ code to third-party servers (e.g. OpenAI). For local models, I could even pull the mains cable out of the wall. The code, prompt, and generated response never leave my Pop!_OS system.
4. Uncensorship and Freedom Large cloud models are becoming increasingly aggressively restricted for security and ethical reasons. While this is useful for the average user, it often causes false positive bans as a developer (for example, when writing an example code for a security test or for protection against SQL Injection, and cloud-based AI refuses to answer). Local models don't pop: they write the code I ask them to.
Summary
Integrating AI into everyday development is no longer the future, but the present. However, it doesn’t matter whether we rely on an expensive black box from an external company or build our own independent infrastructure. Ollama and Continue have proven to me that professional AI-assisted coding doesn’t require a cloud subscription – just a well-configured Linux workstation and a bit of experimentation.
