Where do these digital brains live? What physical reality is behind the seemingly weightless magic of ChatGPT? The answer is both fascinating and surprising.
In the previous article, we looked at what AI models are and what their main types are, from language models to image generators. But one important question remains open: Where do these digital brains live? What physical reality is behind the seemingly weightless magic of ChatGPT?
The answer is both fascinating and surprising: An AI model can be as big as a USB drive, but it can also be as big as a small city. In this article, in the second part of my AI series, we will travel this incredible range from home mini PCs to gigantic server parks.
The home experimenter: AI on your own machine
The biggest revolution of the past year or two is that running AI models (in technical terms: inference) It's not just tech giants anymore. Nowadays, anyone with a more powerful home computer can run surprisingly smart, open source models.
What do you need for this?
- A modern PC or Mac: With a more powerful processor (CPU) and at least 16-32 GB of memory (RAM).
- A good video card (GPU): This is the most important ingredient. A modern NVIDIA (RTX series) or Apple Silicon (M-series) card dramatically speeds up the process. Parallel computing capabilities of GPUs are ideal for running AI models.
- Appropriate software: Free, open source tools like Ollama or is LM Studio, They made the process incredibly simple. Today, you can download and run models like Meta Llama 3 or Mistral by typing a single command.
Imagine you're running a very complex video game. AI models running on your home computer are perfect for experimenting, writing texts, and helping with programming tasks, all in a completely private way, without leaving your data behind.
Digital Titans: city-wide server parks
When you're chatting with ChatGPT-4 or Google Gemini, your request doesn't run on a single computer. The answer comes from a gigantic infrastructure whose dimensions are hard to comprehend.
These models are huge, built for this purpose. the Data Centers They live in server parks.
- Hardware: Instead of working on a single video card, here tens of thousands, GPU specially designed for AI calculations work together. These are not video cards available in the store. Only one NVIDIA H100The accelerator price may exceed HUF 10-15 million. Microsoft and Google are spending billions of dollars building supercomputers from these chips.
- Energy and cooling: These server parks can reach the total energy consumption of a smaller city. Tens of thousands of maximum-load chips generate an incredible amount of heat that needs to be drained by complex liquid or air cooling systems.
- Examples:
- That OpenAI the Models (ChatGPT) Microsoft Azure They run in their data centers, on a special infrastructure built for this purpose.
- A Google use its own custom-designed AI chips (TPU – Tensor Processing Unit) to run Gemini and other models organised in gigantic ‘TPU pods’.
Microsoft Azure datacenter
Source of image: datacenters.microsoft.com
What makes this gigantic difference?
Why does one model fit on a home machine while the other needs a power plant? There are two main reasons:
- Training vs. running (inference): The most brutal resource for a model training He needs it. This is the process where AI ‘reads’ a significant part of the internet. By analogy: training is like a student preparing for graduation by reading all the existing books and trying to understand them. Running (inference) is when you answer a specific question in the exam. The latter requires much less energy.
- Model size (number of parameters): The ‘knowledge’ of AI models is characterised by the number of their parameters. These are essentially the interfaces of the model’s ‘neurons’. The more parameters, the more nuanced and complex knowledge the model can acquire.
- Open source models that run at home are usually 7 and 70 billion They move between parameters.
- The most advanced closed models, such as GPT-4, are estimated to be more than one trillion (1,000,000,000,000) They have a parameter. The operation of this huge ‘brain’ requires urban-scale infrastructure.
So the world of AI models shows a fascinating duality: on the one hand, it is becoming increasingly democratised, allowing us to experiment with them at home, and on the other hand, high-tech is still concentrated in the hands of a few giants who are investing amazing resources in development.
Don't be afraid of big numbers. The most exciting thing is that technology is becoming more and more accessible. In the next part of the series We're going to see exactly how you can install and run a step-by-step open source language model on your own computer with Ollama.
