{"id":1559,"date":"2025-10-11T09:00:00","date_gmt":"2025-10-11T09:00:00","guid":{"rendered":"https:\/\/teszarypeter.hu\/?p=1559"},"modified":"2025-10-24T13:03:47","modified_gmt":"2025-10-24T11:03:47","slug":"elfer-egy-mini-pc-n-vagy-egy-varost-foglal-el-az-ai-modellek-meglepo-fizikai-merete","status":"publish","type":"post","link":"https:\/\/teszarypeter.hu\/en\/elfer-egy-mini-pc-n-vagy-egy-varost-foglal-el-az-ai-modellek-meglepo-fizikai-merete\/","title":{"rendered":"Does it fit on a mini PC or occupy a city? the surprising physical size of AI models"},"content":{"rendered":"<p>In the previous article, we looked at what AI models are and what their main types are, from language models to image generators. But one important question remains open: Where do these digital brains live? What physical reality is behind the seemingly weightless magic of ChatGPT?<\/p>\n\n\n\n<p>The answer is both fascinating and surprising: An AI model can be as big as a USB drive, but it can also be as big as a small city. In this article, in the second part of my AI series, we will travel this incredible range from home mini PCs to gigantic server parks.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The home experimenter: AI on your own machine<\/h2>\n\n\n\n<p>The biggest revolution of the past year or two is that running AI models (in technical terms: <em>inference<\/em>) It's not just tech giants anymore. Nowadays, anyone with a more powerful home computer can run surprisingly smart, open source models.<\/p>\n\n\n\n<p><strong>What do you need for this?<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>A modern PC or Mac:<\/strong> With a more powerful processor (CPU) and at least 16-32 GB of memory (RAM).<\/li>\n\n\n\n<li><strong>A good video card (GPU):<\/strong> This is the most important ingredient. A modern NVIDIA (RTX series) or Apple Silicon (M-series) card dramatically speeds up the process. Parallel computing capabilities of GPUs are ideal for running AI models.<\/li>\n\n\n\n<li><strong>Appropriate software:<\/strong> Free, open source tools like <strong>Ollama<\/strong> or is <strong>LM Studio<\/strong>, They made the process incredibly simple. Today, you can download and run models like Meta Llama 3 or Mistral by typing a single command.<\/li>\n<\/ul>\n\n\n\n<p>Imagine you're running a very complex video game. AI models running on your home computer are perfect for experimenting, writing texts, and helping with programming tasks, all in a completely private way, without leaving your data behind.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Digital Titans: city-wide server parks<\/h2>\n\n\n\n<p>When you're chatting with ChatGPT-4 or Google Gemini, your request doesn't run on a single computer. The answer comes from a gigantic infrastructure whose dimensions are hard to comprehend.<\/p>\n\n\n\n<p>These models are huge, built for this purpose. <strong>the Data Centers<\/strong> They live in server parks.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Hardware:<\/strong> Instead of working on a single video card, here <strong>tens of thousands<\/strong>, GPU specially designed for AI calculations work together. These are not video cards available in the store. Only one <strong>NVIDIA H100<\/strong>The accelerator price may exceed HUF 10-15 million. Microsoft and Google are spending billions of dollars building supercomputers from these chips.<\/li>\n\n\n\n<li><strong>Energy and cooling:<\/strong> These server parks can reach the total energy consumption of a smaller city. Tens of thousands of maximum-load chips generate an incredible amount of heat that needs to be drained by complex liquid or air cooling systems.<\/li>\n\n\n\n<li><strong>Examples:<\/strong>\n<ul class=\"wp-block-list\">\n<li>That <strong>OpenAI<\/strong> the Models (ChatGPT) <strong>Microsoft Azure<\/strong> They run in their data centers, on a special infrastructure built for this purpose.<\/li>\n\n\n\n<li>A <strong>Google<\/strong> use its own custom-designed AI chips (TPU \u2013 Tensor Processing Unit) to run Gemini and other models organised in gigantic \u2018TPU pods\u2019.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"906\" height=\"510\" src=\"https:\/\/teszarypeter.hu\/wp-content\/uploads\/2025\/10\/Blade-17.png\" alt=\"Microsoft Azure datacenter\" class=\"wp-image-1560\" srcset=\"https:\/\/teszarypeter.hu\/wp-content\/uploads\/2025\/10\/Blade-17.png 906w, https:\/\/teszarypeter.hu\/wp-content\/uploads\/2025\/10\/Blade-17-300x169.png 300w, https:\/\/teszarypeter.hu\/wp-content\/uploads\/2025\/10\/Blade-17-768x432.png 768w\" sizes=\"auto, (max-width: 906px) 100vw, 906px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"has-text-align-center\">Microsoft Azure datacenter<\/p>\n\n\n\n<p class=\"has-text-align-center\">Source of image: <a href=\"https:\/\/datacenters.microsoft.com\/globe\/powering-sustainable-transformation\/\" target=\"_blank\" rel=\"noopener nofollow\" title=\"\">datacenters.microsoft.com<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What makes this gigantic difference?<\/h2>\n\n\n\n<p>Why does one model fit on a home machine while the other needs a power plant? There are two main reasons:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>Training vs. running (inference):<\/strong> The most brutal resource for a model <strong>training<\/strong> He needs it. This is the process where AI \u2018reads\u2019 a significant part of the internet. By analogy: training is like a student preparing for graduation by reading all the existing books and trying to understand them. Running (inference) is when you answer a specific question in the exam. The latter requires much less energy.<\/li>\n\n\n\n<li><strong>Model size (number of parameters):<\/strong> The \u2018knowledge\u2019 of AI models is characterised by the number of their parameters. These are essentially the interfaces of the model\u2019s \u2018neurons\u2019. The more parameters, the more nuanced and complex knowledge the model can acquire.\n<ul class=\"wp-block-list\">\n<li>Open source models that run at home are usually <strong>7 and 70 billion<\/strong> They move between parameters.<\/li>\n\n\n\n<li>The most advanced closed models, such as GPT-4, are estimated to be <strong>more than one trillion (1,000,000,000,000)<\/strong> They have a parameter. The operation of this huge \u2018brain\u2019 requires urban-scale infrastructure.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<p>So the world of AI models shows a fascinating duality: on the one hand, it is becoming increasingly democratised, allowing us to experiment with them at home, and on the other hand, high-tech is still concentrated in the hands of a few giants who are investing amazing resources in development.<\/p>\n\n\n\n<p>Don't be afraid of big numbers. The most exciting thing is that technology is becoming more and more accessible. <strong>In the next part of the series<\/strong> We're going to see exactly how you can install and run a step-by-step open source language model on your own computer with Ollama.<\/p>","protected":false},"excerpt":{"rendered":"<p>Where do these digital brains live? What physical reality is behind the seemingly weightless magic of ChatGPT? The answer is both fascinating and surprising.<\/p>","protected":false},"author":2,"featured_media":1557,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"slim_seo":{"description":"Hol \u201elaknak\u201d ezek a digit\u00e1lis agyak? Milyen fizikai val\u00f3s\u00e1g van a ChatGPT l\u00e1tsz\u00f3lag s\u00falytalan var\u00e1zslata m\u00f6g\u00f6tt? A v\u00e1lasz egyszerre leny\u0171g\u00f6z\u0151 \u00e9s meglep\u0151","facebook_image":"https:\/\/teszarypeter.hu\/wp-content\/uploads\/2025\/10\/AI-Category-Cover.jpg","twitter_image":"https:\/\/teszarypeter.hu\/wp-content\/uploads\/2025\/10\/AI-Category-Cover.jpg","title":"Elf\u00e9r egy mini pc-n vagy egy v\u00e1rost foglal el? az AI modellek meglep\u0151 fizikai m\u00e9rete - Tesz\u00e1ry P\u00e9ter"},"cp_status":"planned","footnotes":""},"categories":[93,4],"tags":[],"class_list":["post-1559","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-bejegyzesek"],"_links":{"self":[{"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/posts\/1559","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/comments?post=1559"}],"version-history":[{"count":0,"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/posts\/1559\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/media\/1557"}],"wp:attachment":[{"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/media?parent=1559"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/categories?post=1559"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/teszarypeter.hu\/en\/wp-json\/wp\/v2\/tags?post=1559"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}