Imagine downloading a powerful AI model to your personal computer, ready to analyze your private documents or generate reports without sending data to the cloud. It sounds like the ultimate productivity hack for today’s information workers. But when one tech journalist tried exactly this on his three-year-old MacBook Pro, the experience was so painfully slow it took over an hour to get a simple response. This brutal reality check reveals a critical tension in today’s AI landscape: while software tools make local AI increasingly accessible, hardware limitations create a formidable barrier that could reshape how businesses approach AI adoption.
The Hardware Bottleneck: More Than Just RAM
ZDNET’s experiment with Ollama, an open-source tool for running large language models locally, exposed a fundamental truth many businesses are discovering: AI demands serious hardware. The journalist’s MacBook Pro with 16GB RAM struggled with even “small” models like GLM-4.7-flash (30 billion parameters), taking 76 minutes to answer a basic question. OpenAI’s ChatGPT suggested a minimum of 32GB RAM for similar tasks, highlighting how quickly hardware requirements are escalating.
This isn’t just about consumer frustration – it’s a business reality. As companies consider bringing AI in-house for data privacy, cost control, and customization, they’re facing hardware investments that rival cloud subscription costs. The journalist’s conclusion that he needed to upgrade to an M4 or M5 Mac with at least 32GB RAM reflects a broader trend: AI is pushing hardware requirements beyond what was considered “professional” just a few years ago.
The Counterbalance: Why Businesses Still Want Local AI
Despite the hardware challenges, the appeal of local AI continues to grow, driven by several compelling business factors. First, data privacy concerns are pushing organizations to keep sensitive information on-premises. Second, as cloud AI pricing becomes more complex and potentially more expensive, predictable hardware costs become attractive. Third, customization and fine-tuning capabilities offer competitive advantages that standardized cloud services can’t match.
This tension creates a fascinating market dynamic. While individual users might struggle with hardware limitations, businesses with dedicated IT infrastructure could find local AI increasingly viable. The question becomes: at what scale does the hardware investment make sense compared to cloud subscriptions?
The Hardware Supply Chain Reality
To understand why hardware is becoming such a bottleneck, look upstream to companies like ASML. The Dutch lithography giant reported record-breaking 2025 results with �32.7 billion in revenue, driven by surging demand from AI chip and memory manufacturers. CEO Christophe Fouquet noted: “If I look at logic, we see that our customers are increasingly confident about the sustainability of long-term demand in the AI sector.”
This isn’t just corporate optimism – it’s reflected in hard numbers. ASML sold 327 lithography systems in 2025, including advanced EUV models costing up to �350 million each. Their �13.2 billion in new bookings from chipmakers and �7.4 billion from memory manufacturers both set records. This upstream investment translates to downstream hardware requirements that are only going to increase.
The Startup Innovation Angle
While hardware demands grow, software innovation continues to push what’s possible with existing resources. Consider Arcee AI, a 30-person startup that built Trinity, a 400-billion-parameter open-source LLM, in just six months for $20 million. Their CTO Lucas Atkins argues: “Ultimately, the winners of this game, and the only way to really win over the usage, is to have the best open-weight model. To win the hearts and minds of developers, you have to give them the best.”
This creates an interesting paradox: as models become more capable, they require more resources, but smarter software engineering and optimization could make them more efficient. The race isn’t just about bigger models – it’s about better models that can do more with less.
The Geopolitical Dimension
Hardware availability isn’t just about technical specifications – it’s increasingly about geopolitics. China’s recent approval of over 400,000 Nvidia H200 AI chips for companies like ByteDance, Alibaba, and Tencent reveals how strategic considerations shape hardware access. Alex Capri, a senior lecturer at National University of Singapore’s business school, notes: “Beijing’s approval of the H200 is driven by purely strategic motives. Ultimately, this decision is taken to further China’s indigenous capabilities and, by extension, the competitive capabilities of China tech.”
For businesses outside major tech hubs, this adds another layer of complexity to hardware planning. Will the chips they need be available? At what cost? And with what restrictions?
The Practical Business Implications
So what does this mean for businesses considering local AI? First, hardware planning needs to become more strategic. The days of buying computers with “enough” RAM for the next few years are over – AI requirements are evolving too quickly. Second, total cost of ownership calculations need to include not just hardware costs, but also energy consumption, cooling, and maintenance. Third, businesses need to consider hybrid approaches that combine local processing for sensitive tasks with cloud resources for more demanding workloads.
The journalist’s experience with Ollama serves as a valuable reality check. While tools are making local AI more accessible, the hardware requirements remain substantial. For businesses, this means careful planning and potentially significant investment. But for those who can make it work, the benefits – data privacy, customization, and predictable costs – could be substantial.
Looking Ahead: The Hardware-AI Feedback Loop
As AI continues to evolve, we’re seeing a feedback loop develop: more capable AI drives demand for better hardware, which enables more capable AI. This creates opportunities for hardware manufacturers but challenges for businesses trying to keep up. The key insight from the Ollama experiment isn’t that local AI is impossible – it’s that it requires careful planning and realistic expectations.
For forward-thinking businesses, the question isn’t whether to adopt local AI, but how to do it strategically. This means understanding hardware requirements, considering hybrid approaches, and staying informed about both software tools and hardware developments. The brutal experience on that three-year-old MacBook Pro isn’t a reason to avoid local AI – it’s a valuable lesson in what it really takes to make it work.