{"id":15712,"date":"2025-10-21T15:04:32","date_gmt":"2025-10-21T13:04:32","guid":{"rendered":"https:\/\/haimagazine.com\/uncategorized\/andrej-karpathy-todays-llms-are-like-ghosts\/"},"modified":"2025-10-24T10:25:01","modified_gmt":"2025-10-24T08:25:01","slug":"andrej-karpathy-todays-llms-are-like-ghosts","status":"publish","type":"post","link":"https:\/\/haimagazine.com\/en\/ai-news-2\/andrej-karpathy-todays-llms-are-like-ghosts\/","title":{"rendered":"\ud83d\udd12 Andrej Karpathy: “Today’s LLMs are like ghosts”"},"content":{"rendered":"

In his conversation<\/mark><\/a> with Dwarkesh Patel and in a series of posts on X<\/mark><\/a>, Karpathy doesn’t deny progress. However, he emphasizes that the current models are far from artificial general intelligence (AGI). They lack memory, learning continuity and the ability to operate in the physical world. According to him, for us to move from the current solutions to AGI there’s plenty of work ahead on the foundations \u2014 integration with sensors, safety, reliability, understanding the social context… Which aren’t issues that a new model version will solve.<\/p>

Animals and ghosts<\/h4>

When comparing AI to AGI, Karpathy uses an analogy that he frames as “animals” and “ghosts.” Animals are physical organisms \u2014 they have a body, senses, needs and instincts. They learn through action: they feel hunger, see danger, experience pain and satisfaction, and respond appropriately. Their intelligence is a product of evolution, programmed over millions of years for survival and adaptation.<\/p>

On the other hand, language models don’t have a body, senses or experiences. They don’t learn by acting \u2014 they learn by imitation. These are purely symbolic entities created from billions of pieces of human language. They can speak, predict and create texts, but they don’t understand the world they talk about. Their “intelligence” is encoded in a neural network, but it lacks physical context \u2014 they are like ghosts that can speak with the voice of humans but don’t have a body to feel the world.<\/p>

Karpathy emphasizes that this difference isn’t merely metaphorical. It’s a fundamental issue with contemporary AI. For machines to truly think, they must not only process symbols but also exist in the world \u2014 sensing stimuli, reacting and experiencing the consequences of their actions. Only then will they become more “animal-like”, capable of learning from their own experiences, not just from data gathered online.<\/p>

Less memory, more wisdom<\/h4>

Karpathy is also quite critical of popular reinforcement learning (RL) methods. He likens them to “sucking supervision through a straw” \u2014 the feedback is weak, delayed, and often misleading. As a result, models might learn incorrect habits or dismiss good ideas if they accidentally lead to unsuccessful outcomes. Instead, Karpathy emphasizes the need to develop “system prompt learning,” a process in which AI learns problem-solving strategies and how to conduct dialogues with itself.<\/p>

This resembles a person who makes notes and gradually develops their own set of cognitive tools. Karpathy advocates for AI that collaborates with humans. As he points out, the goal isn’t for the agent to operate in the background for 20 minutes and write a thousand lines of code \u2014 but to work alongside the user, through iteration and verification. This collaboration model isn’t only more efficient but also safer and more transparent. It protects against AI Slop \u2014 errors that accumulate when AI operates without supervision.<\/p>

Here’s where the “cognitive core” appears at the heart of his vision for future AGI. It’s a simplified system stripped of unnecessary encyclopedic memory, designed to learn to understand rather than just to remember. Karpathy points out that humans don’t remember easily, which is a good thing because the challenge of remembering forces understanding. AI models that remember everything can become overly adapted to data, making them less flexible. Therefore, future systems should be more selective \u2014 less “encyclopedic” and more “understanding.”<\/p>

A year of agents? No, an entire decade.<\/h4>

According to Karpathy, calling 2025 the “year of agents” is actually just a series of flashy demonstrations of tools that can perform simple tasks but falter with more complex ones. He argues that it will take at least 10 years to develop truly general, robust systems capable of functioning in a world where humans and AI really collaborate. Karpathy describes today’s LLMs as “digital bit manipulation,” which is cheaper and simpler than “atom manipulation,” or interacting with the real world. He claims that this physical reality represents the ultimate and most challenging hurdle.<\/p>","protected":false},"excerpt":{"rendered":"

OpenAI’s co-founder Andrej Karpathy tempers some of the excitement around artificial intelligence. Speaking about AGI, he mentions a decade of hard work \u2014 on memory, “embodiment” and system safety.<\/p>\n","protected":false},"author":465,"featured_media":15616,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"rank_math_lock_modified_date":false,"footnotes":""},"categories":[813],"tags":[],"popular":[],"difficulty-level":[38],"ppma_author":[892],"class_list":["post-15712","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-news-2","difficulty-level-medium"],"acf":[],"authors":[{"term_id":892,"user_id":465,"is_guest":0,"slug":"kmironczuk","display_name":"Krzysztof Miro\u0144czuk","avatar_url":{"url":"https:\/\/haimagazine.com\/wp-content\/uploads\/2025\/10\/awatar-2.png","url2x":"https:\/\/haimagazine.com\/wp-content\/uploads\/2025\/10\/awatar-2.png"},"first_name":"Krzysztof","last_name":"Miro\u0144czuk","user_url":"","job_title":"","description":"Od lat zajmuj\u0119 si\u0119 nowymi technologiami w biznesie, edukacji i codziennym \u017cyciu. W centrum mojej uwagi pozostaje cz\u0142owiek \u2013 i to, by technologia wyr\u00f3wnywa\u0142a szanse, zamiast tworzy\u0107 bariery."}],"_links":{"self":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts\/15712","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/users\/465"}],"replies":[{"embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/comments?post=15712"}],"version-history":[{"count":1,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts\/15712\/revisions"}],"predecessor-version":[{"id":15713,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts\/15712\/revisions\/15713"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/media\/15616"}],"wp:attachment":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/media?parent=15712"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/categories?post=15712"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/tags?post=15712"},{"taxonomy":"popular","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/popular?post=15712"},{"taxonomy":"difficulty-level","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/difficulty-level?post=15712"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/ppma_author?post=15712"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}