What really is artificial intelligence? What is intelligence in the first place? What is the difference between artificial and human intelligence?

This blog serves as my scratchpad for answering these questions. As I read and learn more, I will occasionally add to this blog.

1. Introduction

Let’s start by observing Moravec’s paradox: it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility. At first glance, this seems counterintuitive. Humans regard activities such as navigating unfamiliar environments, or grasping objects as effortless, while abstract reasoning appears difficult and intellectually demanding. In 1994, Steven Pinker wrote “the main lesson of thirty-five years of AI research is that the hard problems are easy and the easy problems are hard”. To the layman, it might seem that activities that we call “common sense” are the hardest for AIs to achieve excellence in. By the 1990s, AI could play chess better than any human player; by the 2010s, deep learning had taken over vision.

The paradox becomes less surprising when viewed through the lens of evolution. Skills such as perception, motor control, and sensory integration are ancient adaptations that have been refined through hundreds of millions of years of natural selection. By contrast, activities such as mathematics, formal logic, and scientific reasoning are relatively recent developments in human history.

Look at a simple perceptron: z = w^Tx + b, individually they look like a simple formula, but when you organize billions of them they can produce intelligent behaviour. As we scale models, we start to observe emergent abilities across a wide range of tasks [Wei et al., 2022]. Similarly, a singular neuron is not intelligent, but millions of them together form an intelligent system. If you were to look for the source of intelligence, you would find that it does not lie in a singular neuron but in fact it is the data transformation across neurons that results in intelligent behavior. Does that mean intelligence lies in transforming data? How much complexity is enough? And why do some systems exhibit intelligent behavior while others do not? To answer these questions we will have to peel through our layers of knowledge and start from the fundamentals.

2. Measures of Intelligence

Let us approach the problem from another angle: how do we measure intelligence? More importantly, do we have the right tests to measure it? Standard testing is a crude measure for intelligence as it pertains to a very narrow domain [Legg & Hutter, 2007]. These tests have proven useful for predicting performance on certain cognitive tasks, yet they capture only a narrow slice of what we intuitively mean by intelligence. Excelling at an IQ test demonstrates proficiency at IQ-test-like problems, but people often overestimate what this reveals about judgment, creativity, adaptability, wisdom, or success in complex real-world environments. With language models, we develop benchmarks and evaluations designed to test specific capabilities such as coding, mathematics, reasoning, and scientific knowledge. These benchmarks provide a common yardstick for comparing systems and tracking progress. Yet they also introduce a fundamental problem: once a benchmark becomes a target, systems begin to optimize for success on that benchmark rather than for the broader capability it was intended to measure (Goodhart’s Law). Benchmarks that were developed with hopes of measuring intelligence were quickly Goodhart’d by language models, causing researchers to develop newer ones to once again measure intelligence.

Perhaps intelligence is not best understood as performance on a fixed set of tests, but as the ability to learn, adapt, and achieve goals across a wide range of novel environments.

3. Artificial vs Human Intelligence

“Look deep into nature, and then you will understand everything better.” - Albert Einstein

If the goal is to breathe intelligence into silicon, the most natural place to look is biological intelligence. The human brain remains the only known example of a generally intelligent system, making it a useful reference point for understanding the nature of intelligence itself.

A typical human brain contains roughly 86 billion neurons connected by an estimated 100-1000 trillion synapses while consuming only about 20 watts of power. Estimates of its computational capacity vary widely, but are commonly placed between 10¹⁵ and 10¹⁷ floating-point operations per second (FLOPS). By many quantitative measures, modern AI systems and hardware have already surpassed these figures. DeepSeek-V4 contains approximately 1.6 trillion parameters (49 billion active parameters per token) and was trained on more than 32 trillion tokens of text. A single NVIDIA B200 accelerator can deliver roughly 2.2 × 10¹⁶ FLOPS of computation, placing it within the range of many estimates for the computational throughput of the human brain.

Language models also possess capabilities that exceed human limits in several dimensions. They can store and recall vastly more information than a person, process information at far higher bandwidth, and perform competitively across a broad range of cognitive tasks. Modern models can ingest the equivalent of multiple books in seconds, summarize large corpora, and reason over contexts far larger than a human could hold in working memory. Furthermore, AI systems benefit from a powerful scaling advantage: their performance can often be improved simply by allocating more computational resources during training or inference. Yet despite exceeding the human brain on many quantitative metrics, today’s AI systems remain fundamentally different from human intelligence.

Given the human brain has a much smaller memory, it still performs much better than LLMs on generalized and real-world tasks. Our attention is a highly optimized resource that has been honed over years of evolution and determines what should get processed, stored and discarded. Neuroscience has done a poor job in helping us understand the nature of intelligence, but that should not let us down. Technology has often preceded scientific theory, as was the case with steam engines’ role in motivating thermodynamics, which went on to explain much more than engine efficiency. Artificial Neural Networks, despite being a black box, might help us understand more about the nature of intelligence and the human brain than neuroscience did. For example, Temporal Difference (TD) learning predicted the behavior of dopamine neurons decades before neuroscientists knew what those neurons were computing.

4. Search

Human thinking can be understood as a search over a space for the right outcome. Extrapolating this, AI can be thought of as searching over an extremely large space for the best possible outcome. The statement captures this aptly:

“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.”

In that case, we only need to scale data and compute. At first, modern language models can seem to be following this statement. Language models have highly tuned and complex architectures and are bottlenecked by human data. Nonetheless, they have excelled at scaling compute. By hurling raw compute and parameters, model performance has continued to improve, though we are starting to see linear improvements with non-linear compute spend.

Deducing from the bitter lesson, architectures that were more adaptable and could learn quickly by leveraging compute will perform better than finely tuned ones. AlphaGo is one of the clearest products of the bitter lesson. Before AlphaGo, researchers spent significant time hand crafting professional Go knowledge into algorithms with datasets, heuristics, search rules and so on. AlphaGo represented a cataclysmic shift by removing much of the handcrafted knowledge by training a neural network that could perform a massive search. AlphaGo combined extensive tree search, statistical estimation and self-play to produce the greatest Go mind.

4.1 Universal Agents (AIXI)

Instead of building an agent that can maximize intelligence, does there exist an agent that can perform exceedingly well in any computable environment [Hutter et al., 2024]. Such an architecture would have to be generalistic and have capabilities to perform well across many different environments. But what if the agent has absolutely no prior knowledge about the environment? Here we introduce the idea of Kolmogorov Complexity. The Kolmogorov complexity of an object, such as a piece of text, is the length of a shortest computer program that produces the object as output. It is a measure of the computational resources needed to specify the object, and is also known as algorithmic complexity. Consider the following two strings of 32 lowercase letters and digits:

abababababababababababababababab

4c1j5b2p0cv4w1x8rx2y39umgw5q85s7

Intuitively, the first one is “simple.” The second appears random. But what does “simple” actually mean? The first string has a short English-language description, namely “write ab 16 times”, which consists of 17 characters. The second one has no obvious simple description (using the same character set) other than writing down the string itself, i.e., “write 4c1j5b2p0cv4w1x8rx2y39umgw5q85s7” which has 38 characters. Hence the operation of writing the first string can be said to have “less complexity” than writing the second. Thus, Kolmogorov gave us a precise definition of simple, simplicity = shortest program, while Occam’s razor mathematically proved. Building on this idea, Solomonoff’s Universal Prior assigns a probability to every computable hypothesis (or environment), with the probability decreasing exponentially as its Kolmogorov complexity increases. In other words, simpler environments—those describable by shorter programs—are considered exponentially more likely before any observations are made. Rather than being an arbitrary assumption, this prior follows naturally from the principles of algorithmic information theory [Solomonoff, 1964; Hutter et al., 2024]. AIXI adopts this universal prior as its initial belief over all computable environments, allowing it to begin learning without relying on domain-specific assumptions. AIXI solves general RL problems by planning with a posterior belief over environments, and this belief is continuously updated as more observations come in from interacting with the environment.

AIXI’s optimality guarantee serves as the basis for a formal and quantitative definition of machine intelligence, that is, the Legg-Hutter score [Legg & Hutter, 2007]. Accordingly, universal intelligence is measured as the expected cumulative reward over all computable environments and tasks (weighted by their inverse complexity), and, by definition, AIXI is the upper bound for this intelligence measure—it can be shown that no other agent can achieve higher expected cumulative reward. The big crux is that neither AIXI, nor the associated intelligence measure, are computable. AIXI must consider infinitely many computable environments, evaluate infinitely many programs, and search through an exponentially large tree of future action sequences, thus requiring unbounded computation. Consequently, AIXI can be viewed as a theoretical ideal against which agents can be compared. In fact, much of modern AI can be understood as developing increasingly effective approximations to this unattainable optimum.

5. Compression

I was dumbfounded when I came across the Hutter Prize. The Hutter Prize challenges researchers to compress a large snapshot of Wikipedia as much as possible. The contest is motivated by the fact that being able to compress well is closely related to intelligence. In order to compress data, one has to find regularities in them, which is intrinsically difficult. Smarter compressors would allow for the lossless compression for complex datasets. Consider the sequence of 1000 digits “14159…[990 more digits]…01989”. If it looks random to you, you can neither compress it nor can you predict the 1001st digit. If you realize that they are the first 1000 digits of π, you can compress the sequence and predict the next digit. While the program computing the digits of π is an example of a one-part self-extracting archive, the impressive Minimum Description Length (MDL) principle is a two-part coding scheme akin to a (parameterized) decompressor plus a compressed archive. If M is a probabilistic model of the data X, then the data can be compressed (to an archive of) length log(1/P(X|M)) via arithmetic coding, where P(X|M) is the probability of X under M. The decompressor must know M, hence has length L(M). One can show that the model M that minimizes the total length L(M)+log(1/P(X|M)) leads to best predictions of future data. For instance, the quality of natural language models is typically judged by its perplexity, which is equivalent to code length. Finally, sequential decision theory tells you how to exploit such models M for optimal rational actions. Indeed, integrating compression (=prediction) into sequential decision theory (=stochastic planning) can serve as the theoretical foundations of super-intelligence.

A follow-up question that naturally arises is why is the understanding of the text or “intelligence” needed to achieve maximal compression? Let’s understand this using an example. Suppose you were compressing all of Wikipedia. A good compressor must recognize grammar, syntax, semantics, recurring concepts to compress language and technical concepts to be able to compress scientific knowledge and facts. A model must understand the intricacies of language to be able to compress it using fewer bits. We can summarize the relationship as: Compression is equivalent to prediction, and prediction is a fundamental component of intelligence. Stated another way, a system that can compress arbitrary real-world data extremely well must have learned a highly accurate model of the processes that generated that data.

6. Outlook

AIXI helps in providing a framework towards building an intelligent agent, while the Hutter Prize measures progress towards building compressible world models. The ideas that we have explored might seem trivial, but are nearly impossible to implement. Learning environments today are simpler and easily navigable, but the world is far richer and more complex than any model, representation, or set of assumptions we can hand-design [Javed et al., 2020]. Developing algorithms that can exploit the richness and complexity through learning and search remains one of the central challenges of artificial intelligence. While AIXI provides the strongest known theoretical upper bound for machine intelligence, some of its limitations have motivated alternative theoretical frameworks such as reflective oracles [Fallenstein et al., 2015], logical induction [Garrabrant et al., 2016], and Schmidhuber’s self-referential Gödel machines [Schmidhuber, 2003]. Beyond Universal AI, several other theoretical frameworks offer complementary lenses on intelligence and its limits: PAC-learning and statistical learning theory provide sample-complexity bounds for generalization; algorithmic game theory formalizes strategic interactions among rational agents, relevant to multi-agent ASI scenarios; and thermodynamic perspectives connect information processing to physical energy costs—notably, thermodynamic bounded rationality [Ortega and Braun, 2013] formalizes optimal decision-making under information-processing constraints using free-energy principles.

“The problems are solved, not by giving new information, but by arranging what we have always known.” - Ludwig Wittgenstein

The ideas proposed so far have been useful in getting us to where we are today, but there is a lot of research to be done in order to reach superintelligence. World models are a step in the direction of building internal learned representations of the environment, instead of performing complex search directly. Imparting curiosity, motivation to learning and a sense of aesthetic play a key role in allowing agents to develop novel intelligence. My personal belief is that the answers to building intelligence probably exist in the vast literature of AI research. Unifying concepts across this swath into a singular, simple theory might just be the revelation needed!

Additional Resources

1. Universal Intelligence: A Definition of Machine Intelligence: [Legg & Hutter, 2007]
2. Emergent Abilities of Large Language Models: [Wei et al., 2022]
3. It’s the Stupid Economy, Stupid
4. Universal Artificial Intelligence (Book): [Hutter]
5. [Genewein et al., 2026]
6. A Formal Theory of Inductive Inference, Part I, Information and Control: [Solomonoff, 1964]
7. [Javed et al., 2020]