What Do We Actually Want?

Let’s Talk About Games

Before we delve into the topic under this grand title, I want to talk about video games.

I think video games deserve much more attention than being dismissed as a degenerate form of subcultural entertainment. They reveal something profound about the modern world beyond the screen.

I like playing games. I played a lot when I was a kid, and I still play sometimes. From an early age, I wondered how people could so carefully engineer the mechanisms that make a piece of software entertaining enough for someone to spend hours on it.

But recently, after trying to create some tools and apps myself, my curiosity has turned almost into admiration. As it turns out, getting people to spend even a few minutes using something, even for free, is a daunting task.

Diablo II: A Game That Made People Voluntarily Organize Markets and Invent Quasi-Currencies

I want to begin by recalling an ancient memory from the 2000s and an influential cultural artifact of that era: Diablo II.

In this dark fantasy role-playing adventure game, a player assumes the role of a warrior who travels through medieval villages, Egyptian-inspired desert ruins, and primordial jungles to defeat mythical demons and save the world.

At least, that’s how most people come to know and remember the game.

But if you hear about someone spending hundreds of hours farming unique items with complex attributes, you are probably hearing about this game or one of its direct descendants.

The truly intriguing mechanism of Diablo II is not defeating the final bosses (they actually revive every time the player exits and reloads the game), but farming and possessing items within a carefully engineered system.

When players defeat enemies, they randomly drop items distinguished by quality and rarity. Weapons, armor, rings, and jewelry carry different attributes, such as damage or improvements to particular character abilities. Their names sometimes have prefixes and suffixes that describe these attributes or evoke background stories, even urban folklore.

When I was a kid, someone told me that if you embedded six rune stones into a six-socket weapon in a particular sequence—Vex, Hel, El, Eld, Zod, Eth—whose initials spell “VHEEZE,” sounding like “wheeze,” you could create a legendary, immensely powerful weapon called Breath of the Dying, or BOTD.

I thought the guy was making it up. But that was actually how the game worked.

Mechanisms like these encourage players to study, search for, and possess various items. Random drops give the game something of the flavor of gambling, while limited inventory space forces players to examine items carefully rather than simply picking everything up and forgetting about it.

As a paper published in The Review of Austrian Economics observed, players became so enthusiastic about collecting items that, to circumvent inventory limitations, they created new characters solely to carry more items, known as mules in the online community.

Players also created multiplayer games named after the goods they wanted to exchange, typically “X for Y.” Certain gemstones and pieces of jewelry naturally evolved into quasi-currencies.

As the later name of the genre, looter, suggests, the game is fundamentally about the desire for possession.

And that is why so many fascinating economic phenomena emerged within it.

Where Does the Magic Lie?

Sorry for spending so much time discussing details that many non-players may not care about. But as you will see, they are closely connected to the central concern of this essay.

Before getting there, however, I have to bring up a few more names from the history of video games.

If we map out the most influential games that defined the medium—not necessarily the most commercially successful, but those impactful enough to inspire entire genres—we find that they share a common trait.

All these games give players some form of agency.

Diablo gives players reasons to deliberate over acquisition and ownership. The Legend of Zelda gives players a world to explore and adventures to undertake. Doom and Dark Souls place players in environments where they can train themselves to master techniques and become stronger and more skillful, much like practicing basketball or baseball.

My hypothesis is that there is a reason video games are called interactive entertainment.

They attract people by giving back something many of us lack in modern life: a sense of agency, the freedom to exercise control over our surroundings and receive immediate feedback.

Millions of years of evolution have shaped our inclination to interact with the world and evaluate the feedback we receive.

We learned basic survival skills, such as what could be eaten and what should be avoided, partly through direct interaction with our surroundings rather than relying exclusively on socially transmitted knowledge.

But modern society has taken much of that away, if not entirely.

At school, we study for years without producing much tangible output. We pass numerous seemingly meaningless exams, sometimes merely to provide enough signals for the social system to allocate educational resources.

When we start working, titles, tenure, KPIs, and market outcomes become the main focus of our lives, often without giving us a chance to think twice about them. Worse, many of us simply become mired in daily routines.

A cog in the machine.

We are confined to narrow specializations that remain viable under fierce market competition. We no longer feel the direct impact of our decisions and actions without reference to the rules and directions laid out by giant abstract institutions: schools, corporations, and nations.

In fact, those rules and disciplines are themselves ways of conditioning us to obey and become accustomed to being part of the system.

Of course, institutions also enable some of us to access greater resources and act toward common causes, thereby expanding our agency. But not everyone has that access, and much of the time, even those who do don’t experience it that way.

Modern society has subordinated much of our agency to institutions built upon rationalist principles. The problem is not necessarily the institutions themselves, but who gets to determine their objectives, and how much control individuals retain over the systems coordinating their lives.

We are asked to believe in numbers, playbooks, and algorithms, and often find ourselves lost or isolated when we defy them, like the man in Dostoyevsky’s Notes from Underground.

And games give us all of that back.

Agency Within the Rules

I believe many people eventually become disenchanted with games.

One reason is that, as we grow older, we realize there is nothing magical behind the screen, just looping function calls and world states stored in memory.

But more importantly, the agency games provide is constrained in a way that is easy to overlook.

Think of Super Mario Bros. on the NES. There are 256 × 240 pixels in the screen’s resolution, 61,440 in total, with 256 × 224 normally visible. The number of reachable game states—the places to go, situations to encounter, and actions to take—is finite, at least within a fixed implementation.

While players genuinely have choices, the mechanisms governing the world are predetermined, even when they produce unexpected behavior.

Players can discover and exploit bugs, but they cannot freely create possibilities beyond what the underlying system permits without modifying the game itself.

There is an interesting story about Diablo III, recounted in Jason Schreier’s Blood, Sweat, and Pixels (2017). Players reportedly discovered that smashing environmental objects such as jars could be a surprisingly effective way to farm valuable items. As a result, some spent hours doing little more than breaking jars instead of fighting enemies.

The game gives people genuine agency, but largely within rules and objectives they have little power to change.

And so do institutions in real life.

Our preference for gold is, to a significant extent, socially and institutionally constructed. In that respect, it is not intrinsically different from desiring an item in Diablo II.

We could go further and argue that games are highly stylized condensations of institutions themselves, disguised as entertainment.

Both establish systems of incentives, rules, and rewards within which people pursue particular goals, sometimes creating entirely new institutions along the way.

And this is where social media enters the picture. Unlike video games, whose engineered incentives largely operate within a bounded virtual world, social media platforms apply similar mechanisms directly to the information and relationships through which we understand the real world.

Of course, some people acquire genuinely useful information on X or Instagram. But I would argue that social media is intrinsically a reality-shaping machine for much of its audience.

Life becomes sleek resort photos and bikinis. Opinions and political factions become comments and reposts on X.

Worse, algorithmically guided information distribution shapes not only what we know but also how we feel, gradually forging a new reality.

The tendency toward performance and status-seeking changes how we view and evaluate our relationships with others: friends, partners, even family.

Fleeting moods and sentiments can dominate early relationships, rather than genuine appreciation of another person’s values and character.

Democracy itself rests, in part, on the principle that people seek representation for their individual positions and interests, and that they can discuss and organize openly within a public sphere.

That foundation is eroding in a society where information is distributed according to personal preferences inferred by opaque algorithms, and where visibility is heavily influenced by follower counts and engagement metrics.

With the emergence of ChatGPT and other language models, another layer of mediation has appeared. Traditional sources such as books are increasingly bypassed as individuals delegate not only the search for information but also the selection, synthesis, and presentation of that information to algorithms.

You might argue that all of this happened long before, as Ayn Rand depicted in works like The Fountainhead, where the protagonist is an architect who creates independently, in contrast to the masses whose beliefs are shaped by mass media and figures of authority.

I think the underlying tension between creating and participating is similar.

But there is an important difference.

In the earlier age of mass media, influence was more visibly concentrated in identifiable individuals and institutions. That is why the architect in the novel recognizes the press tycoon as, in some respects, a man of his own kind.

Today’s problem is different: abstract platforms and algorithms have become the incumbents.

The mechanisms that shape preferences and distribute influence are no longer necessarily controlled by a single person with a coherent vision. They emerge from complex systems of optimization, competition, and feedback.

And increasingly, nobody is fully in control.

The AI Era

Finally, we can return to the title and talk about AI.

In mainstream discussions, AI agents are often presented as the next great lever of productivity and value creation.

But the discussion of agency is particularly relevant to AI because, as it turns out, we do not even have a single coherent definition of what an agent, or agency, actually is.

There is a hierarchy.

When we abstract a set of function calls and control loops into a concept and call it an agent, what exactly are we talking about?

Is it merely the capability to interact semi-autonomously with the world?

Or is it economic agency: possessing clear preferences among alternatives and being able to determine which to pursue according to some coherent decision rule?

Or is there a further level: the capacity to create the underlying values, institutions, and goals worth pursuing?

These levels are not interchangeable.

And agency does not exist in isolation.

When we seek to delegate human preferences and objectives to AI agents and make their behavior closely aligned with those preferences, have we considered that human preferences and objectives are themselves engineered, socially constructed, and dynamically shaped through interaction with the environment?

Furthermore, what happens when the actions of AI agents themselves cause human preferences and objectives to change?

Like echoes in a pond, the effects propagate outward and return to their source.

An AI system may become increasingly effective at satisfying our preferences while simultaneously shaping the very preferences it seeks to satisfy.

And if those preferences are themselves products of the system’s influence, what exactly does it mean for the system to be aligned with us?

But if our preferences have always been shaped by the environments and institutions we inhabit, does it even make sense to distinguish authentic preferences from manufactured ones?

Perhaps the more important question is not where our preferences originate, but whether we retain the agency to examine, revise, and ultimately determine which ones we wish to pursue.

The mules and quasi-currencies in Diablo II show this agency at work: they emerged from players’ own creativity, beyond the developers’ direct control.

Despite tremendous efforts to measure AI capabilities, from writing code to operating on websites, and substantial research into aligning AI systems with human intentions, this deeper question remains unresolved.

We have become increasingly sophisticated at asking whether an agent can achieve a goal.

We are much less certain about how to determine which goals are worth achieving, who gets to define them, and what happens when the pursuit of those goals changes the people whose interests the agent is supposed to serve.

Perhaps that is the fundamental gap in our current understanding of AI alignment.

Before we start talking about AI becoming a new god, we should pay much closer attention to the research needed to address these questions.

Hey folks, face reality.

---

Reference

Endogenous Currency Formation in an Online Environment: The Case of Diablo II. The Review of Austrian Economics.