The experience economy. The new frontier.

Experience: such an elusive term. What is experience? Does it all come down to electrochemical activity? Is all that we experience ultimately an illusion produced by our brains?

While we’re still figuring that out, companies like Apple, Facebook, Epic, Niantic, Snap, and others are eyeing the “experience economy.” It’s about to change everything.

Nothing Ever Changes

While the kinds of experiences we have and the ways we experience things have changed over the course of human history, the core element of each experience remains the same: you. At the center of any experience is the one who experiences it.

In ancient times, we humans had to rely solely on our brains for memories and information about our previous experiences. Over time, we invented technologies and methods to document these experiences: bone craving, cave drawings, oral storytelling, writing, voice, photo and video recordings. Now, with the development of Augmented Reality (AR), yet another new method awaits us: a way for us to experience and document moments in a way that combines all of the methods above and more, all at once, while preserving the essence of the experience.

A bone carving, one of the world’s oldest works of art: A generated from micro-CT scans, this 3D rendering shows an ancient carving of a giant deer bone, produced by Neanderthals in what is now northern Germany at least 51,000 years ago. Read more about the recent discovery

The Brain

Before we jump into what’s about to change, let’s focus on the main element that helps us to experience moments in life: the brain.

The brain is an ever-changing organ, constantly reacting to our experiences.On the one hand, it initiates new neural connections every time you’re experiencing or learning something new and may even produce new neurons in some cases. On the other hand, stress, depression, aging, and negative experiences can trigger neurons to break down and even die.

fMRI Brain Scan: Brain regions activated by playing Re-Mission. Red/orange/yellow = increased activity. Blue = decreased activity. https://youtu.be/9IK2eDh3cVs

Brain Plasticity and the Environment

While it was once thought that the creation of new neurons stopped after birth, we now know that the brain processes its approximately 86 billion neurons by reorganizing them, creating new connections, and in some cases, even creating new neurons. At birth, each neuron in the cerebral cortex has an estimated 2,500 synapses that allow it to communicate with other cells; at the age of three, this number balloons to 15,000. However, the average adult has about half this number. As we gain new experiences, some connections are strengthened while others eventually die. This natural process is known as synaptic pruning. It allows the brain to adapt to an ever-changing environment.

We know that age and genetics exert significant influence on the brain’s development. For example, during childhood, our brains are more sensitive and responsive to experiences than when we’re older. But that doesn’t mean adult brains aren’t capable of adaptation.

The most significant influence on adult brain adaption is the environment, and you can encourage your brain to adapt and change by modifying your environment. Being in an environment that offers new opportunities for focused attention, novelty, and challenges — activities such as learning a new language, playing an instrument, reading, exploring new places, and creating art or engaging in other creative pursuits — can stimulate your brain.

The Zombie “App”-ocalypse

Artwork: ‘Wide Awake’ by David Moyers

If environment matters so much, what sorts of environments are we offering our brains today? Let’s face it: smartphones are making us dumber. That’s the price we pay for a more mobile, connected world and for having all the world’s information accessible at our fingertips. Even if some of us are indeed being productive in front of our flat screens, our brains have become lazier. We’ve stopped thinking for ourselves.

The world of apps has brought many great opportunities, but it has also created an environment in which we aren’t prompted to use our brains for problem-solving. Furthermore, today’s app-centric milieu also contributes to the increasing atrophy of our social and emotional skills. Before smartphones, most of our interactions were face-to-face. But over time, as we grew accustomed to heavy texting and immediate responses, we became less patient with real, face-to-face conversations. Lack of face-to-face communication has been reliably shown to contribute to depression, and as mentioned above, we know that depression can lead to a loss of neural connections.

The Creator Economy vs. the Zombie Economy

The creator economy has garnered a great deal of attention in recent years as technology allowed for more and more people to become creators or offer their services to others, establishing a generation in which independent craftsmanship has flourished. In most ways, this has been a very positive development, but it hasn’t been an unalloyed good. Unfortunately, much of what is produced provides poor brain stimulation to consumers compared to real-life experiences, leading to further weakening of neural connections.

We can open up our Screen Time app to get reports on how much time we’ve spent in front of our displays, but we ought to be paying more attention to the kinds of activities we spend time on. As we rely more and more on computers for our work and daily lives, brain health becomes increasingly a question of quality as opposed to quantity, like evaluating food: we need to be more attentive to what we consume, and not just how much (pro tip: doomscrolling on Twitter or rewatching that 1,000,034 TikTok dance challenge isn’t exactly health food for the brain!). Creators craft content that, in most cases, stimulates and strengthens their own brains, but their consumers are mostly wasting time staring at screens, and that’s a development that has long-term consequences for brain performance. Fortunately, this paradigm is about to change.

The Unreal is the New Real

This illustration is an example of the peripheral drift illusion. Look at the center of the image, and you’ll think it’s animated (and no, it’s not an animated GIF). The illusion is due to a slight difference in the time it takes the brain to process different luminous intensities from a particular area traveling in a given direction. This lag ticks the brain into perceiving movement, even though in reality there is none.

A peripheral drift illusion by Paul Nasca

Of course, this example isn’t intended to suggest that objective reality doesn’t exist — just that what we experience as reality is constructed not only from what our senses tell us and the way our brains interpret them, but also from our histories, experiences, belief systems, and more.

With the advent of Augmented Reality (AR), we can see and even feel our environment in enhanced ways, with an experienced “reality” that could previously be approximated only by the imagination.

Source: Layered — Neuro-Insight study. Mar 2018; n=151 UK smartphone users. Brain activity measured using SST headsets; unit of measurement is radians, which equates to strength of brain response.

In a research project by Neuro-Insight UK in partnership with Layered, researchers found that, across a series of cognitive function measures, AR experiences delivered almost double the levels of engagement compared to their non-AR equivalents. This is a clear sign of AR’s ability to generate a more powerful response than equivalent ‘non-AR’ experiences.

Moreover, in Layered’s research, it was found that the part of the brain responsible for memory encoding sees almost three times the level of activity while experiencing the AR version of an activity compared to the non-AR version. This could mean that AR can be a powerful way to deliver information for long-term retention.

AR and the Experience Economy

The term ‘Experience Economy’ was first coined in an article from 1998 by B. Joseph Pine II and James H. Gilmore. In their article, Pine and Gilmore describe the next economy as an “experience economy” in which companies must stage memorable and engaging experiences for their customers. The memories themselves become the final product that they consume.

The Progression of Economic Value by B. Joseph Pine II and James H. Gilmore

Joseph and Gillmore claim that an experience consists of a service as the stage and goods as props to engage customers in a way that results in a memorable event.

Disney, Apple, Amazon, and Nike are just a few examples of companies that go beyond offering a service or goods by also focusing on delivering a wholesome experience.

What Makes an Experience “Good?”

Participation

The transition from observer to participant is a key element in any experience. In most digital experiences these days, we’re mostly just passive observers and not active participants. Active participants play key roles in creating the experience.

In Augmented Reality, you are the experience, or at least in the center of the experience. An example would be redecorating your apartment: perusing a magazine or editorial photos of interior designs might spark some ideas or excitement, but having your very own room or apartment redecorated right before your eyes in 360 degrees would be far more engaging, exciting, and memorable. If we were to apply the research mentioned above, we might expect interior designers and furniture companies to start enjoying far more success using AR, since their potential customers would be not just engaging with their content passively, but enjoying far more memorable experiences.

Connection

Joseph and Gillmore mention another dimension to a good experience: connection, or environmental relationships. Connection is a spectrum. At one end lies absorption, and on the other, immersion. Think about a TikTok video — the creator of the video is immersed in the experience they’re creating. On the other hand, the viewer absorbs that experience as they sit in front of their screen, and that’s where their engagement ends.

In an AR world, these two different personas might connect in an experience where both creator and viewer are immersed — where both see, hear, and maybe even feel the experience together. Perhaps, in such a future, our attention span might last more than 15 seconds as we find ourselves immersed and invested in more meaningful and memorable experiences.

Experiences in the Real (Augmented) World

Understanding how our brains work is an important part of building the next generation of augmented experiences. You may be wondering: how much of a change will this be? Just as smartphones did, the next AR platform will change everything we know. Here are a few examples.

Advertising

There’s a reason why Mark Zuckerberg is fully committed to building Facebook’s own Augmented and Virtual Reality headsets: he understands that the future of Facebook relies on this platform. Advertising, Facebook’s primary revenue stream, is about to change significantly.

With today’s advertisements, we participate mostly as absorbers. Today, it’s rare to see an advertisement that truly has an impact, one that creates memories. Over the years, we’ve moved from billboard and magazine advertisements to radio and TV commercials, from web banners and website popups to mobile stories or push notifications. The ones that stand out in your memory are probably few and far between.

With AR, advertisers have a chance to greatly increase their chances of selling, or at least creating, lasting brand awareness in their potential customers. Forget about popups and imagine what advertisements might be like if they were mini-games or short stories where you’re at the center of the experience. We’re already seeing examples of this type of experimental advertising from different companies, like Snap, Inc. or such as this one by Lexus in which the user is able to personalize a car, learn more about its elements, and save and share it with others.

Arts & Graphics

Creatives that specialize in 2D or 3D graphics are poised to truly benefit from Augmented Reality. Today, in the year 2021, more people are familiar with NFT and cryptocurrency. With all the hype around NFT, what really stands out is the ability of artists to bring their creations into real galleries — virtually real, that is. Moreover, people are eager for ongoing, adaptive artistic experiences and are willing to pay for it.

In an unlimited augmented world, creatives are everything. From ever-changing personal artworks that you can hang in your apartment to new interactive worlds and experiences in your neighborhood, artists would be limited by neither medium (paper, canvas, a wall) nor the practicalities distribution: with no need to print or to showcase artwork on a static website, art becomes more accessible to everyone.

An AR mural by Extra Crunchy, created as part of VMF’s 2020 edition

Education

Augmented Reality could be similarly transformative for education. With its high potential for effective memory formation, AR could empower better learning and educational experiences, ones that you remember long-term. Once AR glasses become affordable and accessible for most, expensive physical materials, equipment, and lab supplies needed for the learning experience may no longer be needed.

Just as the Internet gave us the opportunity to access unlimited information resources, AR now offers us unlimited ways to experience, learn, and discover the world from different points of view.

Froggipedia app using Apple’s AR to make a virtual frog. No frogs were hurt in the process

Entertainment

With the rise of the creator economy, more and more people are re-inventing themselves as influencers. Social platforms give them a voice and a following. With Augmented Reality, influencers can enhance their presence in ways that weren’t possible until now. For example, most music entertainers these days make the bulk of their revenues from live performances and brand collaboration. Creating stadium-scale concert experiences was once something that only major stars could do due to logistics and expenses. But if you’re a performing artist today, you can use Augmented Reality to put yourself on an amazing stage with all the glamour of a Beyonce, and more — the rest is up to you.

A Fortnite concert with Travis Scott — YouTube

Nothing Ever Ends

In a world that’s constantly changing, AR experiences are refreshing: familiar sensations in a modern medium. As human beings, the capacity to experience is deep in our core, as it has been from the dawn of civilization till now. Our experiences mature into memories, ones that remain with us in an ever-changing world, and perhaps even beyond our days on earth, thanks to ever-evolving storage media technologies.

Augmented reality holds within itself an exciting future, one that has the potential to truly transform humanity, just as print, TV and the Internet did in the past. In the near future, the question we’ll ask won’t be whether something is real or “augmented,” but rather, whether a thing is meaningful: is that thing you just experienced meaningful enough to be remembered?

Augmented Reality experiences are far more than just a new technology, a new medium, or a collection of 3D objects floating in space. AR is about the way we perceive reality and how we can shape it for the better.