This post is an adaptation of a presentation I had the immense honor of sharing as part of the Mixed Reality track at Interaction 20 in Milan on 05 February.

Before I dive headlong into my creative process for creating spatial apps — I would love for you all to indulge me in a small exercise. In your mind — describe the size of the group you are sharing your current space with — don’t spend a long time — what is the first thing that pops in your head.

If you are in a conference with a gaggle of folks or even working in a local coffee shop, you’re probably hearing very factual statements — 5 people, 10 people, less than a 100. These are very straight forward numerical statements, now what happens if we add only a handful of multi-dimensional relationships? How many are wearing glasses? What is the dominant color scheme of their clothes? What about hair length?

We can quickly see how hard it is to communicate multi-dimensional relationships. We use things like similes, metaphors, and abstract statements to try to summarize the audience’s size now.

This was just a quick demonstration of when things are more complicated; they take longer to expound upon. It also demonstrates human’s unique abilities to quickly grasp and rely upon abstract concepts to communicate complex ideas.

My team at IBM’s client experience centers communicate and explain the highly technical and complex ideas behind AI, quantum computing, and IBM’s innovations to clients, researchers, and students.

The IBM client centers use a 290° immersive room to place visitors in a communal XR experience. Our expert-led experiences leverage 93.3 million pixels to demonstrate AI interacting with data at a scale that allows humans to see how machine learning makes decisions, how it interacts with data, and the value it holds for their organizations.

The creative process is your unique ability to think abstractly. Creativity is not high science, there isn’t a magic incantation, there isn’t an app you can buy — and despite every Medium article saying otherwise — there is no hack or shortcut. Every design shop, creative agency, or innovation consultancy follows the same creative process — just in different hues and colorways.

IDEO calls this Divergent (abstract) and Convergent (concrete) thinking. IBM calls these steps in the Loop Make (abstract) and Observe (concrete). Our group takes the same approach but balances it with a story (abstract) and facts (concrete).

Progressively shifting your mental models from abstraction to concrete thinking and moving forward linearly is the creative process. Creativity is a conversation — expansion and contractions, story & facts, abstract & concrete.

Assimilate, create, iterate, repeat ∞

This is the exact same process we use to guide our experiences, but we constrict the time frame greatly. We smash in a week of intensive and utterly exhaustive workshops and co-creation sessions.

We partner with internal technical experts who co-create with our team for a week — we deep dive into how products work, the technical underpinnings, and unique differentiators. By the end of the week — bleary-eyed, exhausted — we are finally iterating, prototyping, and experimenting at scale in our lab in Austin.

We use tools that we can quickly spin up visuals with — things like Processing/P3, TouchDesigner, some C++ explorations in space — even our sketches in a pinch. The important thing is not letting the technology dictate the craft.

Create tension

Our group develops explanations of IBM innovations and technology. Product stories are primarily driven with obviously the product and outcomes. But between products and outcomes lies the crux of all great stories — challenge. Good vs. evil, friend vs. foe, user need vs. product. Every good story or product needs tension — or balance.

We use that story outline to create a decision tree to create a stateful app that tells a multitude of linear stories — much like choose your adventure games.

Design your verbs

Using that flow as an outline, we can now start to align product actions, user inputs, or data insights as story beats that help to progress the experience. We need to break down the product and present it as a story that demonstrates its value using real-world data.

Following that same creative process, we iterate on a single idea — from workshop sketches to testing interaction wireframes, to exploring inspiration concepts, to refining a visual design, until our final design artifact represents AI report creation.

AI as a design input

We even use AI as a partner in the creative process when we create bespoke visualizations for clients using a combination of publicly available data, AI, and generative art.

Art resides in the quality of doing — process is not magic. — Charles Eames

This quote serves as a constant inspiration. The unique thing about humans is we are all born inherently creative, but staying original is hard because we are pre-wired to find shortcuts.

Make — do — create. I don’t need to know C++, I don’t need a +$7000 cheese-grater pro computer. All I need are me and my creativity — the human brain is the best GPU you can buy — use what you have to create.

Grazie mille. Cin cin.

—

This private publication is not affiliated with my employers or professional associations. Personal blog, personal opinions. Not speaking for anyone but myself. ✌️