Telling Good Stories About the Future of Immersive Worlds Futures
A big challenge with emerging technology derives from the inability of its inventors to tell its stories. They can tell you all about features and functions, perhaps a use case. They have a hard time helping you imagine the future world in which the technology they tout so highly fades into the fabric of life rather than playing an emphasized role where every feature mentioned in a use case receives a big red arrow pointing at it.
One of the goals of my Strategic Storytelling class is to enable my students to tell stories about the future that places the reader in the future by beating them over the head about where they are. Scenarios create the context and the rules for the various futures, and storytellers leverage those constraints to create plots, characters, and scenes that live comfortably within those boundaries. Good stories offer details about how to use the technology in context without commenting on the technology directly.
In the remainder of this blog, I share a series of related stories about virtual immersive futures that I hope will create a sense of place, perhaps of wonder at the seeming simplicity of the interactions. These stories are not intended as science fiction—they attempt to tell plausible stories within scenario logic vetted for viability—filled with experiences, plots, and characters the reader can relate to.
Immersive work
Roberta works in an immersive environment for several hours a day. This is not a requirement but a choice. She is managing the workflow for a new website, which includes copy, images and a WordPress instance.
She finds her virtual environment a much more configurable workspace than her little 4×3 desk crammed into a 6×6 hoteled cubicle.
Several windows float before her. One focuses on SEO data and current statistics. Another contains copy. To the right, there are image databases and a photo editor. There is no need to overlap screens or hide things. The immersive environment reaches into a near-infinite space above and below. Roberta has fashioned her workspace so that with a quick flip of her wrist, she moves from words to pictures to content management.
There is nothing new in the software Roberta is using. With all the same software she runs in her regular environment, even her two 34-inch curved widescreens, Roberta feels more productive in the virtual workspace because if she needs an extra monitor, she just conjures it up rather than asking for budget.
At the moment, the task is words. She looks at the word processing document and focuses briefly on the attention icon. The document zooms before her eyes. The other features of the environment dim, and all distractions from alerts to messages disappear and stay gone until she asks for them. Everything stays current, but hidden.
This, of course, is Roberta’s choice. She could have several applications active windows, and often does when she is working with images or content, but when she is writing, she wants to write.
Sometimes Roberta needs a little mental break. Off to the left of her word processing document is a window complete with sun and trees. If she looks out the window the lights come up, and calming music starts playing in her headphones. Clouds roll past the window. If she leans in, or navigates over, she can lean over the windowsill and look out at the Rhine Valley as it was in 1650, wooded, with the river meandering through small villages. She wrote her thesis on German literature and always had an affinity for the Rhine Valley. Now it is her go-to place.
After several minutes of mental relaxation, she turns her head right and sees her document floating in space, now somewhat less threatening than before. A momentary focus on the document brings it back front and center.
With the copy complete, Roberta checks it into the repository and grabs a set of windows representing the image needs. The background shifts to a uniform grey so as not to distract from the images she needs to evaluate and edit. Her suite of tools includes a photo editor and a clip art repository—she seamlessly drags-and-drops the images so she can easily flip through the stack, or with a wave, spread them all out before her.
Roberta drags the images, copy and design elements around.
One of the new pages is immersion enabled. It opens in a separate window above and to the right of her workspace. A boat sails out of the page, tipping and turning on turbulent waters. That is certainly a better illustration than a static boat when trying to describe how you help companies navigate turbulent markets, she thinks.
She then scrolls down to the slug and the SEO section, where AI-suggested keywords, phrases and meta-description await her approval.
Roberta says, “OK then,” and the code behind her design generates.
Well, that’s done, she says to herself. She takes a deep breath and navigates back out over the Rhine River.
Immersive Design
The in-coming collaboration request from Bangalore chirps off to the left. A smartwatch tingles on Leslie’s wrist. Priyanka will surely want an update on the kiosk design.
Leslie looks left and focuses on the chirping collaboration request, and moves slowly toward it. It dissolves into a perspective room. Front and center of the room is the kiosk for the upcoming conference booth. Next to it stands Priyanka’s avatar, looking as well-rested and well-dressed as ever.
Her avatar certainly doesn’t hint at the late-night dancing she shared earlier, Leslie thinks.
Getting immediately to work, Priyanka says, “Good morning Leslie. I brought a couple of prototypes I thought we would figure out how to integrate into the kiosk.”
“How many products are we planning to show? Last I heard, it was three.”
“Five.”
“Oh, five. So, this design isn’t going to work. We’ll have to rethink the concept.”
“Not so fast. I like the original concept; it just isn’t the right geometry. Let’s see, we are standing inside of an active geometry space, so let’s take a look.”
Priyanka reaches out toward a seam between the product display projections and pulls at it. The design splits open. She opens and tugs at the split. Now a gap sits in the bent design. She places her hand on one of the adjacent display projections and says duplicate. She drags the duplicate and drops it into the gap. The model reforms around the new side and becomes a four-sided kiosk.
“Repeat,” Priyanka says, and without the simulated physical fanfare of the previous design change, the kiosk transforms instantaneously from four-sided, to five-sided.
“OK, I get it,” Leslie intones. Not wanting to appear a mere observer, she then adds, “show the conference floor and the traffic. Camera, eye level from the main entrance.”
Leslie and Priyanka now look out on the conference floor from the incoming mezzanine. They no longer see their kiosk. It is obscured by several other booths and various displays between their kiosk and the door.
“Raise the central cylinder 18 inches.” Nothing appears to happen. “24. 36…72.”
©2017 Daniel W. Rasmus
From the sea of kiosks their cylinder and company logo rises above the floor, visible to anyone 5-feet-seven or taller.
“Return us to origin,” Priyanka clips.
They are now standing back at their kiosk, looking up at its tall central cylinder and functional product display areas, all five of them, placed equidistant around the base.
“Nicely done,” Priyanka acknowledges. Now let’s take a look at what goes where. She sweeps her in a lateral motion and a table of products appears. Leslie does the same—signs and stacks of literature appear on a table.
The two women move their respective items to the various locations, Leslie allowing Priyanka to choose which prototype goes where before she places the product literature, the signage and the business cards.
One of the products is about ready to ship, the others are “vision” projects. On that side of the kiosk, Leslie mounts a couple of proximity tags. When she gets close to one, the face of the product designer appears. He says, “welcome,” and proceeds to explain the product’s benefits. People wearing augmented glasses and sensor identifiers will receive a notification that more information exists at this station. They will have the option of it playing inside of their UI. Those visiting in the immersive environment will be notified as their avatar wanders by. The engineer will also greet them.
Leslie looks up occasionally at the green-hued center of the perspective room’s ceiling. It isn’t really green, but it is for her, which means her agent hasn’t identified anything more important than this task with Priyanka coming from any source.
The conference kiosk model ties back to the design files and the executables. Satisfied with their designs, Leslie looks at Priyanka and says, “submit.”
One of their vendors receives a fabrication, another receives a print request. The virtual conference team recovers links to the files, along with the prototypes (all of which work in the virtual world much as they would in the real world) and the various documents that represent product literature. Whether a person shows up virtually or physically, they will receive much the same experience, minus the handshake and the chocolate kisses, which Leslie will bring with her to the conference floor.
Immersive Professional Development & Training
Looking up, Samuel saw the sky was red. A message of some importance awaits him. He finished his initial analysis of the latest manufacturing downtime details before saying, “retrieve messages.”
Perfectly placed in front of his eyes and filling his field of view was a message about training on the new pick-and-place machine for the printed circuit board manufacturing line. This wasn’t a surprise, in fact, Samuel could see stacks of related messages behind this notice. Dim, and to the left, he saw his calendar with a note that this was the day to expect a schedule request for training.
Samuel simply said, “cool,” and the notice disappeared, zooming into his calendar and leaving its mark at 8 am on the 16th. If he wanted a reminder of details or access to reading material ahead of the class, he could tap on the calendar entry inside of the immersive experience or on his wearable—or he could ask and the information would be delivered, and appropriately formatted, to whatever interface he was currently paying attention to.
On the day of the class, Samuel put on his glasses and navigated to the pick-and-place class. The Yamaha I-Pulse M90 class would be taught by Masato Tanaka.
Tanaka, the principal engineer on the M90, created a multilayered learning experience. Over 60 students sat somewhere, ready to learn. They were connected by audio so they could talk when Tanaka permitted it. Otherwise, they all saw and heard the same thing.
The M90 was installed in a modern surface mount manufacturing facility where it could participate in the manufacturer of virtual printed circuit boards. After reviewing board types in 3D ahead of the class, the majority of the students had selected an automobile audio amplifier as the example. Those who didn’t win the vote could later ask the simulation to make their board. Which board didn’t matter; the activity was really about getting students engaged before the class.
Tanaka wanted to cover four areas during this first session:
- Capability and line balancing
- Programming
- Failure modes
- Basic failure intervention
During the capability session, Tanaka walked the students in and around and through the machine. They watch as the head spun by right in front of their faces.
They took a break. A few of the students saw that their sky had turned red, so they went off to answer messages. Others starting tapping out messages to friends and family after class activities.
At about 3 pm, which was 8 am in Japan, the real Masato Tanaka replaced his avatar’s programming. No one noticed. The avatar stopped working via pre-programmed actions and now reacted to Tanaka’s actual movements and mapped his lips and expressions to his face. Only during questions and answers did the student figure out the real Tanaka was in the room, as no avatar was as good as the principal engineer at answering questions.
Samuel was ready to get this machine up and running. Priyanka would want to turn her prototypes into production units if feedback at the show was good.
Immersive Medicine
“It’s going to require a tendon repair,” Dr. Alvarez stated matter-of-factly. “See, this tendon pulled off the ulna. The bicep rolled up into your shoulder. Let me look at the simulation. See how you can see the muscle in a wad up there?”
Hugo was sharing a very uncomfortable immersive moment with Dr. Alvarez, but it was kind of a relief to be looking at something other than the real lump in his upper arm.
“Follow me, Hugo,” and Dr. Alvarez navigated to the lower arm, pulling back and out of the anatomy. He rolled the camera angle so Hugo could see the back of his arm.
“This is where I will make the new insertion. I’ll show you in a minute.” The camera rolled again, and the inside of Hugo’s arm, opposite the elbow, lighted up. “And this is where we I will pull down the tendon.”
“Now let me show you the gory part.”
The view was pretty real but not really gory. It could have been, but that wasn’t the point here. Alvarez wanted Hugo to understand the procedure in the abstract, without the instruments, the suction or the sponges.
Dr. Alvarez reoriented the camera to the back of the lower arm, made an incision, exposed the bone, and then drilled a hole in the bone. There was a loud whirring sound. Alvarez quickly hit mute to kill the metal-on-bone screech. It was enough to get Hugo’s attention so that even in the abstract, he winced a little.
Alvarez then flipped over the view and opened a large incision. He reached up with simulated forceps and pulled the tendon down, pulling it through to the back of the arm, then pushing it into the hole. He then sutured it to the bone.
“See, my friend, this will never happen again. At least not to that arm.” Alvarez smiled. The room lightened a little.
Because people couldn’t see each other’s faces in situations like this, the experience itself was designed to reflect the mood. When Dr. Alvarez smiled, the room reflected a positive perspective, lighting up with Hugo’s favorite pastel, which ended up being a kind of lime green. Feedback sensors in his wearables, coordinated with moments, reinforced that when Hugo saw the green light, his heartbeat and breathing slowed ever so slightly.
“So what about the scar?”
“Well, let me show you a range. I’m an orthopedic surgeon, not a plastic surgeon. You can pay extra for a different doctor to close. I use staples. We’ll go over how to care for those after surgery. But here, take a look.”
The camera moved out again, and there was Hugo’s arm, in its pre-surgery state, but with the rolled-up bicep temporarily relocated to its proper place.
“So, this is pretty much worse case after a year. These are big incisions,” Alvarez reminded Hugo. “They take time to heal.”
“See, the back is smoother. That inner arm takes a lot of stress, so it tends to welt up more around the scar just from movement—and here’s the view if you bring in a plastic surgeon. Remember, you’ll have to get special permission from your insurer for that, and that’s not likely. Otherwise, it’s out of pocket.”
They discussed the fee, and Hugo decided the scar made him look tough.
“You can just tell people it was a knife fight,” said Alvarez, “I even had one patient work it into a tattoo.”
A beautiful dragon formed on Hugo’s arm, the scar strategically worked into the design. He was intrigued and leaned in, seeing how the artist worked the minute bumps and ridges into the art.
I’ll have to keep this solution in mind, Hugo thought to himself.
The surgeon and his patient’s glasses clear.
“So, there you have it. In the past, I would have just told you about it, maybe shared a picture or two. We had video, but nobody who isn’t in medicine wants to watch a real surgery. I’ve decided this is a perfect way to inform the patient without grossing them out.”
“Well, it worked for me. I don’t like that it has to be done, but I’m certainly comfortable that you know what you’re doing.”
Hugo and Alvarez shook hands, left to right, as Hugo’s right arm would remain in a sling until the surgery.
He looked down at his arm and thought: That damn new pick-and-place machine. If I hadn’t tried to be a hero—if I had just waited to get Joe to help push that crate up, this wouldn’t have happened.
Conference Day
Priyanka sits back in her chair and flips all the sensors on. She wants to be at the show as much as possible without actually being there. Priyanka is attending via a sensor array at the end of a poll being carried by Leslie as she walks the floor. If the camera gets tussled, Priyanka feels the bump in her chair.
“I want a close-up of that one.”
Leslies communicates to Priyanka via audio. She can switch between the sensor array and Leslie’s glasses. With Priyanka in full headgear, she can see Priyanka’s very expressive eyes. She knows her well enough to read her face, even with the goggles on.
“Yes, please have them tell you more.”
Leslie looks around for a representative.
“While you are trying to get someone’s attention, can you flip around and point me at the booth, eye-high, so I can see what it looks like from there.”
Leslie accommodates. Priyanka is pleased that the logo and the new branding colors are easy to see, even from a couple of aisles over.
A voice says, “may I help you with something?”
©2017 Daniel W. Rasmus
Leslie says yes, and introduces Priyanka. The young woman in the booth addresses both Leslie and the camera as she explains the product. Leslie and Priyanka ask a couple of questions, Priyanka as a disembodied voice coming from Leslie’s glasses as they automatically switch from headphones to speakers.
The two women both thank the representative. Priyanka says hold on, I’m going to take us vertical. Leslie watches as the image of Priyanka disappears to be replaced by a view of the show floor from above, the booth locations and footprints aligned precisely with the actual booths.
“Over there,” says Priyanka. “Over there.” Pointing first, then highlighting a path that is rerouted immediately, given congestion inputs from the floor sensors. The destination remains the same.
It’s another competitive product.
They circle back to their booth. Priyanka surreptitiously listens to several customers compare these products to hers.
“I think we may have to rethink our rethink,” Priyanka suggests to Leslie. “We seem to be planning on making pretty much what everybody else is planning on making. Glad we didn’t send a big team. Our prep work is going to pay off, but it looks like we’ll need to work through some of the feature details over the next couple of weeks. We aren’t a ‘me too’ product company.”
Leslie and Priyanka continue to explore the floor, looping in Edgar Martinez in editing, who turns the private expedition into a trip report that will help make the case for the need to innovate a little more before taking the new products to market.
For more serious insights on strategy, click here.
For more serious insights on XR, click here.
Leave a Reply