UTK Off The Rails 2/3/24

One Vision

I have seen the future and it’s a few inches from my eyes right now. I’m typing this - or “typing this” - on a virtual keyboard**, projected into my vision by Apple’s Vision Pro. Apple’s “spatial computing” platform launched on Friday and I was able to get mine.

I elected to do an in-store pickup. Apple’s Indianapolis store isn’t the proving ground it was when native Angela Ahrendts headed up retail and by the late afternoon when my appointment time was, some of the buzz was off. The place is always busy - it’s the only Apple store in the state - but it was normal busy, not iPhone launch day busy.

At my appointment time, a nice Apple employee (Mina, who was great) walked me through a tour, showing me how to put on the Vision, the basics of getting around, some of the experiences, and a highlight reel of movies. While you’ll remember the rhino and shark, it’s the initial experience of spatial videos that stand out. There’s a clip of a birthday party and it felt so real, it was almost intrusive. Even though I have watched every video and many reviews, that actual hands-on - or face-on - is still an experience. I highly recommend doing a demo, even if you don’t intend to buy one. It’s a free experience and one that might change your mind.

The headset itself, you’ve seen. It looks and feels like an Apple device. Everything is a shade of white or silver, save for the screen in front. That “EyeSight” screen is the biggest luxury and it’s not experienced by the user. How others will take to it is interesting and for me, I’m likely to just take the headset off for any non-trivial in-person conversation. To the outside world, it’s an indication of whether the user is immersed or actually “looking” at you. My guess is that on cheaper* future models, we’ll get a simple on/off light to indicate similar.

Inside the headset, there’s a couple interesting things. The first is “breakthrough.” Even immersed (and as I said, there’s a visual signal), when a person comes up to you, they fade through, like they’re coming out of a fog. It’s a bit less good at picking up when my dog wants to be seen, but it’s an easy twist of the digital crown to fade out and get back to the real world.

Or at least what looks and feels like the real world. It’s not. It’s on screen, but there’s no (or so little) lag that it feels absolutely real. In commercials and the guided tour video, one of the things Apple doesn’t point to but definitely shows is that people can move through the world without taking it off. While the field of view is more limited than my natural view, it’s not so much that I can’t walk normally from my office to the kitchen to get a drink. While people are looking for the killer app, it might just be that you can move around normally.

I was sorely tempted to put this on and walk into Starbucks or somewhere, just to see how people reacted. My better self resisted, though I’m guessing we’ll see a bunch of YouTubers doing it. The fact is, I’m hoping no one walks through a city or even thinks about driving, but we all know someone will, for the lulz and the clicks and the views. I do think this would be an AMAZING backseat device. I’m not sure how long it will be before little kids swap from their ubiquitous iPads to something like this, but it’s probably not as far off as we think.

Would I wear this on a plane? You know, I’m not sure. The size makes it just possible to stick it in my backpack and there’s enough storage to put in a couple movies. Wifi on airplanes is always a joke, so I couldn’t do much. I think the iPad is the better device for this, or even the iPhone, but I’m not ruling it out.

I’m also curious about how we’ll develop a “best position” for this. Sitting at a desk makes sense if you’re in work mode, but for a movie, laying back might be better. If so, we’re going to really have to reconsider our ceilings***; the fan in my living room makes placing windows awkward if i’m looking up. The weight of the device isn’t so much that it gets oppressive in either and it’s currently really only limited by the battery. With the original Oculus, it was best done in a swivel chair, as so many of the experiences required you to “look around.” I can remember a U2 video where each of the members were 90 degrees from each other and an African safari where you heard the elephants, only to turn around and see them. My guess is in the future, we’ll have something akin to a massage chair with haptics.

I think the key thing here is that this immersive experience effectively makes the world a screen. As I sit here, I have a pair of screens over my eyes, but on my desk I can see my Mac, my iPad, my iPhone, and my Alexa (the one with the big screen that tracks you.) I have a Nebula Projector which could put a screen on my wall if I hit a button. That’s a lot of screens. I’m not that old, but I’m old enough to remember the 11-inch black and white TV I took to college and the 9-inch screen in my first Mac. Oh, there’s a screen on my wrist too.

There’s a world where the Vision replaces all of those screens, plus the TV in the living room. It’s first gen to be sure, but this is a flag in the ground moment precisely in the way Apple does with first gen tech. It’s a statement of philosophy; this is where we start. There’s a Disney Theater in the Disney+ app and it looks and feels like you’re in a theater. In the Imax app, it looks like an Imax theater. It goes beyond a screen experience.

This is the culmination of so many things that it’s hard to see that Apple has been working towards this for better than forty years. I remember the launch of the Mac (and the Lisa! And the PC Junior!) and there’s still some of the Mac in here. It can be used as a computer or even an extension of a computer. I regularly cast from my computer to my projector, which puts up about a 100 inch screen on my wall. The quality difference is huge. The projector is admittedly 1080p while the Vision is better than 4k per eye.

The digital crown is straight from the Apple Watch and what once seemed like a skeumorphic remnant has become a pretty good controller for the small device. It seems perfectly natural here, even placed high on the device instead of on the side. On the side, we get the MagSafe-like power connection and speakers that are world class, yet vanishingly small. The sound is equivalent to a theater, or at least my living room surround sound. Add in that we get AirPods like “transparency” and noise cancellation and it’s solid.

About that power and battery. It’s fine. My guess is we’ll see bigger batteries from third parties. Keeping the weight off the headset and in the pocket is fine, though getting the length of the cord is going to be an issue. You don’t want too much or too little. A third party needs to have one that’s more like a reel, or a dog leash, that has a bit of give.

(A digression: Years ago, Billy Idol was a user on The Well - I can’t explain The Well at all, IYKYK - and he’d just put out his underrated Cyberpunk album. It holds up well, as does the work of William Gibson that inspired it. The question I asked was whether the Walkman was the first step to virtual reality, in that you could walk around the normal world, but change one sense to anything you wanted. People were using Walkmen to do guided tours of cities and museums, and he agreed with my take. I still think it holds.)

As for the feel, the standard band is very … you know what, it’s Nike-like. It’s a very knit feel with a Boa-like tightener to make fine adjustments. I don’t know if Nike uses a Boa on any of their shoes, but this feels like something they would make, and the companies certainly have been allies. Is it comfortable? It’s an adjustment. If you’ve ever played a sport where you needed a helmet, I think you’ll get used to this quickly. If you’ve never had something heavier than a hat on your head, it’s more unusual.

The apps and experience have to be personalized. I can tell you that it looked like a dinosaur was in my room, but while it’s a nice demo, it’s not useful. Watching a TV show (or my Mac screen) on a giant monitor, or multiple monitors? It’s the wow moment. I haven’t watched a full movie yet, but I can tell that certain things are going to work on this better than others. The 3D movies actually look like 3D was supposed to. If there’s a way to get the kind of immersive experience for a Taylor Swift concert, Apple couldn’t make enough of these. I wonder if the Apple Music halftime show will be captured in spatial, or whatever we call this format. The TV in my living room suddenly seems small.

I started wondering if the way we calculate screen size is going to change. I sit closer to my laptop than my TV, so the 13-inch screen probably takes up more than the 75-inch screen. Would “eye percentage” be a better measure of real function? The Vision makes that irrelevant. Everything is screen. You can choose to put smaller screens inside the virtual world, or you can watch on an IMAX sized virtual screen. It’s. All. Screens. Now.

Third parties will do some things with movies and other content, but 3D movies have always been a bit of a gimmick. I referenced Strange Days when the Vision was first announced and I still think that’s going to be a thing. It’s going to be difficult to capture a lot of things in that spatial POV - though porn will be easy - but for the last couple months, I’ve been capturing things in the iPhone’s spatial format. While it’s far from perfect, it is a “you are there” feel. I have a clip from the Damien Rice show I saw in December and it’s exactly like I was there. Granted, I had seats way in the balcony, but that’s what it felt like.

Aside from porn, I’m trying to think of what could be usefully captured. The answer is anything, especially if we get the “good enough” quality from phones. Imagine you’re a player running through the celebration at the end of the Chiefs-Ravens game. “Oh look, there’s Travis and Taylor. There’s Andy Reid. Look at all this confetti.” We already see cameramen running onto the field after homers and touchdowns to get a great Megaladon view. That, in spatial, is possible now with nothing more than the phone in our pocket. I’m sure ESPN can do better.

I think personal things like the birthday party in the tour, surprise parties, and the like will become things, in the same way that we went from faded sepia photos to 4k videos in a generation. Vacation clips? Maybe, but no one wants to watch someone else’s vacation, whether it’s on a Kodak Carousel or an Apple Vision. Vacation homes? I can see AirBNB, or realtors, using this as a this-is-what-you-get tool.

(Thinking about it, realtors might be a gateway. I can imagine a realtor simply walking through all their homes with an iPhone, then showing their clients the homes on a Vision Pro in the office. Sure, they’ll want to go see the ones they like, but it could be an amazing first experience for a lot of people. I’m not alone in this - I just noticed that Zillow and Wayfair both have day-one native apps.)

Actually, a lot has been made of no YouTube app on the Vision, but after using it and recently re-watching Strange Days, I think there’s an opportunity here for a YouTube of spatial, but paid. (The model is probably OnlyFans, but let’s keep it clean.) I don’t think people will wear this to concerts (rights issues), or action sports (dying issues), or many things, but I’m not sure they won’t and they don’t have to. (See above!) Again, it can be captured with an iPhone and I’m sure Apple (and GoPro) have some kind of rig that could be strapped on - maybe something like Meta’s RayBans with cameras? - and used in situations to capture … skydiving. Tony Hawk’s POV as he drops in. Standing at the top of the Empire State Building. Diving to the Titanic. The Olympics from the face of an athlete. Think of those catcher-cam views in baseball. Now those I’d watch.

Of course, I’m waiting for sports. Apple showed some clips (the same ones from the July demo, but this time I could see the dunk, the goal, and the Angels first baseman diving for the ball), but there’s no spatial content yet. Apple’s control of the MLS rights makes me hopeful that they’ll at least experiment and I’ve detailed my thoughts with how they could (and should) do that. Apple still has the rights to Friday night ballgames and with the NBA rights on the horizon, spending a billion dollars on rights now becomes marketing for a device and a reason to own it. Apple doesn’t even need to own it, but they could help a broadcaster get those rights in return for ensuring it’s shot in spatial.

A reminder here: Disney owns ESPN, which has MLB, NBA, NFL, NHL, and F1 rights, plus I guarantee we’ll get something like pickleball jumping in early. I’m also a bit more convinced that there is a case for Apple to buy Disney, or at least tighten the partnership.

Here’s where Apple has a big advantage over Meta. The Quest has games, some very successful, but they have almost no big content. You can watch movies and such, but they’ve done little in the way of pushing content to Quest users. If nothing else, Apple has TV+, which has become what HBO used to be. I think we’ll see tentpole movies, especially Disney, on the Vision and I wonder how Meta will answer that. Apple Music has their live concerts, which are relatively limited but well done.

Meta doesn’t have a tight partner like Disney, but they do own Facebook and Instagram. Is there a native YouTubeTV app for the Quest? The games question remains, as Apple doesn’t have a signature launch game and iPad games are not built for the Vision’s controls, as amazing as they are. (By the way, the gestures and movements become secondhand quickly. When they used the Star Wars clip of Obi-Wan saying “Use your instincts” in the commercial, Apple was being specific.) Meta may end up needing to iterate YouTube, adopt the rival spatial format for short clips, and spend to acquire a content arm like Paramount or Sony. They’ve been bad at that, and worse, bad partners, so it will be a big ask for a company that gets distracted as easily as its CEO. (That Quest will be able to show spatial videos shot on iPhone starting the same day the Vision launches is telling.)

One area I don’t think will be a game changer is video conferencing. Zoom and FaceTime both support “Personas” - a scanned animated version with tons of tracking - but it comes off as fake. A recent clip of what Meta is doing here was mind-blowing, but how does Meta get those high tech scans of everyone (and would you trust Meta to have a high tech scan of you, or would I end up an extra in some Mark Zuckerberg production?) There’s not even a Memoji fun element here, which surprises me a bit. It’s usable and will be a fun trick for a while, but Zoom still sucks.

(I wonder if Apple is worried about DeepFake Personas and that’s why the quality is where it is. There’s nothing to stop me from setting up a Persona that’s someone else, if I can get them to sit still for the scan.)

I’m curious about the idea that something like Apple’s “Personal Voice” might factor in here. This clip is not me:

Asynchronous conversations are how most of us communicate. On the phone or a Zoom, you’re right there. You have to answer now. On text or WhatsApp or the like, you don’t. You answer when you can, or in line at the grocery store, or when you pause the TV. What if I could type my answers, use my Persona, and be sitting quietly in a Starbucks. (Yeah, I’m still resisting going there with this on.) For a generation that prefers podcasts to reading, this might be something.

You know what wouldn’t suck? Some sort of Persona-driven coffee table discussion. Most of my friends don’t live in Indianapolis and while FaceTime is fine, it’s still not natural. What if we could sit around a virtual living room and have a conversation? What if we could be at the same virtual game? What if this device that seems to distance ourselves from others actually brings us closer?

(Actually, if anyone’s interested, send me a message and let’s FaceTime or Zoom, or whatever. You can tell me how my Persona looks to you … and ask a fantasy question or two.)

The apps are already enough and there’s going to be more coming. If you want the wow moment, looking at the stars and having Dark Sky overlay is pretty amazing, though the screens and cameras aren’t perfect for stargazing. The experience with the phone app might be better, but less awe inducing. There’s a flight tracking app that puts a realistic model on my desk and I wonder if this isn’t how air traffic controllers will be doing their job in ten years. Meditation and relaxation apps sound like dumb ideas, but the immersive nature is better than things like Headspace. My guess is that virtual classes - Tom Colicchio is in your kitchen teaching you to make salmon! - will be big soon. Shopping and seeing the clothes or a piece of furniture where you intend to put it is amazing.

I’m a day into my Apple Vision experience, but the possibilities are there. The first Mac didn’t have a hard drive. The first iPhone was slow and didn’t have apps. The first Apple Watch didn’t know what it was. The first MacBook Air was underpowered and overheated. Now, I have all of those and they’re the best devices of their types, until the next iteration comes out. What’s amazing is that the DNA of those first devices are still very apparent even with decades of iterations in some cases. Apple’s very good at getting that right.

(Fun fact: UTK has not always been written on Macs. For the first couple years, I used a Compaq PC. That company doesn’t even exist, except as a shell brand.)

The Vision Pro is clearly one of those first generation devices. It’s not for everyone, especially at this price. Instead, this feels like Avengers: Endgame, where a decade of things came together for a powerful moment. Putting this headset on and seeing the world, and maybe that movie this weekend, is as big a moment as when the Avengers pop through the magic circle and get ready to face down Thanos. I’m hopeful that this is the start of something that in a decade, is even better and more immersive, with a decade of content, pictures, videos, and experiences to bolster the use. The future is here, it’s just not fully baked.

*No Apple products get cheaper. The original iPhone quickly re-established the base and used the more typical carrier subsidies, but iPad, Apple Watch, and Macs have been very stable in the face of a decade of inflation, the switch to in-house chips, and more. I don’t think this is getting SIGNIFICANTLY cheaper, though I think things like dropping the front screen could lead to some price drops.

**I typed the opening graf using the virtual keyboard. Then I switched to dictation, and did corrections and details on a paired keyboard. The whole thing was done in Safari on the Vision Pro, inside the Substack CMS.

***Tim Cook said in his Vanity Fair interview that he “watched the whole third season of Ted Lasso on my ceiling.” First, it tells me Cook’s been living with this device for at least a year. Second, I’m guessing that if he can watch a full, long season of Lasso, he’s figured out that position might be one of the best. I’m curious because laying back like that usually just makes me sleepy, but I’ve never done it with a computer strapped to my face.