Why Vision Pro Will Change Photography

Over the past few months, I have had the opportunity to spend time with Apple’s soon-to-be-released Vision Pro, a spatial computer that you wear on your face. It is the next step in the long evolution of mixed reality glasses that started with the awkward, and ahead-of-its-time, Google Glass.

I have already stated on multiple occasions that Vision Pro is going to be a device that redefines our relationship with visual media and content—from movies to live sports to our home videos and photography—the display is going to leave us wanting and asking for more from our devices.

Although my time with the device has been limited — and I have yet to play a single game on it or do any kind of work on it—it is hard not to be excited by the possibilities, especially for photographers (like myself) and filmmakers.

But first, spatial video!

My initial experience with the device was soon after it was launched; the next time I got to use the device was when Apple released the developer beta of iOS 17.2. That software had an update that gave the iPhone Pro models the ability to capture spatial video. Since then, Apple has released iOS 17.2 as an update to all phones.

As an iPhone 15 Pro user, you can go to your general settings and turn on spatial video by going to Settings → Camera → Formats. When you capture a video — sadly, only in landscape mode for now — you’ll see a Vision Pro icon. You can turn it on, and videos can be recorded as spatial videos.

By now you might be screaming — what the hell is a spatial video? Spatial video is a mixed-reality video format that allows videos to record the depth and spatial information of the scene, and when you play it back, you get a more immersive, three-dimensional (3D) experience. The iPhone 15 Pro utilizes its main lens and the ultra-wide lens to capture the depth and spatial information of the videos. The spatial videos are captured at 1080p, 30 frames per second, and use the HEIC format.

When viewing on a Vision Pro, it is a unique enough video experience to draw a gasp. Of course, given that the Vision Pro display is over 8K, your eyes can tell you that the spatial video is of slightly lower quality — 1080p. Why the lower quality? The ultra-wide lens captures images and videos at 12 megapixels. As a result, it doesn’t have enough information to create a field of view to match the normal (aka main) camera.

The playback of the videos happens in what seems like a hazy, light, borderless frame, giving the videos a dreamlike quality. It is a very strange feeling — as if you have been transported back in time — and the videos have a dimensionality to them. The spatial videos I experienced felt more like memories — somewhere between reality and an abstraction of it.

During my visit, Apple asked me to visit a special area where a sushi chef was making sushi, and I captured the video to be played back. I zoomed into his fingers massaging the rice, the sushi on the plate. The video was absolutely stunning, but clearly, it lacked the emotional appeal of a family video. On a recent visit, one of Apple’s team members took a video of me walking through the Apple Orchard toward the camera. It was almost as if I was walking out of the frame.

Long after we had moved on from the videos and demos, and I had returned home, I was left wondering — what if I had had the iPhone 15 Pro with me in Greenland? What an amazing way it would have been to capture the majestic beauty of the landscape. What would it have been like to capture inching closer to a mammoth iceberg? Or just having a cable car approaching slowly closer to me.

I was saddened that I couldn’t capture the video of Mr. Gibbs, my dogson, who has now been missing for almost a month. I miss him so much — and it would have been nice to relive his presence, even if just in video form.

Spatial videos — whether from Apple or anyone else — remind me of that scene from Mad Men, with Don Draper pitching the Carousel, a round photo projector.

Technology is a glittering lure, butthere’s the rare occasion when the public can be engaged on a level beyond flash. If they have a sentimental bond with the product, this device isn’t a spaceship; it’s a time machine. It goes backwards, forwards. It takes us to a place where we ache to go again.

In time, we will see creative filmmakers start to think of using spatial videos as a new format to tell a story. We are not there yet — but I can see the possibilities.

A Photography Vision (Pro)

Now, let me turn my attention to the photos and what Vision Pro means for photography. During my first visit, I got to spend a lot of time in the “Photos” app — where I looked at photos that were part of a library curated by Apple. The library was full of all sorts of photos shot with the iPhone 15 Pro (Max) — landscapes, fashion, lifestyles, everyday snaps, family photos, and of course panoramas. These were obviously shot by professionals and in good light. Granted that these were images of top-notch quality — still, they were made with a smartphone, and not a standalone high-end camera. They were astonishingly good.

When I opened the app and then opened a photo, I could pinch and expand the photo as if I were viewing it on the screen of a reference home theater. The photos seemed to have enveloped me — a feeling of floating inside the photo itself. This is such an alien, and yet an intimate, experience.

We are so used to looking at photos zoomed in — on the screens of our phones or our laptops. We never quite experience the magnificence of the entirety of the photos in our daily lives. This is exactly the opposite — there is a whole new appreciation of being there. And that’s why I believe we are going to be thinking about our photography very differently.

For example, smartphones have influenced portrait photography — both as classic portraits and as selfies. With selfies, there’s a tendency to frame faces in close-up shots more frequently. Since selfies emphasize the face, they leave out the environmental details and highlight facial expressions. Selfies and portraits today are in “portrait” or vertical formats. However, they would look awkward on the Vision Pro, for sure. What looks great on a tiny screen wouldn’t be that amazing on a cinema screen, and I wonder if we will start to see a new way of thinking about selfies.

Whether it is portraits, lifestyle photos, or everyday moments, the idea of a “gigantic screen” means we might think differently about photo composition. For the phone (and the small screen), we tend to frame faces or the subjects to fill the frame — aka closer, tighter shots.

Smartphones also allow the use of different angles and perspectives. They allow more dynamic framing — shots taken from high or low angles, or from perspectives that were less common in traditional portrait photography — can now be put to work . Vision Pro opens up opportunities to think about this in more traditional terms — using negative space and the environment with a device like a smartphone can help open up creative opportunities.

Okay, I got a little distracted there!

Next, I opened the panoramas—from Iceland, and from somewhere in the Southwest of the US, I think—it was as if I was completely immersed in the landscape. I looked to the left, I looked to the right. I looked up and I looked down. The ice and snow, the feel of cold was everywhere around me.

Sure, these were probably captured by an expert, on a tripod with a smooth ball head mount, and there was minimal shakiness that normal people encounter when capturing a panorama — but still. The feeling of wonder and elation at suddenly being transported to someplace — it is very trippy (not that I would know anything about that). As a photographer, I have always hated panoramas. I don’t capture them on my phone or my regular camera. Well, I don’t hate them anymore. I am going to be thinking differently about panoramas as a way of creative expression in the near future.

What about my own photos? How do they look on the Vision Pro “screen?” I posed that question to the Apple team, and eventually got an opportunity to do exactly that. As part of the experience, I created an album with iPhone 15 Pro photos and spatial videos. I also included some photos I took with my iPhone 13 ProMax, iPhone 14 ProMax, Sony RX100, and also a handful of Leica SL photos. I didn’t realize that most of my iPhone 15 photos and videos were in RAW & ProRes modes — aka unprocessed. While the rest of them were in the native iPhone formats, except for Leica SL photos that were 2 megapixel jpeg files.

I opened the photos and then expanded the photo as much as I wanted — I really wanted to pixel peep. The iPhone 15 photos, despite being in RAW, looked amazing, even if they were a little flat. I could really zoom in — and I mean really zoom into it, and could look at the finer details. I couldn’t say the same of the photos from the older devices — and it is clear that the larger screen you start to see artifacts. Even the RX100 photos had some artifacts — mostly because they were saved on my phone in very low resolution.

Higher megapixels clearly help with the quality. For example, I was looking at one of my B&W photos of a boat in San Francisco that I took recently with the ProMax — and I realized that there was a bird sitting on the boat. I didn’t realize that on my phone screen, or even on my laptop.

One of my Leica SL photos was that of a cloud over the San Francisco Bay — not only could I see the reflection of the light on the water, but I could also really zoom in to the boats in the port on the other side of the bay. When I was done with the session, I was ruing the fact I didn’t include some of my larger photos — the ones created with my Leica SL (24 megapixels), Leica SL2 (47 megapixels), Leica M11 (61 megapixels), and PhaseOne XT (150 megapixels). I wonder if, unknowingly, Apple has given me a reason to keep suffering from the “gear acquisition syndrome” and upgrade my cameras.

The ability to see on a screen that is 8K+ gives you a new appreciation for photos. The only analogy I can make — going from viewing photos on your 2012 MacBook to a 2023 MacBook Pro. I remember editing photos on my 2017 iMac, and then I got the Apple XDR display — and suddenly, photography changed for me. I could see a lot of minute details, color tones, and contrast. The nuances of the image, the composition, and actually noticing the little things in the field — all became part of my photographic experience. It transformed how I thought about my pictures.

Even though I print some of my photos, most of them end up on my 75-inch Samsung Frame, a 4K QLED TV that is optimized for “art” viewing. Even though it is a step down from Apple’s XDR display — it is a great way to enjoy the landscapes. However, the Vision Pro display is an entirely new beast and would add new vigor to photographs I capture and then edit. Vision Pro is yet another part of the ever-evolving screen-to-screen life of photos.

In 2018, I wrote

In ten years, even the art prints will be digital, locked down by blockchain, and displayed on screens of different sizes. I have two Aura frames at home, and they show all sort of photos of family, friends and of moments that matter. Look around, and even in the real world, the screens are getting digital. The advertisements are digital and are on displays. That’s the new photo workflow and ecosystem. Peer into the future, it wouldn’t be long we are co-existing with augmented reality and world where screens and images have an entirely new meaning.

I think that reality isn’t too far off — and while specifics remain unclear, it is not hard to be excited about the possibilities.

December 14, 2023 2024. San Francisco.