This article is only available in Dutch 🇳🇱
Three months ago, the Apple Vision Pro arrived at our office. At the time, Creative Director Matthijs Klaver shared his expectations in this article. Now that the hype has subsided somewhat, he looks back on the past time together with Stijn Kramer (iOS Lead from Zwolle). What did we want to achieve with the Vision Pro? What did we run into? And why did we choose the title above?
The perfect app?
Klaver: 'Our goal was to learn visionOS and develop the perfect Vision Pro app. The idea for this actually originated as a joke. We wanted to use the real-time camera feed to project information, with a face as the anchor point. So that when you have the glasses on and look at someone, you can see data about that person. During a visionOS hackathon, we discovered that we couldn't use the camera feed to place objects with image recognition through the camera. Pretty weird for a device with 12 cameras.'
Missing APIs
Kramer: 'We also discovered during that hackathon that certain APIs that are in RealityKit, which can be used on iPhone and iPad, cannot be fully invoked on the Vision Pro. We were hoping Apple would push these APIs through, so that was kind of a setback. We realized pretty quickly that you can do a lot less with the Vision Pro than you hope for as a developer. Apple's choices in this are quite understandable from a privacy perspective. Fortunately, at WWDC 2024, improvements were announced for visionOS 2 that will allow more interaction with the environment and the user. But if you want to get the most out of hardware, this will have to change even more in the future.'
'We realized pretty quickly that you can do a lot less with the Vision Pro than you hope for as a developer. Apple's choices in this are quite understandable from a privacy perspective. Fortunately, improvements to visionOS 2 were announced at WWDC 2024.'
Development and testing process
Kramer: 'During the hackathons, we investigated what the possibilities and limitations were in terms of development. Here we quickly ran into some limitations. A big part of development requires us to be able to debug our apps. This gives us more insight into what goes wrong and why certain features do or don't work. However, this is quite cumbersome with the Vision Pro: you can only debug if you are on the same network and registered in the Apple Vision Pro with the same developer account. Debugging over the network is slow and preferably you use a cable. That exists, but is very expensive and only available to developers in America. But our 3D app (exclusively for internal use) finally got there!'
Thinking in 3D
Klaver: 'In the end, we decided to design a spatial app, a 3D experience. That's not a requirement from visionOS, but we wanted to challenge ourselves. We experimented with spatial design during design hackathons in Figma. That requires a totally different mindset compared to regular, flat user interfaces. We got a lot out of Apples design guidelines and tutorials. Also, the design resources in Figma are complete, so you click and drag together a Vision Pro app fairly easily; the building blocks to create an interface are mature. Perhaps the biggest challenge for designers is dúrving to think in 3D.'
Wear comfort
Klaver: 'It was an interesting journey, but in the short term we are not going to do much with the Vision Pro. Apple often launches new products in such a way that the user experience is completely different from what you were used to. Once you then experience that, you think, "Yes, this is what we wanted! You can't imagine beforehand that it could work like this, and yet Apple shows that it can. Think of the scroll wheel on the iPod or the brilliant user interface of the first iPhone. With the Vision Pro, they have partially done that: succeeded on the UI, hand-tracking and eye-tracking, but not on the physical aspect. Most colleagues and myself agree that the glasses are uncomfortable, too heavy and not suitable to wear all day and increase your productivity - a use case with which they explicitly market the device.'
'Apple often launches new products in such a way that the user experience is completely different from what you were used to. With the Vision Pro, they partially did that: they succeeded on the UI, hand tracking and eye tracking, but not on the physical aspect.'
Usability
Kramer: 'In addition to the comfort aspect, the fact that you physically cut yourself off from the room also weighs in. Everyone who tried the Vision Pro with us still seemed to feel part of the room. However, this is a one-way street because others in the room cannot see what you are doing. One possible solution is to share your screen via Apple TV. This does make Apple's goal of integrating the glasses as an overlay into your daily activities difficult. The 'eyes' that shine through them are not enough to connect with the world around you.'
AI vs. hardware
Klaver: 'I actually think it's a shame that Apple opted for bulky VR glasses instead of simple, lightweight AR glasses à la Google Glass. Glasses that I can control with my voice or hand and that project simple information across my world for additional context in everyday life. One that, thanks to seamless integration of AI, answers questions like "how many calories are in a matcha latte?", "how fast can you go on a fatbike?" and "has Elon tweeted any crazy things?" in a matter of seconds as well as allowing you to chat with your personal assistant. Apple has apparently been working on the Vision Pro for fifteen years and I think the use case is already outdated. Back then, we didn't dare dream about what large language models like ChatGPT could do today. Not even two years ago!'
Kramer: 'Right now there are two major limiting factors. The first is hardware: to get the most out of the glasses, they need to be smaller and more portable. The second factor is the moral privacy issue: do you want the glasses to be able to see everything you do and share it with third-party apps - or can it be done differently? The software limits lie mainly with Apple. What do they want to open up? And how do they communicate this to users? Nowadays, developments in software are going so fast that the hardware can hardly keep up.'
Final Verdict
Klaver: 'The Vision Pro has exceeded expectations and just wasn't it for us. Paradoxical, I know, but nuance is important. So has our experiment failed? Absolutely not! It was a valuable project with a large multidisciplinary aspect, in which we jumped into the deep end and learned from technology, design and concept.'
Kramer: 'We think it's hugely important to keep experimenting with new technologies and stay abreast of relevant developments. That's why we continue to experiment with the Vision Pro, and every Move designer and developer is free to work with it.'
