AR/VR designers at Facebook divide our efforts into three phases: ideation, vision work and prototyping.
If you’re a designer, ideation is probably familiar. It’s a quick and iterative way to generate lots of ideas to address a problem and learn rapidly. We use collaborative brainstorming, storyboarding to tell a narrative and — unique to AR/VR — bodystorming. For storyboarding, our team is fond of Procreate for creating digital sketches in 2D and Quill for sketching in 3D. For bodystorming, we use real-world props and activities to act out interactions and narratives. This is especially effective in AR/VR, because you get a spatial feel for objects and scale while iterating much faster than in digital prototyping.
For vision work, we generally use 3D modeling and animation apps, such as Cinema 4D, Blender or Maya, to render videos on top of recorded footage.
The third phase, prototyping, is the highest fidelity of the three phases and is usually reserved for smaller, more high-touch interactions or project details. Prototypes are also usually the best artifacts to bring into user research, since they allow participants to test our work and give tangible, direct feedback. AR/VR prototyping contains a couple of key differences compared to other disciplines. First, interactions take longer to build, as best practices have yet to be defined completely, and second, there are significantly more variables to consider when designing in 3D than 2D.
In this phase, our team usually uses a 3D modeling app — the same ones mentioned above — to create low poly assets for our real-time engines.We generally do interaction prototyping in the same tool we use for the end product so we can test, learn and iterate fast. This usually means using Spark AR Studio for mobile AR, adding interactivity through either visual programming or scripting with code and using Unity or Unreal Engine for HMD-based AR/VR for products like the Oculus Rift. Whether you select Unity or Unreal as your tool of choice is a hotly debated topic, so I’ll leave it up to you to decide.
This may seem like a broad skill set, but luckily I didn’t have to become an expert on all phases. Each of my team members has a strong domain expertise that helps raise the rest of the team up. I have a team member who is amazing at motion graphics and visualizing ideas, a coworker and friend who knows shaders and real-time engines inside and out, a teammate who is a master of design processes and practices, and, of course, there’s me. I’m more a generalist and know these skills more broadly but not as deeply in any one category. A multidisciplinary team like ours shows how broad and open the skill sets are for an AR/VR designer. The real magic happens when we apply our different areas of expertise to the challenge and collaborate to find a solution.
Now that I’ve shared one approach to designing for AR/VR, let’s dig into some unique learning methods.