SUMMARY
Facebook’s Sound Design Director shares thoughts about his role, his team’s work, and how to design sound with empathy.
How do you define a person who has toured the world for a decade as a synth musician; designed some of the first sounds to ship in a cell phone; contributed to Oscar-nominated, Emmy- and BAFTA-winning projects; produced the music for the legendary “Guitar Hero” videogame; and supervises every audio experience for Facebook? “Renaissance man” might come to mind, but that doesn’t quite do Will Littlejohn justice.
Littlejohn’s sound design team crafts every tone the world hears across Facebook apps, services, and products—including Messenger, Instagram, Whatsapp, Workplace, Oculus and Portal. His team includes some of the most accomplished sound designers, audio producers, composers, and design operations specialists in the business. In 2019, Littlejohn’s team celebrated a Primetime Emmy win for “Wolves in the Walls,” a groundbreaking interactive VR film based on a story by Neil Gaiman and produced by Oculus. Littlejohn’s team created the entire audio experience, including thousands of individual sounds, the original music score, and the behavior of every sonic action in the story.
Back in 2017, Littlejohn shared some top tips for enhancing mobile interactions with sound and unveiled Facebook Sound Kit, a collection of interaction sounds for prototypes. In this recent conversation, he discusses what excites and challenges him, how to produce empathy through audio, and how he keeps his team inspired.
Q: Sound Design . . . at Facebook? Your job title must mystify many people.
A: It certainly does! Most people haven’t considered the extensive use of sound in our products, but if you think about it, there is some type of sound component in almost everything we make these days. As we’ve grown as a company, sound has become more important across all of our offerings. The mission of our team is to craft the best experiences possible, and enable more resonant connections between people by activating the amazing sense of sound.
We collaborate closely with multidisciplinary design teams and engineers to create holistic and meaningful experiences across every aspect of our products—from the smallest audio cues in Messenger to entire immersive VR experiences for Rift and Quest. In most cases, we’re working to create a seamless audio experience that you almost don’t realize is there, and yet it would be a diminished experience if it weren’t. This is a terrible truth sound designers eventually realize at some point in their careers: The closer you get to perfecting the sound experience, the less people will notice!
The trailer for “Wolves in the Walls,” Facebook’s Primetime Emmy-winning interactive virtual reality film.
Q: Your team sounds like rock stars—both literally and figuratively. How do you choreograph the amount of company-wide collaboration that your work requires?
A: The spectrum of domain expertise across this sound design team is truly staggering. Our practice is comprised of four main groups: our interaction design group focuses on crafting the sounds that happen as people intersect with technology; our music group brings new experiences to people through in-house composition and collaborations with external artists; our production group focuses on sound design for video, film, animation, as well as the development of our voice assistants; and our immersive group focuses on experiences across the augmented reality/virtual reality ecosystem.
Setting up for a recording session at Facebook’s Fremont, CA studio.
Q: How does your team keep people at the center of everything you create?
A: We are all aligned with our company mission in that our work is in service of people. A focus on empathy is ingrained in the culture of our team; we’re always looking at our work through the lens of the impact on individuals in the real world. It’s easy in lots of forms of creation to get too much into a production mindset, where you focus on getting stuff out the door and you lose touch with why you’re doing it. “Why are we making this sound?” is the most important question when we’re designing an audio component for a product. The answer should always be that we’re making it to help people have a better, more delightful experience, while making the product more useful in the process.
It’s also very inspiring to be sonically present as much as possible. I really encourage the folks who work with me to get out into the world to listen to and experience what’s going on around them. It’s amazing what you hear and observe when you listen with intention. This is a constantly evolving, interesting world, and we’re all students together.
Sound is fundamental to our human experience.
Q: You’ve worked as a touring musician for many years, and now you create technology that is part of the daily lives of billions of people around the globe. How do all of these experiences impact your approach to user-experience design?
A: One thing I carry with me is a realization that I came to while playing music for people: Sound, in its infinite forms, can transform the experiences people have in a profound way. Those sounds can be the focus of an experience, such as a musical performance; the soundscape of a visceral snapshot in time, such as a beautiful beach at sunrise; or the smallest sound associated with an everyday movement or action, like a button or a door opening. All sounds have the potential to influence how we feel about a moment in time, and I keep this close as I approach our work.
Also, humans don’t actively think about sound—that’s not how we are wired. We’re optimized to pay attention to what we are seeing: More than half of our brain’s cortex is devoted to visual sensory input and processing. The sense of sound is optimized for passive monitoring—listening in the background without being consciously aware of what we are hearing. Generally speaking, people don’t pay attention to the sounds coming into our ears until our brains decide they are sufficiently important or novel to focus on.
Familiar sounds quickly become associated with events and behaviors.
Humans also have a fascinating trait of subconsciously using sounds to cue events and adapting our behavior to take advantage of this sensory input. Think about microwave ovens: you don’t stare at the device waiting for your food to get hot, you listen for the beep! This behavior is not a conscious choice—you do it without a thought. As soon as you associated the sound with the cooking cycle ending, you promptly offloaded this monitoring to your subconscious. This is amazing to me, and just one of the interesting ways sound influences our experiences.
One of the opportunities we have as sound designers is to help our collaborators working in visual mediums understand the importance of sound in designing a holistic experience, and make sure we collectively approach our work with this in mind. Really understanding the ways we, as human beings, relate to the sonic stimulus we are constantly processing as we move through our day helps us all make better decisions.
Q: Facebook is committed to accessibility. How does your team work to ensure people with different abilities can have as rich an experience with our platforms, products, and devices as possible?
A: As audio enthusiasts, we are passionate about ways sound can help people feel more connected. For example, we approach our accessibility work as an opportunity to use the unique attributes of sound to bridge the gap of experience for people who rely on their ears as their primary sense.
An example is a concept we call embedded information. The nature of a sound can contain strong clues as to its meaning: a quick sweeping tone hints to a rapid transition on a screen, or a sound that winds down strongly hints at a closing or stopping action. In working on accessibility projects in any medium, we work hard to embed as much information in the designs as possible.
We are also deeply involved in the development of our in-product voices, which is an extremely exciting area of sound design. When coupled with voice recognition technology, speech can be the primary interface in a product, and can be the basis of amazing experiences for everyone. As an example, our Portal devices speak with a voice our team created in collaboration with our speech technology group. It’s possible to make video calls, play music, ask questions and navigate around to different apps without relying on any visual navigation. Informative tones coupled with our in-product voices guide your navigation and let you know what’s happening in an intuitive way without using your eyes. This is accessibility by design.
Q: Let’s be honest—some sounds are intrusive and unpleasant. What is crucial to not do when designing sound?
A: We carefully consider how audio is going to impact people. First of all, I encourage our designers to consider whether a sound should be introduced at all. Will it make the experience more meaningful, delightful, or useful to people? If not, then don’t use sound. Sound has the potential to be intrusive and unpleasant if not thoughtfully designed and implemented. Consider repetitive tolerance—say, if you had to listen to a sound a thousand times, how would that make you feel? If we can get that sound to a place where it’s not going to negatively impact the people using our products, but still provides value in the experience, that’s the goal. Every time we create sounds, we keep people at the center of our work.
Q: What complex challenges did the design of Portal—Facebook’s smart video calling device—present for your team?
A: Some of the biggest challenges were rooted in designing the flow of Portal’s audio experiences, some of which are very complex. Take a voice interaction, such as someone saying, “Hey Portal, call my mom.” The device then calls your mom. It’s going to play a ring tone on her end and a different ring back tone on yours. When she answers, you’re going to have a real-time conversation. If you initiate Story Time—a feature which can include music and sound effects—and you receive another call, how do you handle all of those audio flows at the same time in a way it’s intuitive to everyone participating? Every discipline of sound design was involved in creating the experiences in Portal, and we were challenged by several audio-flow scenarios which required very innovative thinking by our designers.
A distinct ringtone, such as this one from Portal, is one of many considerations in a well-crafted sound experience.
Q: How is sound design evolving with emerging technologies such as AR and VR?
A: We're actually helping define these new technologies, dreaming up the experiences alongside our engineering partners and then building the technology to get there. Our team is highly collaborative with software and hardware engineers. We love to put the right experts in a room and then see what happens.
With the launch of the Oculus Quest 2 in October 2020, our team celebrated several major contributions to the overall audio experience in the platform, including the design and behavioral attributes of all interaction sounds across the ecosystem. We also designed the sonic environments in all of the Oculus offerings within the platform, and contributed to many of the content experiences available in the Oculus Store.
Designing sound for VR is both similar to and different from real-world design, and the team brought all our previous experience in the medium—as well as traditional audio design—to this amazing product.
In terms of our evolving approach to design for both AR and VR, we are considering spatial audio in new and interesting ways, incorporating the position of sounds in space as a more integrated part of the designs themselves.
Q: What has surprised you most in your 25+ years in the sound design space?
A: The moment I realized a small thing that I may design could have a positive impact on many people. It was profound for me to discover that I designed small sonic moments in “Guitar Hero” that became important to countless enthusiasts. Seemingly minor decisions I made during some very late nights in a dark studio changed people’s lives in a positive way.
A composer works with modular synths at Facebook’s Fremont, CA recording studio.
Q: What can you tell us about the new Fremont facility for sound design?
A: I’m excited about upcoming opportunities to collaborate in novel ways in this new facility. It’s a purpose-built complex designed to support all forms of sound development, including music. The space is currently closed due to the pandemic, but we are looking forward to collaborations both with internal teams and creatives outside of the company. With state-of-the-art technology and acoustically controlled areas in the facility, we’ll be able to push the boundaries of audio and use sound to bring people together.
—
So what does Will Littlejohn—a person with a rare combination of artistry, technical prowess, and leadership skills—find most rewarding in his work? “We’re connecting people in new and more meaningful ways,” he says. “I connect every week with my uncle, who is 92 years old, through Portal. We’re connecting generations: grandparents and their grandchildren.”
RELATED STORIES
Read more stories about culture
Design at Meta is for everyone who touches user experience and design.
Whether you’re a product designer, writer, creative strategist, researcher, project manager, team leader or all-around systems-thinker, there’s something here for you.
