Changing behavior through education

One approach we’re taking is to prevent bullying by giving people the opportunity to play a role in keeping Instagram a safe place for themselves and others.

To equip people with the tools they need to do this, we developed a simple warning that nudges them during possible moments of conflict, and asks them to reconsider posting comments that might be mean or offensive. It’s powered by machine learning, which detects comments that are similar to others that have been reported in the past. It pairs simple, friendly language with visual cues to create an educational opportunity during a critical moment: right before sharing. By doing so, it gives those who are on the verge of posting a hurtful comment time to reflect and reconsider.

But because we’re intervening at times when people may be behaving emotionally or reactively, we need to consider what they may think or how they may feel at that moment: are they putting down a stranger, or standing up for their family? For this reason, the warning is designed to be subtle, rather than disruptive. We also use non-judgmental language that communicates transparency, education, and control over what happens next.

The experience is made up of a behavioral nudge, followed by a moment of pause. We ask people, “Are you sure you want to post this?” and give them 8 seconds to reflect on what they’ve written. If they choose, they can undo their comment to edit it or delete it completely. We also include a visual timer to convey a sense of urgency.

Designing tools to remediate conflict

While we’re taking steps to prevent bullying, we’re also implementing more tools to help people stand up for themselves when it does happen.

There are several options people can use to control what they see or who can interact with them, including muting, blocking, or switching to a private account. Most commonly, people block their bully because it stops them from viewing their profile or sending them messages. And because it prevents all interactions between their accounts, this is effective in stopping bullying between strangers. But when it comes to people who know each other and have friends or family in common, this can actually make the situation worse in real life. If the bully notices they’ve been blocked, they may retaliate by encouraging others to make fun of them, or creating a new account to continue harassing them.

We wanted to better understand the many ways bullying can manifest on- and offline. Through research, we heard that people experiencing bullying are concerned about three main things:

• Visibility: They want to be able to keep an eye on the bully, so they can monitor what they’re doing or saying about them. Shutting the bully out of their life isn’t always the best solution, especially if they’re a close friend or relative.

• Privacy: They want a private action that helps them stop the behavior without the bully finding out.

• Context: They want a tool that stops or hides bullying where it happens, like in comments or messages.

To address these needs, we designed a tool called “restrict.” It enables people to prevent unwanted interactions without having to block or unfollow people they know, empowering them to manage conflict on their own terms.

We know people might use this tool at times when they feel vulnerable or powerless, so we support them by explaining that this tool gives them control over the circumstances they’re in. We also know that people experiencing bullying don’t want to feel like victims, so we avoid implying that they need to protect themselves in order to have a safe experience on Instagram.

When a bully is restricted, they can’t see when the other person is online or when they’ve read their messages. We added this functionality because we heard in research that people don’t like to give their bullies the satisfaction of a read receipt, but want to be able to see what they’re saying, especially if they’re apologizing. The person experiencing bullying also stops getting notifications about their bully’s comments or messages, and any new comments the bully makes are hidden from others.

Restrict not only reduces people’s exposure to potentially negative or hurtful comments, but also gives them a powerful alternative to blocking someone when it isn’t right for the situation they’re in (like when the bully is a friend, acquaintance, or someone else they need to keep an eye on). We added restrict next to options to block, report, or mute in places where people told us bullying happens most, so it’s easy to access when they need it. And we explain that, unlike blocking, restricting someone is a private and undiscoverable action, so they know they can protect themselves without others knowing.