Families can sue app developers if they break their anti-bullying promise

An appeals court has reinstated a lawsuit against anonymous messaging service Yolo, which it says broke its promise to expose bullies on the app. In a ruling announced Thursday, the Ninth Circuit Court of Appeals said Section 230 of the Communications Decency Act should not block a lawsuit that Yolo misrepresented its terms of service, overturning a lower court decision. But it found that the app cannot be held liable for alleged design flaws that enabled harassment and left another part of the earlier ruling in place.

Yolo was an app integrated into Snapchat that allowed users to send anonymous messages. But in 2021, it faced a lawsuit after a teenage user died by suicide. The boy, Carson Bride, had received harassing and sexually explicit messages from anonymized users he believed he likely knew. Bride and his family tried to contact Yolo for help, but Yolo reportedly never responded, and in some cases, emails to the company simply bounced back. Snap banned Yolo and another app targeted in the lawsuit, and a year later, it banned the integration of all anonymous messages.

Bride’s family and a group of other aggrieved parents argued that Yolo had broken a legally binding promise to its users. They pointed to a notice in which Yolo claimed that people would be banned for inappropriate use and anonymized if they sent “harassing messages” to others. But as the ruling summarizes, the plaintiffs argued that “with a staff of no more than ten people, Yolo could not possibly monitor the traffic of ten million active users daily to keep its promise, and in fact it never did.” Moreover, they claimed Yolo should have known that its anonymous design facilitated harassment, making it flawed and dangerous.

A lower court dismissed both suits, saying that Yolo could not be held responsible for its users’ postings under Section 230. The appeals court was more sympathetic, accepting the argument that families instead held Yolo responsible for making promises to users that it could not keep. “Yolo repeatedly informed users that it would unmask and ban users who violated the Terms of Service. However, this never occurred and may never have been intended to do so,” writes Judge Eugene Siler Jr. “While online content is implicated in these facts and content moderation is a possible solution for Yolo to fulfill its promise, the underlying duty … is the promise itself.”

The Yolo lawsuit built on an earlier 9th Circuit ruling that allowed another Snap-related lawsuit to bypass Section 230 protections. In 2021, the court found that Snap could be sued for a “speed filter” that could implicitly encourage users to drive recklessly, even if the users were responsible for making posts using that filter. (The full case is ongoing.) In addition to their claim of misrepresentation, the plaintiffs argued that Yolo’s anonymous messaging feature was similarly risky, an argument the 9th Circuit did not accept—“we refuse to support a theory that would cast anonymity as an inherently unreasonable risk per se,” Siler wrote.

This latest ruling is part of a longer back-and-forth over the scope of Section 230. Several cases have argued that apps are illegally flawed if they cause harassment or other harm, even if that harm was caused by users. Despite periodic successes, this is still far from an established doctrine, and the Supreme Court declined to consider it applicable to the Herrick vs Grindr case in 2019. The Supreme Court also refused to cut Section 230 in a case about whether YouTube and Twitter supported illegal terrorism. After that Ninth Circuit ruling, Yolo can still defend itself by saying it reasonably tried to enforce its user agreement, and the case isn’t over yet.

But allowing users to sue a company for violating its content policies could theoretically open up the possibility of bringing suit against nearly any service that doesn’t practice (often impossibly) perfect moderation. The Ninth Circuit insists that’s not the case. “Today’s decision neither expands liability for internet companies nor makes all violations of their own terms of service actionable claims,” ​​Siler writes. “In our caution to ensure that (Section) 230 has its full effect, we must resist the resulting urge to extend immunity beyond the parameters set by Congress and thereby create permissive immunity for technology companies.”