Working Together to Keep Kids Safe Online

The National Center for Missing and Exploited Children (NCMEC) recently released its 2024 CyberTipline Report. NCMEC is a non-profit organization that works with law enforcement, corporations, families, victims, and the public to deter and combat child sexual exploitation, among other things. It’s also where companies based in the US report incidents of online child sexual abuse and exploitation that they become aware of. 

Last year, NCMEC received 20.5 million reports of suspected child sexual exploitation from companies and the public. Notably, reports of online enticement continued to rise, with an increase in boys being targeted through financial sextortion. The scale of online child sexual exploitation is shocking and underlines the importance of our continued efforts to find, remove, and report this behavior to law enforcement.

Rec Room joined NCMEC in 2023, and since then, we’ve been relentlessly focused on combating child exploitation. We’ve improved our team operations, developed new tech, and forged partnerships with other organizations to help tackle this global problem. 

When we identify potential child safety concerns in Rec Room, our dedicated Trust & Safety team conducts a thorough investigation. Subject to their investigation, we report cybertips to NCMEC and work with local and international law enforcement as they investigate. By strengthening our team operations and technology, we’ve become more effective at identifying bad actors, leading to a 4x increase in reports in 2024.

We develop and use technology to help us in this fight. That includes tried-and-tested tech, like photo-matching to identify known child exploitation images, and cutting-edge tools that we’re building that triage signals and language flags to help us identify accounts that may be engaging in inappropriate interactions with our players. 

Earlier this year, Rec Room joined the Tech Coalition, a global organization of technology companies committed to addressing online child sexual exploitation and abuse. This coalition of safety experts in organizations worldwide is working together to share valuable information and resources to help each other target online harms against children, breaking the typical barriers between companies to promote a safer online world. 

We want Rec Room to be a place where everyone can safely create and socialize. We take pride in the work our child safety team is doing and the tools we have built to assist with this mission. If you come across something concerning, you can report via our in-game reporting tools or send us a Zendesk ticket. We automatically prioritize reports related to child safety harms, quickly investigating and taking action if needed. 

We are grateful to our partners in law enforcement and the child safety experts we work with for their continued collaboration in promoting online safety. If you are a parent and would like to learn more about the safety features that Rec Room has to offer, please see our Guide to Rec Room for Parents & Families. 

Keeping Inventions Safe with AIRS

We have millions of players making millions of rooms, inventions, and avatar items every week. Your creations make Rec Room a hub of creativity and fun. Our job is to make sure all of you get to enjoy all of those creations without the risk of coming across content or behavior that could negatively impact your experience. Online, that can sometimes be a tough job. So, we’re always looking for ways to make your time in Rec Room more positive. 

We have made good progress on our player safety systems - our voice moderation system is 99.999999999% accurate, and we have increasingly sophisticated child safety systems in place. Over the last few months, we’ve been turning more of our attention to all of the content made by our players. We’ve provided better tools for creators to moderate their rooms, improved our avatar item moderation and appeal system, and are taking action on more rooms and repeat offenders than ever before. 

Now we’ve turned our attention to inventions. Each day, tens of thousands of inventions are created. Over 96% of those inventions are good inventions, but even a small fraction of bad inventions means potentially thousands of opportunities to find content that shouldn’t be in Rec Room. Finding these manually is like searching for a needle in a haystack. 

So, we built something to make that easier. Today, we’re sharing our new moderation system: Automated Invention Review System (AIRS). AIRS is specifically designed to ensure inventions meet our Creator Code of Conduct and help us keep pace with all of your creative output.  

How AIRS Works

AIRS is a complex system that involves scanning inventions and taking automated action when needed. We’ve been testing AIRS for the last few weeks, and we’ve found that it’s very good at catching sexual and discriminatory content. 

When an invention is published or reported, we immediately start the scanning process. We evaluate not only the name and description of the invention, but also the images the invention contains, such as printed photos and canvas art. This is all tuned to content that breaks our Creator Code of Conduct. 

What Happens When a Violation is Found

When AIRS identifies that an invention contains inappropriate content, it triggers a series of automatic enforcement actions designed to prevent its spread:

1. Invention Ban
   The violating invention is immediately banned. This also places a two-week restriction on the creator’s ability to create new inventions.

2. Lineage Scan
   We scan all parent and child inventions connected to the banned invention. We also scan other inventions created by the same player. This ensures that related content doesn’t slip through the cracks and means we prioritize reviewing content from creators who are known to have broken the Creator Code of Conduct.

3. Room Content Cleanup
   Any room saves that include the banned invention are automatically removed. If the invention is part of the room’s latest or current save, the room is closed to prevent exposure.

What This Means for You

AIRS will start taking automatic action against violating inventions and rooms that contain them at the end of next week. Why are we telling you this now? Well, if you have an invention in your room that violates our Creator Code of Conduct, please remove it before then to avoid losing your content. 

We will be reviewing how the system works and listening to your feedback. We’ll come back with any updates or changes we need to make. 

So, let us know what you think. And keep reporting when you find inappropriate content - our systems are getting better and better every day. Together, we can make Rec Room an even safer place to create. [|=)]