The State of Voice Moderation
We've received a lot of questions about voice moderation, so we got the 411 from our trust and safety team on how voice moderation actually works in Rec Room.
We use voice moderation systems in all public rooms in real time. These are cutting-edge machine learning systems that detect hate, harassment, and other types of nasty speech based on Rec Room’s community standards.
What’s The Deal?
We’ve spent the last year running analyses and reworking our systems. Here’s the most important takeaway: the data confirms what we knew all along - the vast majority of players are fantastic community members. Most have never broken the Code of Conduct. Never. Not once. Angels.
Of those that have broken the rules, most players only slip up once. Say it gets a bit tense, maybe you miss *that* paintball shot at the last second and a few “$%@!s” slip out - our voice moderation system is designed for exactly this. It will catch the one-off and give you a friendly warning - because we get it - we all get caught up in the moment from time to time.
What If It’s Not Just A Moment Then?
Players only lose mic privileges if that moment escalates. Temporarily revoking mic privileges is a very direct form of feedback - hey, that wasn't cool! - but it’s also a chance to reflect; "how can I be excellent with other Rec Roomers when I get my mic back?"
If a player keeps using speech to be hateful or harass, that mic restriction will increase, and may eventually turn into a ban to give them a chance to cool off. We don’t like banning players. We do it when we have no other way to protect the experience for the rest of our community.
As we said, our players are top notch. So, bans for voice chat toxicity are actually really rare- like 1 in 135,000 people rare. You’re literally more likely to roll all sixes on six dice! And those very rare bans are now also short - we don’t need to (and we aren’t) pressing the 40-year ban button or even the one week ban button for voice toxicity. Long or permanent bans are used only for serious code of conduct violations.
But What If I Didn’t Actually Mess Up?!
Hang on though… you’ve all seen the social posts that say “I didn’t say anything and then I got banned” or one of our favorites “It was just a mic echo”. But it’s not true! If players are getting banned or muted, we are pretty dang sure they said a few things they shouldn’t have. Is it 100% perfect? No. But it almost is…
This is where our internal data wizards - come in: every detection system has a False Discovery Rate (FDR) i.e. a chance of being wrong. The FDR limits the confidence we’re making the right decision on any issue. If the FDR is ever above 5%, you can never reach 99% confidence. Unless… you wait. If you see not 1, but 2 or 3, or even 25 potential-rule-breaks in any given period - then you can have quite a bit more confidence that someone did cross the line.
So even if the system has a 5% FDR, if we see a player caught 10x, then we’re not 95% confident - we’re more like 99.999999999% confident. So yeah…almost 100% perfect! We don’t want to interrupt games unnecessarily, so to reduce false positives, we don’t ban or mute based on one thing a player says in one fragment of speech - instead we wait until we hit that 99.999999999% confidence rating based on multiple hits, which can be over a few sessions or in the moment.
This really is the best of both worlds - we catch the worst toxicity, we forgive the one-offs, and we stop those gnarly false positives. And you know what? It works! This chart shows what’s been going on over the last year - instances of toxic voice chat have fallen by around 70%!
What’s Next??
So, job done then? Not quite…we’re getting to a good place on improving the systems’ accuracy, but accuracy is only half the story. The other half is about what we catch - and more importantly, what we don’t. Right now, if the system is catching you, you slipped up more than a few times. But we know - and you know - that there are times we probably should have caught something and we didn’t. Players being unfairly banned is lose-lose, so we worked on accuracy - but now we’re turning our attention to broader coverage. More on that in future updates, but for now, thank you to all of you who continue to make Rec Room the great community we want it to be. And keep doing your Rec Room Best!