Once upon a time (summer of 2022), in a faraway land (the roomiverse), and long before Avatar Studio, we launched the Shirt Customizer. It was the first time players could customize clothing and the creativity the community has shown confirmed that when we give you the right tools, you make all kinds of cool stuff and a lot of it - we’ve seen millions and millions of shirts being created and sold.
So why bring this up now, two years after the feature launched? Well, we’ve got a new feature we’re rolling out for custom shirt publishing and are long overdue for an explainer on how custom shirt moderation has been working behind the scenes. We also want to walk through how we’re applying what we’ve learned from Custom Shirts to our vision for Avatar Studio - so you can create and wear avatar items to be your favorite version of you, while still being fun and welcoming for everyone!
We’ve seen so much creativity come out in custom shirts, and it really got us thinking about the potential for UGC avatar items. Just look at some of these really fun examples!
We also saw some folks who wanted to push the boundaries and be a little provocative (you know who you are!). A few people really do go too far though, and we saw some customization that just wasn’t appropriate. We’re pretty sure you all know it, but to be clear, if it breaks the Creator Code of Conduct, then we won’t allow it.
We never want the small subset of rule breakers impacting the play or creation experience for the majority of players, so we put some great tools in place that help us detect inappropriate shirts within seconds of publishing. It works by scanning shirts for things that would be a no-go like nudity, hate, or drug content. It took us a while to get these systems fine-tuned, heightening and lowering thresholds for different categories and types of no-go content so that rule breakers had less chance of slipping through the cracks. And that system is now doing a pretty good job of making sure all your excellent creations get published ASAP and those handful of bad ‘uns don’t.
If your shirt is detected as violating our CCoC, we take down the shirt so it doesn’t make it to the store. We operate on the premise of no harm, no foul - so creators get a warning with no ban because we want you to be able to continue creating great stuff. If our creation tools are repeatedly abused though, we may remove your shirt creation privileges - so consider this fair warning.
When we were doing that fine tuning and decreased the thresholds for some of the Creator Code of Conduct categories, it got us thinking. What if the AI gets it wrong? How do we match up the automated rules with reality? This can be harder than it sounds as so much of these rules are about context and taste. Technically, if the AI does get it wrong, the risk to the creator is small because we just take down the shirt, but that wasn’t good enough for us. So, we’ve built in a new appeal system for shirts that are flagged by our AI systems. If you think the AI got it wrong and the shirt is not in violation of the CCoC, you can appeal and we will review it and reinstate it if it was a false flag. These appeals will even be used to fine-tune our systems and keep making them more accurate, so please use (but don’t abuse) this new function!
What happens when the AI doesn’t work the other way and something slips through that shouldn’t? Like always, you can report shirts you see that breach the CCoC and we’ll review and take action on those too.
We’ve learned a lot of lessons enabling everyone to create the shirts they’ve dreamed of, and now those lessons are key to the development of our moderation systems for Avatar Studio. As you’ve seen, we’re taking it slow, building out and fine-tuning our automated scanning and appeal systems to match our CCoC, and planning out extra protections for avatar items that appear in high profile areas like our featured clothing. Keep an eye out for a future update that shares more detail on how we’re integrating trust and safety into the fundamentals of Avatar Studio AND creating tools with the potential for new clothing and accessory styles - Pumpkin Head and Sigma Headband anyone?