Roblox must make changes after failing to block adults creeping on kids - Ars Technica

Overview

Roblox must make changes after failing to block adults creeping on kids

Roblox is first platform to submit to independent audits under the Online Safety Act.

Details

Despite making changes last fall to help block adults from messaging unknown children, Roblox is still failing to stop adults from creeping on kids in other ways, Australia’s e Safety commissioner, Julie Inman Grant, said on Thursday.

Suspecting that Roblox may not be doing enough to stop child exploitation, e Safety conducted testing earlier this year that flagged alarming gaps in Roblox safeguards. The agency found that adult strangers could still send connection requests to kids, whose “profiles and biographies, including account names, number and names of connections, avatar images, and non-sensitive biographical information such as their interests were visible to anyone on the Roblox platform, with no option to restrict the visibility of this information.”

Such requests did not trigger parental alerts, e Safety found, and bad actors could also search kids’ visible-to-anyone contact lists for more targets.

Perhaps even more concerning, adults could also search for kids and comment on their posts “without parental or carer consent” in “forums outside game environments,” e Safety said.

Australia’s Online Safety Act requires that platforms prevent unwanted contact between adults and minors. It also mandates setting children’s accounts to private by default. Although Roblox “believes it is meeting its obligations,” the online gaming platform—which 1.7 million Australian kids use—has entered a court-enforceable agreement to make changes in the next three months.

Upcoming measures will prevent adults from contacting kids without parental consent across the platform, Roblox has promised. Providing better protections for kids starts with “strengthening the account settings of children’s accounts so they are private by default,” e Safety said, and ends with Roblox making it easy to report unwanted contact and notifying users of outcomes after reports are made. Those are critical protections for Australia’s second most popular online game for kids, Inman Grant said.

“We want to ensure that any platform with a high preponderance of kids to be a safe place where they can enjoy the benefits of online gaming, creativity, and connection, including building their own worlds and experiences online,” Inman Grant said. “But those benefits can only be realized when effective safeguards are in place to protect children from serious harms such as grooming, child sexual exploitation, sextortion, or recruitment into violent extremism.”

Additionally, Inman Grant said that the regulator’s testing suggested that Roblox can no longer be trusted to independently verify that its child safety tools are compliant with Australian laws. Moving forward, Roblox cannot “mark their own homework,” Inman Grant said, adding that the platform has agreed to third-party auditing.

Roblox is likely hoping that allowing the audits will demonstrate that it is going further than other companies to protect kids. In January, the company announced that it would become the first large online gaming platform to require facial age checks. Such biometric age checks have proved controversial, unpopular with users, and alarming to privacy advocates, who see even the latest age verification tools as flawed. And while Roblox says it usually automatically deletes selfies submitted for age verification, it may retain that data for up to 30 days, leaving it open to potential breaches.

“This undertaking follows constructive dialogue with e Safety and comes after the recent rollout of industry-leading safety features on Roblox—including the introduction of new Roblox Kids and Select accounts for under-16s,” Roblox’s spokesperson said. “We have delivered on the commitments we made to e Safety over the last 12 months and will continue to work towards our shared goal of keeping Australian children safe online.”

Inman Grant emphasized that this is the first time Australia has required third-party auditing since the Online Safety Act became enforceable.

The agreement came after e Safety warned Roblox last October that officials remained “highly concerned by ongoing reports regarding the exploitation of children on the Roblox service, and exposure to harmful material.”

For Roblox, the deal goes beyond helping e Safety monitor how well its safeguards are working. The third-party auditing will also give the agency a window into Roblox’s age estimation tools, e Safety said. If it seems like Roblox is mislabeling kids as adults, giving them access to adult content, or exposing them to adult predators, the company could face steep fines under the Online Safety Act. Last October, e Safety warned that it was prepared to “use the full range of its enforcement powers, as appropriate” to stop grooming and exploitation on Roblox, including “seeking penalties of up to $49.5 million.”

Roblox seems confident in its age-estimation system, though. In a blog post earlier this month, Roblox shared that its Facial Age Estimation system—which was built “completely in-house”—uses artificial intelligence to keep “age checks current.”

The system so far has “processed 338 million age-check records to help Roblox users unlock age-appropriate chat and access age-appropriate games on the platform,” Roblox reported. It relies on “a high-throughput” machine learning (ML) “layer that evaluates whether an account’s ongoing behavior still matches its estimated age over time.”

Although age estimation with ML is “difficult” because “age boundaries are not behavioral cliffs,” Roblox says it is using the “right signals” to estimate age. But the platform hedged a bit when discussing how it sorts users into age bands, noting that “a single accuracy number is not very meaningful” because quality also depends on “ranking quality, calibration, stability over time, and behavior near important age-band boundaries.”

Roblox does share some statistics, though, reporting that its fused model produces “high-confidence estimates” that stay “consistent from day to day.”

“More than 99 percent remain in the same broad age band on consecutive days,” Roblox said.

Further, tests that compared the model “against independent ground truth produced by trained human reviewers through our Trust and Safety workflows” showed that “the model’s age-band estimate agrees with the human consensus label for adults more than 80 percent of the time,” Roblox said.

Roblox suggested that “owning this system in-house lets us leverage Roblox-specific signals safely, operate with extreme rigor, and continuously iterate,” including by allowing Roblox to easily drop ineffective submodels or optimize others. But e Safety is clearly taking the cautious approach, planning to continually audit the age-estimation tool’s effectiveness so that it isn’t blindsided if Roblox’s self-evaluations rate the system more highly than the regulator in charge of enforcing the Online Safety Act might.

Later this year, e Safety will release a transparency report detailing how successful online gaming services like Roblox have been at mitigating harms to kids. Generating that report required all platforms to explain “the steps they are taking to protect children from harms, including child sexual exploitation and abuse, violent material and activity, and cyberbullying online,” e Safety said.

“Ultimately, our goal is to drive companies to make their services safer for children and young people. Where safety by design has not been deployed comprehensively at the front end, we won’t hesitate to ensure that these risks have been responsibly remediated,” Inman Grant said Thursday. “Holding Roblox to the binding safety improvements through this enforceable undertaking is a great example of how these systemic powers can deliver real and meaningful results.”

1.          FCC abolishes gigabit speed goal, suggesting it is unfair to slower technologies
   

2.          m RNA cancer vaccine succeeded in Phase 3 melanoma trial, Moderna and Merck say
   

3.          Google Pixel 11 series review: Is the magic fading?
   

4.          The floodgates are open after another Chinese company lands a reusable rocket
   

5.          Satellite operators are in panic mode due to a worsening launch crisis
   

Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don’t need to know everything, only what’s important.

Key Takeaways

• Roblox must make changes after failing to block adults creeping on kids

• Roblox is first platform to submit to independent audits under the Online Safety Act

• Despite making changes last fall to help block adults from messaging unknown children, Roblox is still failing to stop adults from creeping on kids in other ways, Australia’s e Safety commissioner, Julie Inman Grant, said on Thursday

• Suspecting that Roblox may not be doing enough to stop child exploitation, e Safety conducted testing earlier this year that flagged alarming gaps in Roblox safeguards

• Such requests did not trigger parental alerts, e Safety found, and bad actors could also search kids’ visible-to-anyone contact lists for more targets