728 x 90

Discord admits AI moderation bug wrongly banned users for harmless images | TechCrunch

Discord admits AI moderation bug wrongly banned users for harmless images | TechCrunch

Discord has acknowledged that a bug in its AI moderation system mistakenly banned more than 8,000 users in the past two months, after harmless images (including spreadsheets, chess boards, game textures, as well as white and gray transparent backgrounds) were incorrectly flagged as harmful content. The company confirmed that the issue had been affecting accounts

Discord has acknowledged that a bug in its AI moderation system mistakenly banned more than 8,000 users in the past two months, after harmless images (including spreadsheets, chess boards, game textures, as well as white and gray transparent backgrounds) were incorrectly flagged as harmful content.

The company confirmed that the issue had been affecting accounts since May, with an additional 200 users banned over the weekend before its team identified and fixed the issue. All affected accounts are currently being restored.

The incident highlights one of the growing challenges surrounding AI-assisted moderation, as many platforms increasingly rely on automated systems to identify illegal or abusive material at scale.

In a detailed thread about X, Discord explained that its automated security system works by comparing uploaded content to databases of known harmful material. While the technology is designed to detect illegal content, the company acknowledged that it can sometimes generate false positives. A human moderator reviews the content, but an error caused the system to immediately ban the affected accounts.

“We are working to improve safeguards so this does not happen again,” the company wrote.

On X and Reddit, users claimed that they had been permanently banned simply for uploading images containing square grid patterns. Several users speculated that Discord’s AI moderation tools have become increasingly sensitive to grid-like patterns because they have previously been used in attempts to hide or disguise NSFW and child exploitation content from automated detection systems.

Affected users have expressed frustration on social media, with some arguing that permanent account bans based solely on automated detection can have serious consequences, particularly for users who rely on Discord for work, gaming communities, or long-distance social connections.

“Losing a Discord account over something as unfair as this can be extremely devastating and severely impact users, and every day millions of users are affected by fake AI bans. This needs to stop,” one X user wrote.

Discord is not the only one facing moderation issues due to automated systems. Last year, Instagram and Facebook Groups users reported widespread and unexplained account suspensions that many believed were caused by AI moderation systems. Although users pointed to automation as the likely culprit, Meta never publicly confirmed whether AI errors were responsible. Now the Meta Oversight Board is pushing for greater transparency.

Last year, Tumblr also faced complaints from users who said their accounts had been massively suspended without clear explanations.

When you buy through links in our articles, we may earn a small commission. This does not affect our editorial independence.



For more tech updates, stay tuned to our blog.

Posts Carousel

Latest Posts

Top Authors

Most Commented

Featured Videos