AI Content Moderation and the EU AI Act

AI-driven content moderation has become standard infrastructure for online platforms. Image, video, audio, and text classifiers automatically detect spam, hate speech, child sexual abuse material, terrorist content, copyright infringement, and many other categories. Large language models increasingly play a role too, both as text classifiers and as reasoning engines that adjudicate edge cases.

The EU AI Act applies to AI content moderation, but in a layered way that depends heavily on the platform context. This article walks through the classification analysis, the Article 50 transparency obligations, and the critical interaction with the Digital Services Act — which is usually the more demanding regime for content-moderation operations.

The Classification Question

Most AI content moderation systems are not high-risk under the EU AI Act. The reason is structural: high-risk classification under Annex III requires deployment in one of eight specific sensitive areas, none of which directly captures online content moderation.

Annex III, point 3 (education) does not apply. Point 4 (employment) does not apply. Point 5 (essential services) does not apply unless the content moderation is integrated into a determination about access to essential services. Point 8(b) (democratic processes) is closer — but it captures AI systems "intended to be used for influencing the outcome of an election" rather than general moderation.

Some specific moderation systems may be high-risk. AI used for emergency-call moderation falls under Annex III, point 5(d). AI moderation systems deployed by public authorities to determine access to public benefits or essential services may fall under point 5(a). But the typical commercial content-moderation deployment — for a social media platform, video-sharing service, marketplace, or forum — does not.

This does not mean AI content moderation is unregulated. The AI Act applies through other provisions, and the Digital Services Act applies in parallel and is typically the more demanding regime.

What the AI Act Requires for Content Moderation

Even where content moderation is not high-risk, three AI Act provisions apply directly:

Article 5 Prohibitions

AI moderation systems that implement prohibited practices cannot be deployed. The most relevant prohibitions:

• Article 5(1)(a) — manipulative or deceptive AI causing significant harm. A moderation system designed to materially distort users' behaviour or political views, where this distortion causes significant harm, could potentially be captured.
• Article 5(1)(b) — exploitation of vulnerabilities of specific groups. A moderation system specifically designed to manipulate vulnerable users could fall here.
• Article 5(1)(g) — biometric categorisation inferring sensitive attributes. If a moderation system uses facial analysis to categorise users by race, religion, etc., it is prohibited.

For most content moderation, these prohibitions are not triggered. They are more relevant for recommendation and personalisation than for moderation.

GPAI-Provided Models

Many modern content-moderation systems are built on general-purpose AI models. Where this is the case:

• The GPAI model provider has obligations under Chapter V — technical documentation, training-data summary, copyright compliance, downstream-provider information
• You as the moderation operator rely on the downstream-provider information for your own compliance

If you fine-tune a model specifically for moderation, you may take on GPAI provider obligations under Article 54 — though this is unlikely for narrow fine-tunes that do not change the model's general capabilities.

Article 50 Transparency

Article 50 may apply to user-facing aspects of moderation outputs:

• If automated removal decisions are communicated to users, the user-facing message should make clear that the determination was made by an AI system (Article 50(1) applies in spirit, though it primarily targets direct AI-user interaction).
• If content is added or modified by AI (for example, a moderation system that adds warning labels generated by AI), the deepfake provisions in Article 50(4) may apply to AI-generated labels or annotations.
• AI-generated text used in user-facing notifications (for example, AI-drafted notification messages explaining why content was removed) may trigger Article 50(4) requirements for AI-generated public-interest text, though the editorial-review carve-out generally applies if a human reviewer signs off.

In practice, the platform-level transparency obligations under the DSA are more comprehensive and more strictly enforced than the AI Act's Article 50 in the content-moderation context.

How the Digital Services Act Applies

The Digital Services Act (Regulation (EU) 2022/2065) is typically the more demanding regime for content moderation. Key requirements:

Article 14 — Notice-and-Action Mechanisms

Hosting providers must put in place "easily accessible and user-friendly mechanisms" for any third party to submit notices of allegedly illegal content. AI moderation systems are often the first line of detection for content addressed in notice-and-action mechanisms.

Article 15 — Reasoned Decisions

Hosting providers that decide to remove content, demonetise content, suspend accounts, or take other actions must provide a "clear and specific statement of reasons" to the affected user. For AI-moderated decisions, this means the platform must be able to articulate why the AI system flagged the content — even if the underlying technical reasoning is opaque.

Article 16 — Statement of Reasons Database

Platforms must submit statements of reasons to the European Commission's transparency database. The Commission analyses these for patterns including over-moderation and under-moderation.

Article 17 — Internal Complaint-Handling

Affected users must have access to a free, efficient internal complaint-handling system for at least six months after the moderation decision. This is a critical check on AI moderation: users can challenge AI decisions, and the platform must review.

Article 21 — Out-of-Court Dispute Settlement

For VLOPs and VLOSEs, certified out-of-court bodies can resolve disputes between users and platforms over moderation decisions.

Article 24 — Transparency Reporting

All intermediaries must publish annual transparency reports describing their content moderation activities, including:

• Volume of moderation actions broken down by type
• Use of automated tools, including their accuracy and error rates
• Average time to respond to notices
• Internal complaint-handling outcomes

For VLOPs and VLOSEs, the reports must be far more detailed.

Article 26 — Advertising Transparency

Platforms must label advertisements as such and provide users with information about why a specific ad was shown to them. Where AI is used to target ads, the platform must explain the parameters.

Article 27 — Recommender System Transparency

Online platforms using recommender systems must explain the main parameters in plain language. VLOPs must additionally offer at least one non-profiling-based recommender option.

Article 34–37 — Systemic Risk Obligations (VLOPs/VLOSEs)

Very Large Online Platforms and Very Large Online Search Engines face the most extensive obligations:

• Article 34 — systemic risk assessment, including risks from AI-driven moderation
• Article 35 — risk mitigation measures
• Article 37 — independent audit of compliance

The systemic-risk regime is the most likely route to AI-Act-style scrutiny for AI moderation: VLOP/VLOSE audits explicitly assess the AI systems used in moderation.

When Content Moderation Is High-Risk Under the AI Act

A handful of moderation contexts are high-risk under Annex III:

Annex III, point 5(a) — public-service eligibility. AI used by public authorities or on their behalf to evaluate eligibility for public assistance is high-risk. A moderation-style system used by a public agency to assess social-benefit applications would qualify.

Annex III, point 5(d) — emergency dispatch. AI used to classify emergency calls is high-risk. This includes AI moderation of emergency-line communications.

Annex III, point 6 — law enforcement. AI used by law enforcement to evaluate evidence reliability or profile individuals is high-risk. Forensic moderation tools fall here.

Annex III, point 8(b) — election influence. AI specifically designed to influence elections falls under this point. Moderation systems used solely on political content during election periods, if they could be argued to influence election outcomes, might be in scope — though commercial platforms have generally argued this should not apply to general moderation.

In each case, where AI moderation is high-risk, the full Article 8–15 regime applies.

Specific Deployment Scenarios

Hate-Speech Classification on a Social Media Platform

A typical machine-learning classifier that flags hate speech for review:

• AI Act: not high-risk; Article 5 prohibitions generally not triggered; Article 50 disclosure where users see AI-generated notifications
• DSA: Article 14 (notice-and-action), Article 15 (reasoned decisions), Article 17 (complaint handling), Article 24 (transparency reporting)
• Practical compliance: extensive DSA documentation; modest AI Act compliance

CSAM (Child Sexual Abuse Material) Detection

Hash-matching and AI classifiers detecting CSAM:

• AI Act: not high-risk; Article 50 disclosure not generally relevant (the moderation is not user-facing in real time)
• DSA: full content-moderation regime; mandatory reporting under the proposed Child Sexual Abuse Regulation (CSAR)
• GDPR: balancing of detection with data protection; case law (notably C-115/22) provides specific guidance
• Practical compliance: heavy regulatory overlay from DSA and CSAR; AI Act adds limited requirements

Copyright Infringement Detection

Content ID-style systems that detect infringing material:

• AI Act: not high-risk; could face GPAI-provider implications if built on GPAI models
• DSA: full content-moderation regime; specific provisions for trusted-flagger frameworks
• Copyright Directive: Article 17 of Directive 2019/790 provides the substantive basis for many automated copyright moderation systems
• Practical compliance: copyright-specific regulatory framework; AI Act and DSA layer on top

Spam Filtering in Email

AI-driven email spam filtering operated by email providers:

• AI Act: not high-risk; not directly captured by Article 50
• DSA: applies to email service providers as intermediaries
• ePrivacy Directive: provides specific rules on processing of communications
• Practical compliance: established practice; AI Act adds minimal direct requirements

AI Moderation of User-Generated Content in a Marketplace

AI flagging fraudulent listings, prohibited items, or misleading information:

• AI Act: not high-risk; Article 50 disclosure where AI-generated messages are sent to sellers
• DSA: full content-moderation regime; specific marketplace provisions for traceability of traders (Article 30)
• Consumer law: Unfair Commercial Practices Directive considerations
• Practical compliance: substantial DSA overlay; AI Act adds limited direct requirements

AI-Drafted Moderation Decision Explanations

LLM-drafted explanations of moderation decisions sent to users:

• AI Act: Article 50(4) for AI-generated text used in public-interest communications — but the editorial-review carve-out applies if a human signs off
• DSA: Article 15 reasoned-decision requirements apply substantively to the content of the explanation
• Practical compliance: human review of AI-drafted explanations satisfies both regimes

Compliance Checklist for Platforms Using AI Moderation

1. Classify the moderation system. Is it deployed in any Annex III area? If yes, treat as high-risk. If no, focus on DSA and Article 50.
2. Verify no Article 5 prohibitions apply. Moderation that materially distorts user behaviour with significant harm, or that uses biometric categorisation to infer sensitive attributes, is prohibited.
3. Implement DSA compliance. Notice-and-action, reasoned decisions, complaint-handling, statement-of-reasons database submission, transparency reporting.
4. Document AI accuracy. Even outside the high-risk regime, document the moderation system's accuracy and error rates for transparency reporting and complaint-handling.
5. Provide human review paths. DSA Article 17 internal complaint-handling, plus voluntary human-in-the-loop for high-impact moderation decisions.
6. Apply Article 50 where applicable. AI-generated user notifications, AI-drafted moderation decisions, and any user-facing AI interaction should be disclosed.
7. Coordinate with GDPR. Moderation often involves personal data; Article 22 GDPR may apply to fully automated significant decisions.
8. For VLOPs/VLOSEs, conduct systemic risk assessment. AI moderation is in scope for the DSA's Articles 34–37 assessment and audit obligations.
9. Retain GPAI-provider documentation. If you use GPAI models in moderation, retain the documentation that the GPAI provider makes available to downstream providers under Article 53(1)(b).

Conclusion

AI content moderation sits at the intersection of the EU AI Act and the Digital Services Act, with the DSA generally being the more demanding regime for platform operators. The AI Act adds direct obligations through Article 5 prohibitions and Article 50 transparency, but the bulk of compliance work for moderation systems flows through DSA notice-and-action mechanisms, transparency reporting, complaint-handling, and (for VLOPs) systemic-risk assessment.

For the broader picture of how AI systems are regulated based on use case, see Annex III explained. For the Article 50 framework that applies to user-facing AI more generally, see transparency obligations under the EU AI Act.