Content moderation
AI content moderation with your own rules
Short answer
AI content moderation checks each piece of user content against your rules and returns a verdict such as allow, review or remove, with a probability for each. Clear cases are handled automatically. Borderline content goes to a human moderator. Because you write the rules as descriptions, the policy matches your community rather than a generic filter.
Generic moderation endpoints are good at the obvious categories: explicit content, hate speech, violence. Real communities have more specific rules: no self-promotion in the help forum, no pricing discussion in reviews, no off-platform payment requests in a marketplace. Those rules are where most moderation work actually goes.
A decision model lets you express those rules directly and get a verdict you can act on, with a review queue for the cases in between.
Get Early Access to ClassifierHub: 2× credits in your first paid month.
A three-way verdict beats a yes/no
Binary moderation forces a trade-off between letting bad content through and removing good content. A three-way decision (allow, review, remove) plus probabilities gives you a middle lane: automate the confident ends and spend human time only where judgment is needed.
Add separate yes/no questions for things you track independently, such as spam or personal data, so one verdict doesn't have to carry every policy.
curl https://classifierhub.com/v1/decide \
-H "Authorization: Bearer $CLASSIFIERHUB_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"config": {
"input": { "kind": "object" },
"outputs": [
{
"key": "verdict", "type": "choice",
"instructions": "Does this comment follow the community rules?",
"options": {
"allow": "On-topic and respectful, even if critical",
"review": "Borderline: heated, off-topic or possibly misleading",
"remove": "Harassment, hate, spam, scams or explicit content"
}
},
{
"key": "is_spam", "type": "boolean",
"instructions": "Is this comment promotional spam or a scam?",
"criteria": {
"true": "Unsolicited promotion, links to sell something, scams",
"false": "A genuine comment"
}
}
]
},
"input": { "comment": "Great write-up. DM me for cheap followers!!" }
}'The ClassifierHub API opens to Early Access members first.
Writing moderation rules the model can apply
- Describe behavior, not keywords: "asks users to pay outside the platform" rather than "contains PayPal".
- Say what is allowed at the boundary, for example "critical but respectful reviews are allowed".
- Keep one policy per question when policies are enforced differently.
- Review a sample of removals every week and refine descriptions where moderators disagree.
Where it fits in your stack
Run the check when content is submitted. Publish allowed content immediately, hold content in review, and reject or hide removed content with a clear message. For high-risk surfaces such as marketplace listings, raise the threshold for automatic allow. ClassifierHub also ships a spam-detection template for a single yes/no gate on contact forms and messages.
Limits to keep in mind
AI moderation is a filter, not a legal compliance program. Keep human review for appeals and high-impact decisions, log verdicts for auditability, and don't rely on it alone for categories with legal reporting obligations.
Frequently asked questions
Related guides
Last updated . ClassifierHub is an independent product built on top of the Jev decision model, accessed through OpenRouter. It is not affiliated with or endorsed by TypeSafe or OpenRouter.