Moderate posts
Tag each issue and score each post against your own policy.
Every community draws its lines in a different place. Here, honest criticism is fine and members may promote their own work once, but scams and threats come down. This guide writes that policy into the request: Sage2 tags each issue the policy names and scores how harmful the post is.
Clear cases are handled at once. Unclear ones go to a moderator.
- Tags
- Scale
- Policy sent with every post
- Honest criticism is allowed.
- A member may promote their own work once.
- spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam
- scam_or_fraud: deceptive offer, fake results, or payment trickery
- threat: threatens or hints at harm to a person, even in veiled words
- Tag the issues in the postTags
- spamno · under 0.01%
- scam or fraudapplies · 99.99%
- threatno · under 0.01%
- Score the harm of the postScale155 ms
2.97 of 3
Severe: a threat, or a scam asking for money
- Your codePost hidden
The application hides the post for scam or fraud. Harm score 2.97 of 3.
Both questions ran in one request: 155 ms.
Get the code
Set SAGE_API_KEY (get a free key), then run it. Python: pip install requests. TypeScript: Node 22+. Save as recipe.mts and run npx tsx recipe.mts.
A null answer is not an error. Sage2 isn't sure, and each step below says what to do with it. Limits, errors and retries: Limits and errors.
import os
import requests
def sage(path, body):
r = requests.post(
"https://sage.levanto.ai" + path,
headers={"Authorization": f"Bearer {os.environ['SAGE_API_KEY']}"},
json=body,
timeout=30,
)
r.raise_for_status()
return r.json()
post = "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left."
out = sage("/decide/batch", {
"requests": [
{
"content": post,
"questions": [
{
"id": "issues",
"kind": "tags",
"instructions": "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.",
"tags": [
{
"id": "spam",
"name": "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam",
},
{
"id": "scam_or_fraud",
"name": "scam_or_fraud: deceptive offer, fake results, or payment trickery",
},
{
"id": "threat",
"name": "threat: threatens or hints at harm to a person, even in veiled words",
},
],
},
{
"id": "severity",
"kind": "scale",
"instructions": "How harmful is this post to the community?",
"levels": [
{"level": 0, "description": "No harm"},
{"level": 1, "description": "Low: rude or off-topic, but allowed"},
{"level": 2, "description": "High: spam or a misleading offer"},
{
"level": 3,
"description": "Severe: a threat, or a scam asking for money",
},
],
},
],
},
],
})
def answers(out, group=0):
# Each answer is a full decision. Failed ones (ok is false) are left out: treat a missing id as "not sure".
return {a["result"]["id"]: a["result"]["result"] for a in out["results"][group]["answers"] if a["ok"]}
a = answers(out)
# A missing answer means Sage didn't answer it: treat it as not sure.
tags = a["issues"]["tags"] if "issues" in a else None
harm = a["severity"]["expectation"] if "severity" in a else None
issues = [t["id"] for t in tags or [] if t["applies"]]
if issues:
print("Hide it:", issues)
elif tags is None or harm is None or any(t["applies"] is None for t in tags):
print("Queue for a moderator: Sage is not sure")
elif harm >= 1.5: # nearer "High" than "Low"
print(f"Queue for a moderator: harm {harm:.1f}/3")
else:
print("Leave it up")
How it works
Send the post as
contentand ask two questions in one/decide/batchrequest. The first,issues, is a Tags question with one tag per thing your policy forbids.Each tag's
nameis a one-line definition. Say what doesn't count, too: "a single genuine offer is not spam" keeps a member's one-time promotion up. Theinstructionsstate the policy itself.Each tag comes back with
applies(true,falseornull) and aprobability.Runs with the helper from Get the code.
post = "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left." out = sage("/decide/batch", { "requests": [ { "content": post, "questions": [ { "id": "issues", "kind": "tags", "instructions": "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.", "tags": [ { "id": "spam", "name": "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam", }, { "id": "scam_or_fraud", "name": "scam_or_fraud: deceptive offer, fake results, or payment trickery", }, { "id": "threat", "name": "threat: threatens or hints at harm to a person, even in veiled words", }, ], }, { "id": "severity", "kind": "scale", "instructions": "How harmful is this post to the community?", "levels": [ {"level": 0, "description": "No harm"}, {"level": 1, "description": "Low: rude or off-topic, but allowed"}, {"level": 2, "description": "High: spam or a misleading offer"}, { "level": 3, "description": "Severe: a threat, or a scam asking for money", }, ], }, ], }, ], })Example Sage2 output
{ "results": [ { "answers": [ { "ok": true, "result": { "id": "issues", "kind": "tags", "result": { "tags": [ { "id": "spam", "probability": 0, "applies": false }, { "id": "scam_or_fraud", "probability": 1, "applies": true }, { "id": "threat", "probability": 0, "applies": false } ] }, "meta": { "model": "levanto-sage-2" } }, "error": null }, { "ok": true, "result": { "id": "severity", "kind": "scale", "result": { "expectation": 2.9738, "probabilities": [ { "level": 0, "probability": 0.0002 }, { "level": 1, "probability": 0.0002 }, { "level": 2, "probability": 0.0253 }, { "level": 3, "probability": 0.9743 } ], "confidence": 0.9123 }, "meta": { "model": "levanto-sage-2" } }, "error": null } ] } ], "meta": { "model": "levanto-sage-2", "request_count": 1, "question_count": 2, "latency_ms": 154.7, "usage": { "billed_input_tokens": 592, "image_count": 0, "image_tokens": 0, "input_tokens": 592, "output_tokens": 0 } } }The second question,
severity, is a Scale question. You define thelevels, from 0 (no harm) to 3 (a threat, or a scam asking for money), each with a description. Keep the levels apart, so each post fits one.Sage2 returns a
probabilityfor every level andexpectation, the probability-weighted average level. Here it is between 0 and 3.The
answershelper turns the batch response into a dictionary keyed by questionid.Runs with the helper from Get the code.
def answers(out, group=0): # Each answer is a full decision. Failed ones (ok is false) are left out: treat a missing id as "not sure". return {a["result"]["id"]: a["result"]["result"] for a in out["results"][group]["answers"] if a["ok"]} a = answers(out) # A missing answer means Sage didn't answer it: treat it as not sure. tags = a["issues"]["tags"] if "issues" in a else None harm = a["severity"]["expectation"] if "severity" in a else None- Your code
Decide what happens to the post
- Hide it when any tag applies.
- Queue it for a moderator when any tag is
null, an answer is missing, or the harm score is 1.5 or more (nearer High than Low). - Leave it up otherwise.
Use the score, not just the tags. A post can match no tag and still do harm. The harm score goes with the post, so moderators see the worst cases first.
Runs with the helper from Get the code.
issues = [t["id"] for t in tags or [] if t["applies"]] if issues: print("Hide it:", issues) elif tags is None or harm is None or any(t["applies"] is None for t in tags): print("Queue for a moderator: Sage is not sure") elif harm >= 1.5: # nearer "High" than "Low" print(f"Queue for a moderator: harm {harm:.1f}/3") else: print("Leave it up")
Before you ship
- Say what doesn't count. Tag definitions that name the allowed cases keep honest criticism and one-time promotions up.
- Send history when the policy needs it. "Promote their own work once" can't be judged from one post. Add the member's recent posts to
content. - Fail closed. If the call errors or an answer has
ok: false, queue the post for a moderator. - Sort the queue by harm. Show moderators the posts with the highest
expectationfirst. - Fast and small. Both questions come back in one request, in about 0.15 s.
- Log the tags and score with every action. A moderator can then see why a post was hidden.