# Moderate posts

Moderate posts against your own policy: tag each issue and score how harmful the post is.

Category: Review payments and posts. Live demo: https://levanto.ai/examples/content-moderation

Every community draws its lines in a different place. Here, honest criticism is fine and members may promote their own work once, but scams and threats come down. This guide writes **that policy into the request**: Sage2 tags each issue the policy names and scores how harmful the post is.

Clear cases are handled at once. Unclear ones go to **a moderator**.

## Setup

Base URL: `https://sage.levanto.ai`. Send `Authorization: Bearer $SAGE_API_KEY` on every call. Get a key, free, at https://platform.levanto.ai. Keep the key on your server.

A `null` answer is not an error: Sage2 isn't sure. Each step says what to do with it.

## Step by step

### 1. Write your policy as tags

Send the post as `content` and ask two questions in one `/decide/batch` request. The first, `issues`, is a Tags question with one tag per thing your policy forbids.

Each tag's `name` is a one-line definition. **Say what doesn't count**, too: "a single genuine offer is not spam" keeps a member's one-time promotion up. The `instructions` state the policy itself.

Each tag comes back with `applies` (`true`, `false` or `null`) and a `probability`.

```python
post = "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left."

out = sage("/decide/batch", {
    "requests": [
        {
            "content": post,
            "questions": [
                {
                    "id": "issues",
                    "kind": "tags",
                    "instructions": "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.",
                    "tags": [
                        {
                            "id": "spam",
                            "name": "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam",
                        },
                        {
                            "id": "scam_or_fraud",
                            "name": "scam_or_fraud: deceptive offer, fake results, or payment trickery",
                        },
                        {
                            "id": "threat",
                            "name": "threat: threatens or hints at harm to a person, even in veiled words",
                        },
                    ],
                },
                {
                    "id": "severity",
                    "kind": "scale",
                    "instructions": "How harmful is this post to the community?",
                    "levels": [
                        {"level": 0, "description": "No harm"},
                        {"level": 1, "description": "Low: rude or off-topic, but allowed"},
                        {"level": 2, "description": "High: spam or a misleading offer"},
                        {
                            "level": 3,
                            "description": "Severe: a threat, or a scam asking for money",
                        },
                    ],
                },
            ],
        },
    ],
})
```

```typescript
const post = "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left.";

const out = await sage("/decide/batch", {
  requests: [
    {
      content: post,
      questions: [
        {
          id: "issues",
          kind: "tags",
          instructions: "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.",
          tags: [
            {
              id: "spam",
              name: "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam",
            },
            {
              id: "scam_or_fraud",
              name: "scam_or_fraud: deceptive offer, fake results, or payment trickery",
            },
            {
              id: "threat",
              name: "threat: threatens or hints at harm to a person, even in veiled words",
            },
          ],
        },
        {
          id: "severity",
          kind: "scale",
          instructions: "How harmful is this post to the community?",
          levels: [
            { level: 0, description: "No harm" },
            { level: 1, description: "Low: rude or off-topic, but allowed" },
            { level: 2, description: "High: spam or a misleading offer" },
            { level: 3, description: "Severe: a threat, or a scam asking for money" },
          ],
        },
      ],
    },
  ],
});
```

Response, abridged (a real run):

```json
{
  "results": [
    {
      "answers": [
        {
          "ok": true,
          "result": {
            "id": "issues",
            "kind": "tags",
            "result": {
              "tags": [
                {
                  "id": "spam",
                  "probability": 0,
                  "applies": false
                },
                {
                  "id": "scam_or_fraud",
                  "probability": 1,
                  "applies": true
                },
                {
                  "id": "threat",
                  "probability": 0,
                  "applies": false
                }
              ]
            },
            "meta": {
              "model": "levanto-sage-2"
            }
          },
          "error": null
        },
        {
          "ok": true,
          "result": {
            "id": "severity",
            "kind": "scale",
            "result": {
              "expectation": 2.9738,
              "probabilities": [
                {
                  "level": 0,
                  "probability": 0.0002
                },
                {
                  "level": 1,
                  "probability": 0.0002
                },
                {
                  "level": 2,
                  "probability": 0.0253
                },
                {
                  "level": 3,
                  "probability": 0.9743
                }
              ],
              "confidence": 0.9123
            },
            "meta": {
              "model": "levanto-sage-2"
            }
          },
          "error": null
        }
      ]
    }
  ],
  "meta": {
    "model": "levanto-sage-2",
    "request_count": 1,
    "question_count": 2,
    "latency_ms": 154.7,
    "usage": {
      "billed_input_tokens": 592,
      "image_count": 0,
      "image_tokens": 0,
      "input_tokens": 592,
      "output_tokens": 0
    }
  }
}
```

### 2. Score the harm

The second question, `severity`, is a Scale question. You define the `levels`, from 0 (no harm) to 3 (a threat, or a scam asking for money), each with a description. Keep the levels apart, so each post fits one.

Sage2 returns a `probability` for every level and `expectation`, the probability-weighted average level. Here it is between 0 and 3.

The `answers` helper turns the batch response into a dictionary keyed by question `id`.

```python
def answers(out, group=0):
    # Each answer is a full decision. Failed ones (ok is false) are left out: treat a missing id as "not sure".
    return {a["result"]["id"]: a["result"]["result"] for a in out["results"][group]["answers"] if a["ok"]}

a = answers(out)
# A missing answer means Sage didn't answer it: treat it as not sure.
tags = a["issues"]["tags"] if "issues" in a else None
harm = a["severity"]["expectation"] if "severity" in a else None
```

```typescript
// Each answer is a full decision. Failed ones (ok is false) are left out: treat a missing id as "not sure".
const answers = (out: any, group = 0): Record<string, any> =>
  Object.fromEntries(out.results[group].answers.filter((a: any) => a.ok).map((a: any) => [a.result.id, a.result.result]));

const a = answers(out);
// A missing answer means Sage didn't answer it: treat it as not sure.
const tags: { id: string; applies: boolean | null }[] | null = a.issues?.tags ?? null;
const harm: number | null = a.severity?.expectation ?? null;
```

### 3. Decide what happens to the post

- **Hide it** when any tag applies.
- **Queue it for a moderator** when any tag is `null`, an answer is missing, or the harm score is 1.5 or more (nearer High than Low).
- **Leave it up** otherwise.

**Use the score, not just the tags.** A post can match no tag and still do harm. The harm score goes with the post, so moderators see the worst cases first.

```python
issues = [t["id"] for t in tags or [] if t["applies"]]

if issues:
    print("Hide it:", issues)
elif tags is None or harm is None or any(t["applies"] is None for t in tags):
    print("Queue for a moderator: Sage is not sure")
elif harm >= 1.5:  # nearer "High" than "Low"
    print(f"Queue for a moderator: harm {harm:.1f}/3")
else:
    print("Leave it up")
```

```typescript
const issues = (tags ?? []).filter(t => t.applies).map(t => t.id);

if (issues.length) {
  console.log("Hide it:", issues);
} else if (tags === null || harm === null || tags.some(t => t.applies === null)) {
  console.log("Queue for a moderator: Sage is not sure");
} else if (harm >= 1.5) { // nearer "High" than "Low"
  console.log(`Queue for a moderator: harm ${harm.toFixed(1)}/3`);
} else {
  console.log("Leave it up");
}
```

## Before you ship

- **Say what doesn't count.** Tag definitions that name the allowed cases keep honest criticism and one-time promotions up.
- **Send history when the policy needs it.** "Promote their own work once" can't be judged from one post. Add the member's recent posts to `content`.
- **Fail closed.** If the call errors or an answer has `ok: false`, queue the post for a moderator.
- **Sort the queue by harm.** Show moderators the posts with the highest `expectation` first.
- **Fast and small.** Both questions come back in one request, in about 0.15 s.
- **Log the tags and score** with every action. A moderator can then see why a post was hidden.

## Full recipe (Python, `pip install requests`)

```python
import os
import requests

def sage(path, body):
    r = requests.post(
        "https://sage.levanto.ai" + path,
        headers={"Authorization": f"Bearer {os.environ['SAGE_API_KEY']}"},
        json=body,
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

post = "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left."

out = sage("/decide/batch", {
    "requests": [
        {
            "content": post,
            "questions": [
                {
                    "id": "issues",
                    "kind": "tags",
                    "instructions": "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.",
                    "tags": [
                        {
                            "id": "spam",
                            "name": "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam",
                        },
                        {
                            "id": "scam_or_fraud",
                            "name": "scam_or_fraud: deceptive offer, fake results, or payment trickery",
                        },
                        {
                            "id": "threat",
                            "name": "threat: threatens or hints at harm to a person, even in veiled words",
                        },
                    ],
                },
                {
                    "id": "severity",
                    "kind": "scale",
                    "instructions": "How harmful is this post to the community?",
                    "levels": [
                        {"level": 0, "description": "No harm"},
                        {"level": 1, "description": "Low: rude or off-topic, but allowed"},
                        {"level": 2, "description": "High: spam or a misleading offer"},
                        {
                            "level": 3,
                            "description": "Severe: a threat, or a scam asking for money",
                        },
                    ],
                },
            ],
        },
    ],
})

def answers(out, group=0):
    # Each answer is a full decision. Failed ones (ok is false) are left out: treat a missing id as "not sure".
    return {a["result"]["id"]: a["result"]["result"] for a in out["results"][group]["answers"] if a["ok"]}

a = answers(out)
# A missing answer means Sage didn't answer it: treat it as not sure.
tags = a["issues"]["tags"] if "issues" in a else None
harm = a["severity"]["expectation"] if "severity" in a else None

issues = [t["id"] for t in tags or [] if t["applies"]]

if issues:
    print("Hide it:", issues)
elif tags is None or harm is None or any(t["applies"] is None for t in tags):
    print("Queue for a moderator: Sage is not sure")
elif harm >= 1.5:  # nearer "High" than "Low"
    print(f"Queue for a moderator: harm {harm:.1f}/3")
else:
    print("Leave it up")
```

## Full recipe (TypeScript, Node 22+, nothing to install: save as `recipe.mts`, run `npx tsx recipe.mts`)

```typescript
// Node 22+. Save as recipe.mts and run npx tsx recipe.mts
async function sage(path: string, body: unknown): Promise<any> {
  const r = await fetch("https://sage.levanto.ai" + path, {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.SAGE_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify(body),
    signal: AbortSignal.timeout(30_000),
  });
  if (!r.ok) throw new Error(`${r.status} ${await r.text()}`);
  return r.json();
}

const post = "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left.";

const out = await sage("/decide/batch", {
  requests: [
    {
      content: post,
      questions: [
        {
          id: "issues",
          kind: "tags",
          instructions: "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.",
          tags: [
            {
              id: "spam",
              name: "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam",
            },
            {
              id: "scam_or_fraud",
              name: "scam_or_fraud: deceptive offer, fake results, or payment trickery",
            },
            {
              id: "threat",
              name: "threat: threatens or hints at harm to a person, even in veiled words",
            },
          ],
        },
        {
          id: "severity",
          kind: "scale",
          instructions: "How harmful is this post to the community?",
          levels: [
            { level: 0, description: "No harm" },
            { level: 1, description: "Low: rude or off-topic, but allowed" },
            { level: 2, description: "High: spam or a misleading offer" },
            { level: 3, description: "Severe: a threat, or a scam asking for money" },
          ],
        },
      ],
    },
  ],
});

// Each answer is a full decision. Failed ones (ok is false) are left out: treat a missing id as "not sure".
const answers = (out: any, group = 0): Record<string, any> =>
  Object.fromEntries(out.results[group].answers.filter((a: any) => a.ok).map((a: any) => [a.result.id, a.result.result]));

const a = answers(out);
// A missing answer means Sage didn't answer it: treat it as not sure.
const tags: { id: string; applies: boolean | null }[] | null = a.issues?.tags ?? null;
const harm: number | null = a.severity?.expectation ?? null;

const issues = (tags ?? []).filter(t => t.applies).map(t => t.id);

if (issues.length) {
  console.log("Hide it:", issues);
} else if (tags === null || harm === null || tags.some(t => t.applies === null)) {
  console.log("Queue for a moderator: Sage is not sure");
} else if (harm >= 1.5) { // nearer "High" than "Low"
  console.log(`Queue for a moderator: harm ${harm.toFixed(1)}/3`);
} else {
  console.log("Leave it up");
}
```

## Requests

### 1. POST /decide/batch

```bash
curl -s https://sage.levanto.ai/decide/batch \
  -H "Authorization: Bearer $SAGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<'JSON'
{
  "requests": [
    {
      "content": "Guaranteed 40% monthly returns!! Send a deposit to my wallet today and I will double it by Friday. Only 3 spots left.",
      "questions": [
        {
          "id": "issues",
          "kind": "tags",
          "instructions": "Tag the moderation issues in this post, under a policy that allows honest criticism and allows members to promote their own work once.",
          "tags": [
            {
              "id": "spam",
              "name": "spam: unsolicited bulk or repetitive posting; a single genuine offer is not spam"
            },
            {
              "id": "scam_or_fraud",
              "name": "scam_or_fraud: deceptive offer, fake results, or payment trickery"
            },
            {
              "id": "threat",
              "name": "threat: threatens or hints at harm to a person, even in veiled words"
            }
          ]
        },
        {
          "id": "severity",
          "kind": "scale",
          "instructions": "How harmful is this post to the community?",
          "levels": [
            {
              "level": 0,
              "description": "No harm"
            },
            {
              "level": 1,
              "description": "Low: rude or off-topic, but allowed"
            },
            {
              "level": 2,
              "description": "High: spam or a misleading offer"
            },
            {
              "level": 3,
              "description": "Severe: a threat, or a scam asking for money"
            }
          ]
        }
      ]
    }
  ]
}
JSON
```

Response, abridged (a real run):

```json
{
  "results": [
    {
      "answers": [
        {
          "ok": true,
          "result": {
            "id": "issues",
            "kind": "tags",
            "result": {
              "tags": [
                {
                  "id": "spam",
                  "probability": 0,
                  "applies": false
                },
                {
                  "id": "scam_or_fraud",
                  "probability": 1,
                  "applies": true
                },
                {
                  "id": "threat",
                  "probability": 0,
                  "applies": false
                }
              ]
            },
            "meta": {
              "model": "levanto-sage-2"
            }
          },
          "error": null
        },
        {
          "ok": true,
          "result": {
            "id": "severity",
            "kind": "scale",
            "result": {
              "expectation": 2.9738,
              "probabilities": [
                {
                  "level": 0,
                  "probability": 0.0002
                },
                {
                  "level": 1,
                  "probability": 0.0002
                },
                {
                  "level": 2,
                  "probability": 0.0253
                },
                {
                  "level": 3,
                  "probability": 0.9743
                }
              ],
              "confidence": 0.9123
            },
            "meta": {
              "model": "levanto-sage-2"
            }
          },
          "error": null
        }
      ]
    }
  ],
  "meta": {
    "model": "levanto-sage-2",
    "request_count": 1,
    "question_count": 2,
    "latency_ms": 154.7,
    "usage": {
      "billed_input_tokens": 592,
      "image_count": 0,
      "image_tokens": 0,
      "input_tokens": 592,
      "output_tokens": 0
    }
  }
}
```

## Docs

- Tags: https://docs.levanto.ai/decision-model/tags
- Scale: https://docs.levanto.ai/decision-model/scale
- Limits and errors: https://docs.levanto.ai/decision-model/limits
- OpenAPI: https://sage.levanto.ai/openapi.json