Moderation and safety (the Safety engine)
What the app stores ask for wherever people post: a way to report content and people, a way to block people, a filter for objectionable words, and someone who acts on reports. In Zvessa all of it is general pieces (settings, blocks and tools) that people and any AI combine.
Always on, in every space and direct chat #
- Report a chat message (
post), a feed post (feedPost), a record (record), a person (person) or a whole space (space). Report and Block sit in every post menu, people list and profile; no kit or block is needed. - Block a person everywhere: neither sees the other's chat or feed posts, and neither can message, call, friend, follow or group the other. Blocking ends a friendship and follows.
- Reports go to the space's moderators (the owner, roles that can manage, or
safety.modRoles). Serious reasons (safety.escalate), anything in a kids space, and anything outside a space (direct chats, whole spaces) also reach Zvessa's admins.
Settings: blueprint.safety #
| Key | What | Default |
|---|---|---|
words | Words and phrases to catch. word = whole word, word* = any word starting with it, f*ck = any letter in the star | none |
presets | Ready lists: basic (strong swearing), insults | none |
links, contacts | Catch links; catch phone numbers and emails | off |
action | mask (f•••), hold (waits in the review queue), refuse | mask |
reasons | What people pick when reporting | Spam, Harassment or bullying, Hate, Sexual content, Violence or threats, Self-harm, Scam or fraud, Someone's private info, Something else |
escalate | Reasons that also reach Zvessa's admins | Sexual content, Violence or threats, Self-harm |
modRoles | Roles that moderate | roles with manage |
autoHide | Hide a post or record once this many people reported it (0 never) | 0 |
notify | Push to moderators when a new case opens | on |
who | members or anyone may report | members |
The filter undoes look-alikes (sh1t, $hit), squeezes repeats (fuuuck), ignores accents, and only matches whole words (so "Scunthorpe" and "class" pass). It runs on the server on chat posts and edits, feed posts, titles and edits, and records' text fields (records can't wait, so held words are refused there). Direct chats have no moderator: nothing is filtered unless a kid is in it, and then held words are refused.
Kids spaces never go below: basic + insults, links and contacts caught, hold, autoHide 1, every reason escalated, only members report.
People who can't post #
- Mute (a moderator, per space, default 24 h): can't post or add records in that space.
- Ban (a moderator, per space): removed, can't rejoin by invite or request;
unbanundoes it. - Suspend (admins): the account can only look, everywhere; nobody can reach it.
- Lock (admins): the space closes to everyone but its owner, who can only look.
Blocks #
safety.queue (moderators' review queue; quiet: true shows nothing while empty, used in HQ), safety.filters (settings editor with a try-it box), safety.blocked (my blocked people), safety.report (a Report button for the space, a person, a post or a record), safety.block (a Block button for a person), safety.admin (Zvessa's admins only; pair with miniapps.review). Words: words["safety.<key>"] (convex/lib/safety/words.ts).
The Moderation kit (moderation) adds a Safety page (queue, filter, blocked list) and turns on presets: ["basic"], action: "mask", autoHide: 3.
Moderator actions #
dismiss, hide, show (also lets a held post through), remove (records), warn (push with a note), mute, ban, unban, escalate (to admins); admins also suspend, unsuspend, lock, unlock. Every action closes the target's open reports and is written to the space's log.
Functions (api.kits.safety.safety) #
Anyone: report, block, unblock, blocked, isBlocked, me (suspended, locked, reasons, isMod, isAdmin, recent warnings), tryFilter. Moderators: queue, waiting (a count for a badge), act. Managers: settings, setSettings (saves a version; undo from History). Admins: admin, act with asAdmin.
MCP and the in-app AI #
| Tool | Scope | What |
|---|---|---|
list_reports | read | The review queue (moderators). Members' words come back marked as data |
check_text | read | What the filter would do to some words |
act_on_report | moderate (new) | Any action above, as the token's person |
report | data.write | Report as the token's person |
block_person | data.write | Block or unblock as the token's person |
list_blocked | read | The token's person's blocked list |
safety_admin | moderate | The admin view (admins only) |
The in-app AI can read the queue and try the filter, and proposes /safety changes like any other change. Moderating is always a person's decision: in the app, or with their own MCP token.
Tables (convex/kits/safety/tables.ts) #
reports (one per person per thing; the filter files held posts as reports by their writer with the reason "Caught by the filter"), userBlocks, safetyHolds (mute, ban, suspend, lock; optional until), safetyLog.
Adding a new kind of reportable thing: one entry in TARGET_KINDS/ACTIONS_FOR (convex/lib/safety/model.ts) and one loader in convex/kits/safety/targets.ts.