This bot can be installed into an OpenChat community or group to check messages against the chat's own rules.
It reads the enabled rules text of the group, or of the community and the channel, and treats each line as one rule. It asks TypeSafe's Jev model how likely it is that a message breaks each rule, in a single call per message. A message is flagged when any rule's score is at or above the chat's threshold. OpenChat already applies general content moderation to public messages, so the bot does not repeat it.
It is also possible to configure how the bot behaves when it decides that a message violates the rules it is applying. It can either react to the message with a custom emoji that you define or it can delete the message. Start with reactions. Jev reads message text literally, so a message can try to talk it out of a flag, and a bad call is easier to undo when the bot only reacts. In the future we can extend this to add more actions. For example, it might be desirable to give a user the benefit of the doubt once or twice and then gradually escalate up to and including ejecting that user from the community or group.
And finally it is also possible to configure if and how the bot explains its decisions. It can either offer no explanation, it can quote reply to the offending message with the explanation, or it can reply in a thread with the explanation. The explanation names each broken rule, whether it is a community, channel or group rule, and its score, e.g. It looks like this message broke the community rule "No promo links" (0.94 confidence).
/threshold sets how strict the bot is. Higher values are more permissive. The default is 0.8.
This bot must be given permission to read all messages in a chat in order to function. In public chats with rules enabled, it sends the content of each message, its type and whether it is in a thread to TypeSafe (without the sender's identity). As a community or group owner, you must decide for yourself whether that is acceptable before installing the bot.
The bot never sends messages from private chats, or from chats with no rules enabled, anywhere. If TypeSafe can't be reached, the message is left alone.
You can use the following commands to interact with this bot:
/help: Display this summary of commands
/pause: Pauses moderation in this chat
/resume: Resumes moderation in this chat
/status: Display current configuration and the rules being checked in this chat
/action: Configure action taken when rules are broken
/explanation: Configure if and how the bot explains decisions
/threshold: Configure how strictly the chat rules are applied
/explain: Explain the reason for moderation on a single message
This bot is implemented as a set of AWS lambda functions which are provisioned using the serverless-framework.
State is managed via a cloudbase postgres db hosted by Neon.
We use the Drizzle ORM for interacting with database and the drizzle schema should be considered the source of truth for the data structure. Migrations should be generated by changing the code schema and using drizzle-kit to apply the migration to the database.
npx serverless dev: deploys the dev version of the lambda functions and routes traffic through to your local dev evironment
npx serverless deploy --stage prod: deploy the prod version of lambda to live
npx serverless logs --function [functionName] --stage prod --tail: tails the production logs of a live lambda function
To pull messages from the dead letter queue for the purposes of debug, you can do this:
aws sqs receive-message \
--queue-url https://sqs.eu-west-2.amazonaws.com/253040722768/moderation-dlq \
--max-number-of-messages 1 \
--attribute-names All \
--message-attribute-names All \
--region eu-west-2
assuming that you have aws cli configured