Slack

Connect A vs B to a Slack workspace and route experiment + flag notifications to specific channels. Notifications are pre-filtered by the org-wide keyword gate so only experiments matching your release-vs-test naming convention reach Slack.

Prerequisites

• You need the Manage integrations permission in your A vs B org.
• A Slack admin must approve the install (Slack requires admin consent for new apps).
• For private channels: invite @AvsB in the channel before adding it as a destination.

Connecting a workspace

1. Open Organization Settings → Integrations in A vs B.
2. In the Slack section click Add to Slack.
3. You'll be redirected to Slack's OAuth consent screen. Review the requested scopes:
   • chat:write: post messages.
   • chat:write.public: post to public channels we haven't joined.
   • channels:read + groups:read: list channels for the picker.
   • team:read: read the workspace name and icon.
   • users:read: resolve who installed the app.
4. Click Allow. You'll be redirected back to A vs B and the workspace will appear in the Slack section.

Adding a channel destination

1. Click Add channel on the workspace card.
2. Search the channel picker. Public channels show up automatically. Private channels show up only after you invite @AvsB in Slack first.
3. (Optional) Override the display name. Defaults to A vs B → #channel-name.
4. Click Add destination.
5. Use Send test on the new channel row to verify delivery. You should see a sample experiment launched card in Slack within a few seconds.

Routing events to channels

A destination on its own doesn't fire: you need a route. Routes live under each project and map event types to destinations. Click Manage routing on a Slack destination card to pick which events should post to it.

Event vocabulary

Slack supports the full A vs B event vocabulary. Some events are gated (only fire for experiments matching your keyword convention) and some are safety events that bypass the gate so an incident can't be silenced by a naming typo.

• Experiment lifecycle (gated): launched, paused, resumed, completed, changes published, visual edits saved.
• Analytical (gated, except safety): winner declared, results-ready daily digest.
• Safety (bypass gate): SRM check failed, guardrail breached. Cards are tagged "Safety alert: name gate bypassed" so you know an out-of-convention experiment triggered them.
• Flags (not gated): flag published, rule activated.
• Flag A/B rules: SRM check failed and guardrail breached (both bypass the gate, like their experiment counterparts), plus the results-ready daily digest (gated on the rule name). A flag rule running as an A/B test is a real experiment, so it raises the same alerts: on its own toggles, so you can route them to a different channel.
• Exclusion groups (not gated): created, updated, deleted.

Daily results digest threading

The experiment.results_ready event fires once per gated experiment per day. To keep channels tidy, A vs B posts a single parent message per workspace per day ("Daily results: 2026-05-18, N active experiments") and threads each experiment summary as a reply to it. The threading is dedupable: if two cron invocations race, only one parent message is posted (we use a Postgres unique constraint on (destination, digestRunId)).

flag.results_ready runs on the same daily schedule and obeys the same keyword gate, but posts as its own message instead of threading under that parent: the parent's headline counts experiments, and a flag rule isn't counted in it.

Org-wide format preferences

Under Org Settings → Notifications you can configure how lift, confidence, and theme render in Slack messages:

• Lift decimal precision (1 / 2 / 3, default 2). Affects +5.42% formatting.
• Confidence display: "percentage" (95.0%) or "posterior probability" (P(better) = 0.950).
• Theme preview: light or dark. Used only in docs screenshots.

Disconnect / reconnect flow

Click Disconnect on a workspace to revoke the bot token in Slack and remove every channel destination on that workspace. Delivery history is preserved for the audit log. To reconnect, click Add to Slack again: if the workspace was previously connected and is now revoked, the OAuth callback rotates the token in place and re-enables existing destinations without losing routes.

Audit-log entries

The Slack integration emits these audit actions:

• SLACK_WORKSPACE_CONNECTED: fresh install or reconnect.
• SLACK_WORKSPACE_DISCONNECTED: user-initiated disconnect.
• SLACK_WORKSPACE_REVOKED: system-source, fired when Slack returns token_revoked on a send attempt (e.g., admin removed the app from Slack).
• SLACK_TEST_MESSAGE_SENT: captured success or failure from the "Send test" button.

All entries use the resource type SLACK_WORKSPACE_CONNECTION.

Troubleshooting

I revoked the A vs B app from Slack: how do I reconnect?

Open Organization Settings → Integrations. The workspace card shows a red repair banner: "Slack workspace is disconnected. Reconnect to resume notifications." Click Reconnect. The OAuth flow runs again, rotates the bot token, and your existing channel destinations resume sending.

A private channel doesn't appear in the picker

Private channels show up only after you invite the bot. In Slack, type /invite @AvsB inside the channel, then re-open the channel picker in A vs B: the channel will appear.

A notification didn't arrive: is the experiment name gated?

Most experiment events are filtered by the org-wide keyword gate. If your experiment name doesn't contain one of the configured keywords (e.g., LIVE, RELEASE), the event is suppressed by design. Check Settings → Notifications → Keywords and adjust if needed. Safety events (SRM failed, guardrail breached) are never suppressed: those always reach Slack with a "name gate bypassed" badge.

Test send worked, but real events don't arrive

Open Settings → Integrations → Deliveries. Find the latest row for the destination. If status is PERMANENT_FAILURE the error message tells you exactly what Slack rejected. If status is PENDING with retries, the dispatcher is backing off: usually Slack rate-limiting under burst.

Managing the connection

Slack is an OAuth connection managed entirely from the dashboard: Settings → Integrations → Slack. From there you can connect a workspace, pick channels, send a test message, and disconnect (which revokes the bot token and removes any notification destinations routed to it).