-
Notifications
You must be signed in to change notification settings - Fork 848
Replaced Pylon with SupportHog #18160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 6 commits
ba39e69
9ca46ca
72f1868
882d80c
274667c
4b4930d
31000d8
cfe5808
8884463
58be99c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,21 +16,20 @@ We use [Zendesk Support](https://posthoghelp.zendesk.com/agent) as our internal | |
|
|
||
| ### Tickets from Slack | ||
|
|
||
| Customers can [create tickets from Slack](/handbook/engineering/support-hero#pylon-to-create-zendesk-tickets-from-slack-posts) by adding the 🎫 emoji reaction. This means they can get help even when you're asleep or on holiday. Let your customer know about this, and remind them now and then. | ||
| Customers can [create tickets from Slack](/handbook/engineering/support-hero#creating-zendesk-tickets-from-slack-posts) by adding the 🎫 emoji reaction (or mentioning `@SupportHog`). This means they can get help even when you're asleep or on holiday. Let your customer know about this, and remind them now and then. | ||
|
|
||
| If it isn't working, check that you've [invited Pylon to the channel](/handbook/growth/sales/slack-channels). Fill out the [automated message asking for `Group` and `Severity`](/handbook/engineering/support-hero#pylon-to-create-zendesk-tickets-from-slack-posts) so the ticket goes to the right team — customers often forget, so fill it for them. Check [feature ownership](/handbook/engineering/feature-ownership) if you're not sure who owns a product area. | ||
|
|
||
| If you're working on a ticket the customer raised in Slack, let support know to avoid duplicate effort. Leave an internal note in Zendesk. | ||
| If you're working on a ticket the customer raised in Slack, assign yourself as the owner in PostHog Support, and then either resolve it or assign it to Team Support if you need to escalate it. | ||
|
|
||
| > Tip: Customer messages from Pylon channels also go to [#support-customer-success](https://posthog.slack.com/archives/C05MUMZLC13). Find the ticket there and reply in the thread — that also creates an internal note in Zendesk. | ||
| > Tip: Customer messages from SupportHog channels also go to [#support-managed-customers](https://posthog.slack.com/archives/C05MUMZLC13). Find the ticket there and follow the link to see the ticket in PostHog. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. at the moment it's only when a ticket is created by customers - I can update it to any time they send a reply to the ticket as well if you want?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's fine for now just to be done on ticket creation |
||
|
|
||
| ## Investigating issues | ||
|
|
||
| Ask for specifics: links to the insight, feature flag, or dashboard; a screenshot or the exact error message. | ||
|
|
||
| If it helps, log in as the customer. Clicking a link from their PostHog instance will sometimes offer a "log in as" option. Otherwise, go to <PrivateLink url="https://us.posthog.com/admin/">US admin</PrivateLink> (<PrivateLink url="https://eu.posthog.com/admin/">EU admin</PrivateLink>), search for the org or user, and click "Log in as user". If you don't see that option, ask <TeamMember name="Dana Zou" photo /> to add you as a staff member in admin. | ||
|
|
||
| Use our [docs](/docs), [troubleshooting tips](/handbook/support/troubleshooting-tips), and search Slack, Zendesk, GitHub, and Pylon for similar issues. If you've just joined, spend 30 mins to an hour investigating yourself before asking for help — onboarding is when you learn the products best. Use common sense based on urgency. | ||
| Use our [docs](/docs), [troubleshooting tips](/handbook/support/troubleshooting-tips), and search Slack, PostHog Support, Zendesk, and GitHub for similar issues. If you've just joined, spend 30 mins to an hour investigating yourself before asking for help — onboarding is when you learn the products best. Use common sense based on urgency. | ||
|
|
||
| Keep the customer in the loop while you investigate — progress, blockers, next steps. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -204,8 +204,6 @@ As a business we need to ensure we are focusing support on our paying customers, | |
|
|
||
| \* Try to be especially responsive to any customers marked as `Sales/CS Top 20`. This set of customers is regularly reviewed by the sales team, and this priority is applied to those customers we'd like to have an especially fantastic support experience. | ||
|
|
||
| \*\* Due to the way we're using Pylon, "new" tickets from high prio customer Slack channels only appear as `New` in Zendesk for a few seconds, then a webhook updates the ticket and quickly changes it to `Open`. | ||
|
|
||
| ### What if I need to confirm priority by checking a customer's MRR? | ||
|
|
||
| You've got a couple of options. By order of quickness: | ||
|
|
@@ -267,7 +265,7 @@ We use [Zendesk Support](https://posthoghelp.zendesk.com) as our internal platfo | |
|
|
||
| Zendesk allows us to manage all our customer conversations in one place and reply through Slack or email. | ||
|
|
||
| Zendesk is populated with new tickets when issues are sent via the in-app [Support panel](https://us.posthog.com/home#panel=support) (the `Help` tab in the righthand sidebar), from people outside the PostHog GitHub organization adding issues to the `posthog` and `posthog.com` repos, and new [community questions](/questions). High priority customers also have Slack channels they can post support questions in. We can [create Zendesk tickets from Slack questions via Pylon.](#pylon-to-create-zendesk-tickets-from-slack-posts) | ||
| Zendesk is populated with new tickets when issues are sent via the in-app [Support panel](https://us.posthog.com/home#panel=support) (the `Help` tab in the righthand sidebar), from people outside the PostHog GitHub organization adding issues to the `posthog` and `posthog.com` repos, and new [community questions](/questions). High priority customers also have Slack channels they can post support questions in. We can [create Zendesk tickets from Slack questions via SupportHog.](#creating-zendesk-tickets-from-slack-posts) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. a bit wrong, but seems fine for now - I'll be going through replacing all the Zendesk wording soon enough |
||
|
|
||
| The Zendesk tickets will include links to the GitHub issue, Slack thread, or the community question so we can answer in the appropriate platform. After replying to a community question, make an `internal note` on the Zendesk ticket confirming that you've replied outside of Zendesk, and set the [ticket status](#ticket-status) accordingly when submitting the internal note. | ||
|
|
||
|
|
@@ -340,11 +338,9 @@ This looks at the Content Warning field on the Zendesk Organization, and adds th | |
|
|
||
| If you do discover any potentially offensive content in a customer account then please update this field on the Zendesk Organization so that other team members are aware of the content. | ||
|
|
||
| ### Pylon to create Zendesk tickets from Slack posts | ||
|
|
||
| We use [Pylon](https://usepylon.com/) to create Zendesk tickets from Slack posts. To do so, add the `:ticket:` (🎫) emoji reaction to the post that you want to create a Zendesk ticket from. | ||
| ### Creating Zendesk tickets from Slack posts | ||
|
|
||
| Adding the `:ticket:` emoji reaction will cause Pylon to add a couple of replies in a thread under the post. The last of those replies includes options for the Zendesk ticket you're creating: Use the `Group` menu to send the ticket to the appropriate team, and the `Severity` menu to set the severity flag on the Zendesk ticket, then hit the `Submit` button. | ||
| We use [SupportHog](/handbook/growth/sales/slack-channels), our own tool, to create PostHog Support tickets from Slack posts. To do so, add the `:ticket:` (🎫) emoji reaction to the post that you want to create a PostHog Support ticket from, or mention `@SupportHog` in the thread. | ||
|
|
||
| Zendesk tickets created this way will normally be marked as high priority tickets. You can respond to them either in Zendesk or Slack, as there is a two-way sync. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,7 +34,7 @@ Reach out before customers ask, even if it’s just to say, “We’re aware and | |
|
|
||
| | Level | Description | Examples | Channels | Cadence | | ||
| | ----- | ----- | ----- | ----- | ----- | | ||
| | **SEV 0 – Emergency** | Existential service failure; all or most customers impacted with no workaround. CMOC sends immediately via broadcast. Account owners do **not** gate comms. | Full platform outage, data loss, security breach with active customer impact. | Pylon broadcast → Email → DM/SMS | Immediate broadcast, then every 15–30 min; postmortem within 24h | | ||
| | **SEV 0 – Emergency** | Existential service failure; all or most customers impacted with no workaround. CMOC sends immediately via broadcast. Account owners do **not** gate comms. | Full platform outage, data loss, security breach with active customer impact. | Slack broadcast → Email → DM/SMS | Immediate broadcast, then every 15–30 min; postmortem within 24h | | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think there's a broadcast
simfish85 marked this conversation as resolved.
Outdated
|
||
| | **SEV 1 – Critical** | Major outage or data loss; widespread impact. | API unavailable, ingestion halted, login failures. | Slack → Email → (DM or SMS if needed) | Every 30–60 min; postmortem within 48h | | ||
| | **SEV 2 – Major** | Partial degradation or downtime; workaround available. | Replay or query delays \>30 min, flag evaluation slow. | Slack or Email | Every 1–2 hrs | | ||
| | **SEV 3 – Minor** | Limited impact or slow recovery. | Billing sync delays, isolated org issues. | Slack | Start and close | | ||
|
|
@@ -44,9 +44,9 @@ Reach out before customers ask, even if it’s just to say, “We’re aware and | |
|
|
||
| ### **Emergency (SEV 0)** | ||
|
|
||
| > **This overrides the standard workflow.** CMOC sends directly to all affected customer channels via Pylon broadcast without waiting for account owners. Account owners follow up individually once online. | ||
| > **This overrides the standard workflow.** CMOC sends directly to all affected Slack customer channels without waiting for account owners. Account owners follow up individually once online. | ||
|
|
||
| **Initial broadcast (Pylon):** | ||
| **Initial broadcast (SupportHog):** | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think there's a broadcast
simfish85 marked this conversation as resolved.
Outdated
|
||
|
|
||
| We're investigating a major incident affecting [feature/service]. [Symptom — e.g., "Event ingestion is fully stopped" or "The PostHog app is unreachable."] | ||
|
|
||
|
|
@@ -143,7 +143,7 @@ Heads up — maintenance on \[system\] from \[time window\]. No downtime expecte | |
|
|
||
| ### **Workflow** | ||
|
|
||
| > **SEV 0 override:** For SEV 0 incidents, the CMOC skips steps 4–5 and sends the initial message directly via [Pylon broadcast](#using-pylon-for-broadcasts) to all affected customer channels. Account owners are notified in [#group-cs-sales-support](https://posthog.slack.com/archives/C090RCG671C) simultaneously, and take over individual follow-up threads once online. The CMOC continues to own broadcast updates until the incident is resolved or downgraded. | ||
| > **SEV 0 override:** For SEV 0 incidents, the CMOC skips steps 4–5 and sends the initial message directly via Slack broadcast to all affected customer channels. Account owners are notified in [#group-cs-sales-support](https://posthog.slack.com/archives/C090RCG671C) simultaneously, and take over individual follow-up threads once online. The CMOC continues to own broadcast updates until the incident is resolved or downgraded. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think I understand enough about what this is and if it still works that way
simfish85 marked this conversation as resolved.
Outdated
|
||
| 1. **Incident declared** (Engineering). | ||
| 2. **CMOC activated**, notified of impact. | ||
| 3. **Assess customer impact**, this <PrivateLink url="https://us.posthog.com/project/2/insights/EBiXOD91">insight</PrivateLink> (or this <PrivateLink url="https://docs.google.com/spreadsheets/d/1EyV55L0vWTfD4W02j5A3Vx3YZYPUODmQQC1toFPcKXU/edit?gid=1396499197#gid=1396499197">Google Sheet</PrivateLink> as a backup) will help you understand which customers are using which components in which cloud environment. | ||
|
|
@@ -163,20 +163,3 @@ Heads up — maintenance on \[system\] from \[time window\]. No downtime expecte | |
| 4. **Account Owner sends the message** to their customers. Example outbound: “We’re investigating an outage affecting event ingestion. Updates every 30 minutes.” | ||
| 5. **During:** “Root cause identified (Redis queue saturation). Fix in progress.” | ||
| 6. **Resolution:** “Resolved at 11:42 UTC. Write-up soon.” | ||
|
|
||
| ## Using Pylon for broadcasts | ||
|
|
||
| It's best that communications are shared directly from the account owner; however, if speed is of the essence, i.e. for a SEV 1 or security issue, and some folks are not yet working or on PTO, the CMOC can use Pylon to send a broadcast to all customer Slack channels en masse: | ||
|
|
||
| 1. Log into app.usepylon.com with your PostHog Slack account. | ||
| 2. Go to the _Broadcasts_ link on the left-hand side of the navigation. | ||
| 3. Click _Create Broadcast_ in the top right-hand corner of the UI. | ||
| 4. Enter the message you want to send, ensuring the formatting looks correct in the preview on the right-hand side. | ||
| 5. Ensure the Send as option is set to PostHog, not your own user (unless you want to handle 450+ potential separate threads) | ||
| 6. Click _Next_ in the top right-hand corner of the UI. | ||
| 7. Select your audience. You can use the filters to select all channels not owned by specific people, e.g., those who are currently online and communicating 1:1 with customers. | ||
| 8. Make sure you click _Add to Audience_ to add the selected channels to the broadcast. | ||
| 9. Click _Next_ in the top right-hand corner of the UI. | ||
| 10. Set the engagement notification channel to be [#support-customer-success](https://posthog.slack.com/archives/C05MUMZLC13) | ||
| 11. Check that you're happy with the message and audience and click _Send Now_ in the top right of the UI. | ||
| 12. Ask everyone online to monitor [#support-customer-success](https://posthog.slack.com/archives/C05MUMZLC13) for replies and respond where necessary. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,7 @@ As a general principle, we try to make sure as much customer communication as po | |
|
|
||
| For existing customers, you'll sometimes send emails directly from <PrivateLink url="https://posthog.vitally-eu.io/">Vitally</PrivateLink>. To ensure these also make it to Salesforce, first look up your _Email to Salesforce Address_ from the [personal settings page](https://posthog.lightning.force.com/lightning/settings/personal/EmailToSalesforceUserSetup/home) in Salesforce, and then add it to your <PrivateLink url="https://posthog.vitally-eu.io/settings/profile/gmail">Vitally gmail settings</PrivateLink>. | ||
|
|
||
| All Slack messages sync up with the corresponding account in Salesforce. We use [Pylon](https://app.usepylon.com) for this sync, so make sure Pylon is added to the customer Slack channel integrations and the channel is [linked to the Salesforce account](https://app.usepylon.com/integrations/salesforce?tab=account-mapping) properly for the sync to work smoothly. | ||
| All Slack messages sync up with the corresponding account in Salesforce. We use [SupportHog](/handbook/growth/sales/slack-channels) for this sync, so make sure SupportHog is [added to the customer Slack channel](/handbook/growth/sales/slack-channels) and the channel's name is recorded on the relevant Salesforce account record for the sync to work smoothly. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this is how this works.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @tjcran can you help with how Slack <> Vitally/Salesforce/PostHog sync works at the moment?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @abhischekt can assist here too
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hello! I'm already working on this in: PostHog/posthog#66974 in this current PR I'm focused on the aggregate stats that pylon sent to SFDC per: https://github.com/PostHog/posthog/pull/66974/changes#diff-ea9e087a5e10f2ff92bf65e3842a9098565c38709f69346877978128319fe5c9R32 Once this is shipped and running I'll figure out how to get the actual |
||
|
|
||
| You are most likely to use the following regularly: | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't know how specific we want to be -
It's not that SupportHog links to Salesforce. I actually think that Abe was looking at syncing all conversations.
Maybe worth mentioning here that you can go to conversations and view tickets from your accounts by filtering on your own name tag (will be ae_emaillocalpart or csm_emailloalpart)