Webhooks Failure

Hey there! While it is not possible today, these are problems that we are considering for future development. Some things we are interested in understanding is who should be notified of a failure (e.g. all admins, just the account owner etc.) and what types of failures are worth alerting on.

We would love to hear more about if you have any thoughts about what you would expect for your use cases.

Hello, I think an admin or end user can be alerted. On my side, we have technical admin that take in charge complex triggers, html format, routing, Webhooks, integration... and production admin that take care of views, workflow.

I think technical admins need to be alerted, and maybe a end user as webhhoks is used on our side to send information from Zendesk to other tools. This other tools admin can be put in Zendesk as end user to be alerted.

I think a message can be sent for each failure with the error code detected.

Weighing in here as I'm about to cross the webhook bridge too. 

Notifiable users should be definable on the individual webhook. It should not be a global setting as it's perfectly feasible for webhooks to be related to completely different departments. I'd suggest a free-text box allowing the creator to enter comma-separated email addresses (which also allows us to notify monitoring systems etc. )

I'm also interested to know how a fail webhook call affects subsequent triggers? 
What happens if I have a trigger with two actions: 1 to call a webhooks and 1 to apply a tag. If the webhook fails, is the tag applied? 

Thanks for the feedback. This definitely helps us understand what would be most valuable in terms of configuring notifications.

@nathan12 in terms of your question around webhook failures, the scenario you described should result in the tag being applied. This is because when a business rule fires a webhook, it queues the webhook job to be run but doesn't actually run the job immediately. After queueing the webhook job, the business rule (trigger) will continue processing.

Hope this helps!

We need this.

On multiple clients, we've had to mitigate this with temporary solutions using tagging logic to determine if the webhook was fired successfully or not. But it's not very scaleable. 

Upvoting this. I'm currently trying to determine whether my organization can even use webhooks, given that there's no way to be alerted of endpoint failures except for someone to check the activity log of every webhook every day, which is not sustainable. It would be especially useful if webhook failures after the standard 3 retries could be used as a trigger condition, so we can decide how we want to handle the failure...notify someone, log it, etc.

Voting for this feature too. My team would love to have alert of webhook because we realized the webhook failure when we checked by ourselves or someone noticed the behavior changing.

This would be incredibly helpful. We had a case recently where a webhook started to fail for several days as a result of a change on another system and we only became alerted to this when internal staff noticed a business critical function stopped working...

I see this functionality in Hubspot / Jira - Zendesk desperately need this.

We have several mission-critical integrations with Zendesk and this makes it hard to rely on when I get error notifications from every system except Zendesk. I'd like to be able to set up the notifications on a per-webhook basis. Some I don't need to be notified about, but others I need an email directly (or even better, the option to be notified via another method, like Slack). 

this is a must have to track the system automations uptime

Upvoting this as well. It’s hard to know it failed unless you’re actually in the webhooks screen refreshing. It is not feasible to check this on any cadence when you have 60+ webhooks like we do. I am sure there are customers with more.

It has become all the more important with the URL to webhooks migration.

Please provide this business critical feature to ensure the continued reliability of the support we provide our customers. Thanks.

@zach22  any admin should at least be able to subscribe to a notification about any specific webhook failure. I think this is a critical feature that there's no reasonable way around.

Subscription should be flexible too, via email, or slack or another external system through webhook

Totally agree, this is absolutely needed for large scale webhook use. 

I also voted for this because even some Webhooks failed through updates from ZD directly and we don't see any issue without checking it manually.

We recently wrote a python script for this exact purpose, it runs daily, consuming the WebHook API: https://developer.zendesk.com/api-reference/webhooks/webhooks-api/webhooks/

The script works in the following way:

1. Get all Webhooks: https://developer.zendesk.com/api-reference/webhooks/webhooks-api/webhooks/#list-webhooks
2. Loop over all Webhooks and get the invocations (each time it ran): https://developer.zendesk.com/api-reference/webhooks/webhooks-api/webhooks/#list-webhook-invocations
   1. Use filter[status] to filter on only the error statuses
   2. Use filter[from_ts] and filter[to_ts] to capture only the errors in a specific time range (i.e. since the script last ran till now)
3. Add each error as a message to then send in Slack in 1 go.

As the Invocation history last 7 days we run this on a daily basis and we can see the list of errors per WebHook