SEMRUSH crawl returns 403 errors on Zendesk links

Hey @roberto15, this is actually related to a bug that we're tracking. Do you mind if I move this into a ticket?

Same problem here. How do we fix it?

Hello @eric27! Of course, please go ahead and if possible let me know when there is a fix available. We currently have a ton of errors just for that.

Thanks!

@eric27 - Hi there. Has there been any progress on this? I believe we are experiencing the same issue. 

Hi,

Did you get a fix for this?

Thanks

Wes

We have the exact issue.

Our Zendesk HC page also gets 403 error in semrush. But page loads fine when accessed.

Please let us know if there is a fix .

We got following info from Semrush too.

Thank you for contacting Semrush! 

We apologize for the delay as we have been experiencing high request volumes. 

We are getting a 403 status error based on their domain blocking our bots at this time. If our bots are blocked it will cause them to time out. This does not directly mean there is an error with the page, thusly why you may be seeing a 200 code on your end. 

Additionally information on whitelisting can be found within our Site Audit section of our knowledge base: 
https://www.semrush.com/kb/681-site-audit-troubleshooting. To recap how the domain reads or blocks our bot is why we specifically are retaining a 403 status code at this time. 

Please let us know if you have any additional questions or requests! 

Hello, we are having this exact same issue too. Can I get this created into a ticket as well?

Confirming this issue as well. This is not specific to semrush however, our google search console report also reports this which leads to impact on SEO. 

We're getting the same error. Any progress on a fix? I see at least from this thread that it was first reported on December 1, 2021 and today is May 20, 2022. 

Same issue here, any updates on the fix?

Thanks