vibezuloo.blogg.se

Sitesucker retry 404 errors
Sitesucker retry 404 errors












sitesucker retry 404 errors

#Sitesucker retry 404 errors code

Generalizingįor HTTP status code 500 in general, there is nothing inherently retryable about it. Lambda) and whether the service considers that error code retryable. In the AWS case, the language-specific SDKs have logic which decides whether or not to retry, not based on the HTTP status code, but based on the granular error code provided by the specific service ( e.g. Some of them are obviously retryable, and some of them are obviously not retryable, just by looking at the description. For example, the AWS Lambda Invoke operation has a detailed list of error messages. For example, some AWS services include a response header called x-amzn-ErrorType which classifies the error at a more detailed level than the HTTP status code can provide. Most web services document how their API works at a more granular level than HTTP status codes. IMO the correct answer is read the API documentation, try to retry on errors specificially indicated as retryable, and don't blindly retry on generic 500 until you've weighed the pros and cons. Though these calls can fail as well, at least you may be able to lower the failure rate to an acceptable degree by using these features.Īs a general engineering principle: if you have an unreliable component in your system, create enough redundancy to make the system "reliable enough". Or maybe the API itself will allow you to inspect a certain status or logging information for certain operations. Of course, the blind copy may not reach your system in time because of a network lag, but you can reduce the frequency of duplicate email operations through the API.

sitesucker retry 404 errors

That gives you an opportunity to implement an additional checkpoint if the original email was sent or not. Let us take the email example: if the API allows you to send email requests with an included blind-copy request, you may utilize this feature to send your own system such a blind copy. This is not a technical design decision, it depends on the specific operation and the consequences of such failures in the domain.Īs a technical measure, you could check if there are additional options for validating if a certain operation has happened (at least with a certain probability). This boils down to what is worse for the specific non-idempotent operation: I've never had to add an retry logic to API calls as getting 500 errors is very slim. When the operation failing is worse than the operation succeeding 10 times (or whatever the retry limit is).When we are okay with the operation running more than once.My two thoughts on when we should add retry: Should we actually be adding retry logic for 500 errors that are not idempotent? If I retry the API call 10 times, I might get 10 500 errors, but the recipient might still get 10 emails. We've noticed at times that the API might return a 500 error, even though the email was sent. For example, one API call might be to send an Email. However, for non-idempotent operations it seems like it could be dangerous. For idempotent operations, I could see this being useful. The 3rd party told us that we should implement retry logic when receiving a 500 error. We experienced this yesterday where 90% of the time the API call would fail. It usually lasts for about 4 hours and occurs every few months. After ~10 attempts, it will finally work.

sitesucker retry 404 errors

Occasionally, they will have times where the API call fails with a 500 error more than 90% of the time. Maybe 0.1% of the time an API call just fails with a 500 error, and then when you try again, it works fine. My team and I are integrating with a 3rd party company and using their API to perform different CRUD operations.














Sitesucker retry 404 errors