Retry in Python requests
Retry in Python Requests relates to automatically resubmitting unsuccessful HTTP requests. There are several reasons why a request made over a network can fail, including network issues, server failures, and rate limitations. Retrying unsuccessful requests can help you increase the robustness and dependability of your code as well as prevent interruptions or downtime brought on by temporary faults.
Python requests is a trendy Python package that makes your life easy by giving you APIs to make HTTP requests quickly. Not only that but also it provides a way to implement retry a specified number of times whenever your requests fail.
A built-in retry feature of the Python Requests module enables you to set up automated retries for unsuccessful requests. The maximum number of retries, the time delay between retries, and the HTTP status codes that should be retried can all be specified. With each unsuccessful attempt, the duration between retries in this feature’s exponential backoff mechanism rises exponentially.
What is the backoff factor in Python requests?
Assume you want to retry HTTP request a maximum of 5 times on failure with a backoff factor of 3s. Once the initial request is failed, the Python request will trigger retries after each following second. The first retry will start just after the initial request is get failed.
- First retry → After 3*0 → 0s,
- Second retry → After 3*2 → 6s
- Third retry → After 3*4 → 12s
- Fourth retry → After 3*8 → 24s
- Fifth retry → After 3*16 → 48S
If the request is still failed, requests will throw an error after 0s+6s+12s+24s+48s → 90s
What are connect retries in Python requests?
Connect retry specifies the number of retries on failure that could happen while marketing the connection to the server.
What are read retries in Python requests?
Read retry specifies the number of retries on failure that could happen while reading data from the server.
Python requests Retry examples
Here we are using a publicly available API endpoint. This API will return user details as JSON. Also, We are using an in-built Retry class in the Python requests library to trigger retry requests automatically. In Each example, we are feeding the number of total retries, the number of connecting retries, the number of read retries, and the backoff factor.
GET request retry
# retry on get
import json
import requests
from requests.adapters import HTTPAdapter, Retry
REQUEST_RETRY = 3
BACKOFF_FACTOR = 3
def get():
requests.packages.urllib3.disable_warnings()
try:
url = "https://gorest.co.in/public/v2/users"
headers = {
"Content-Type": "application/json",
"Accept": "application/json"
}
with requests.Session() as s:
retries = Retry(
total=REQUEST_RETRY,
backoff_factor=BACKOFF_FACTOR,
connect=REQUEST_RETRY,
read=REQUEST_RETRY
)
s.mount('http://', HTTPAdapter(max_retries=retries))
s.mount('https://', HTTPAdapter(max_retries=retries))
res = s.get(url, headers=headers, verify=False)
if res.status_code != 200:
raise Exception(
"Request {} failed, status: {}".format(
url, res.status_code
)
)
json_res = res.json()
print(json.dumps(json_res, indent=4))
except Exception as e:
raise Exception("Error while request: {}".format(e))
if __name__ == "__main__":
get()
POST request Retry
# retry on post
import json
import requests
from requests.adapters import HTTPAdapter, Retry
REQUEST_RETRY = 3
BACKOFF_FACTOR = 3
def post():
requests.packages.urllib3.disable_warnings()
try:
url = "https://gorest.co.in/public/v2/users"
headers = {
"Content-Type": "application/json",
"Authorization": "Bearer <ACCESS-TOKEN>",
"Accept": "application/json"
}
payload = {
"name": "Tenali Ramakrishna",
"gender": "male",
"email": "[email protected]",
"status": "active"
}
with requests.Session() as s:
retries = Retry(
total=REQUEST_RETRY,
backoff_factor=BACKOFF_FACTOR,
connect=REQUEST_RETRY,
read=REQUEST_RETRY
)
s.mount('http://', HTTPAdapter(max_retries=retries))
s.mount('https://', HTTPAdapter(max_retries=retries))
res = s.post(url, headers=headers, json=payload, verify=False)
if res.status_code != 200:
raise Exception(
"Request {} failed, status: {}".format(
url, res.status_code
)
)
json_res = res.json()
print(json.dumps(json_res, indent=4))
except Exception as e:
raise Exception("Error while request: {}".format(e))
if __name__ == "__main__":
post()
PUT request Retry
# retry on put
import json
import requests
from requests.adapters import HTTPAdapter, Retry
REQUEST_RETRY = 3
BACKOFF_FACTOR = 3
def put():
requests.packages.urllib3.disable_warnings()
try:
url = "https://gorest.co.in/public/v2/users/547"
headers = {
"Content-Type": "application/json",
"Authorization": "Bearer <ACCESS-TOKEN>",
"Accept": "application/json"
}
payload = {
"name": "Tenali Ramakrishna",
"gender": "male",
"email": "[email protected]",
"status": "active"
}
with requests.Session() as s:
retries = Retry(
total=REQUEST_RETRY,
backoff_factor=BACKOFF_FACTOR,
connect=REQUEST_RETRY,
read=REQUEST_RETRY
)
s.mount('http://', HTTPAdapter(max_retries=retries))
s.mount('https://', HTTPAdapter(max_retries=retries))
res = s.put(url, headers=headers, json=payload, verify=False)
if res.status_code != 200:
raise Exception(
"Request {} failed, status: {}".format(
url, res.status_code
)
)
json_res = res.json()
print(json.dumps(json_res, indent=4))
except Exception as e:
raise Exception("Error while request: {}".format(e))
if __name__ == "__main__":
put()
DELETE request Retry
#retry on delete
import json
import requests
from requests.adapters import HTTPAdapter, Retry
REQUEST_RETRY = 3
BACKOFF_FACTOR = 3
def delete():
requests.packages.urllib3.disable_warnings()
try:
url = "https://gorest.co.in/public/v2/users/547"
headers = {
"Content-Type": "application/json",
"Authorization": "Bearer <ACCESS-TOKEN>",
"Accept": "application/json"
}
with requests.Session() as s:
retries = Retry(
total=REQUEST_RETRY,
backoff_factor=BACKOFF_FACTOR,
connect=REQUEST_RETRY,
read=REQUEST_RETRY
)
s.mount('http://', HTTPAdapter(max_retries=retries))
s.mount('https://', HTTPAdapter(max_retries=retries))
res = s.delete(url, headers=headers, verify=False)
if res.status_code != 200:
raise Exception(
"Request {} failed, status: {}".format(
url, res.status_code
)
)
json_res = res.json()
print(json.dumps(json_res, indent=4))
except Exception as e:
raise Exception("Error while request: {}".format(e))
if __name__ == "__main__":
delete()
Conclusion
Python requests library is a fast way to make an API request. Also, its retry feature is a great way to trigger API requests again and again automatically on failures.
Comments
There are no comments yet.