Tenacity: a retrying library for python
jd/tenacity: Retrying library for Python1:
from tenacity import (
retry,
stop_after_attempt,
wait_exponential,
before_sleep_log,
)
# ...
@retry(
stop=stop_after_attempt(10), # Maximum number of retries
wait=wait_exponential(multiplier=1, min=1, max=60), # Exponential backoff
before_sleep=before_sleep_log(logger, logging.INFO),
)
@staticmethod
def do_basic_uri_ops_when_crawling(
# ...
pass
Related: 231207-1529 requests and urllib3 exceptions adventures
This is much better than the various retrying mechanisms in requests (e.g. needing session adapters: Handling Retries in Python Requests – Majornetwork), and likely better than most reinvented wheels (231206-1722 Overengineered solution to retrying and exceptions in python).
Nel mezzo del deserto posso dire tutto quello che voglio.