Add exponential backoff and retry when hitting API throttling
Also, process feeds in random order, to avoid hitting the same provider in quick succession. With all those measures we can now successfully query 568 of the 657 feeds from the NABSA systems.csv, about 120 more than before.
parent
7653d8cd
Please register or sign in to comment