handle rate limit and implement retry logic

[smaeda-ks]

Adding rate limit handling and retry logic.

Usage

• rate-limit

CONSUMER_KEY = 'your consumer key'
CONSUMER_SECRET = 'your consumer secret'
ACCESS_TOKEN = 'access token'
ACCESS_TOKEN_SECRET = 'access token secret'
ACCOUNT_ID = 'account id'

client = Client(
    CONSUMER_KEY,
    CONSUMER_SECRET,
    ACCESS_TOKEN,
    ACCESS_TOKEN_SECRET,
    options={
        'handle_rate_limit': True  # default is False
    })

account = Account.load(client, ACCOUNT_ID)
campaigns = Campaign.all(account)

• retry

client = Client(
    CONSUMER_KEY,
    CONSUMER_SECRET,
    ACCESS_TOKEN,
    ACCESS_TOKEN_SECRET,
    options={
        'retry_max': 3,  # default is 0
        'retry_delay': 5000,  # milliseconds (default is 1500)
        'retry_on_status': [404, 500, 503]  # status codes you want to retry on
    })

account = Account.load(client, ACCOUNT_ID)
campaigns = Campaign.all(account)

• combine

client = Client(
    CONSUMER_KEY,
    CONSUMER_SECRET,
    ACCESS_TOKEN,
    ACCESS_TOKEN_SECRET,
    options={
        'handle_rate_limit': True,  # default is False
        'retry_max': 3,  # default is 0
        'retry_delay': 5000,  # milliseconds (default is 1500)
        'retry_on_status': [404, 500, 503]  # status codes you want to retry on
    })

account = Account.load(client, ACCOUNT_ID)
campaigns = Campaign.all(account)

Test with actual requests

• code

client = Client(
    CONSUMER_KEY,
    CONSUMER_SECRET,
    ACCESS_TOKEN,
    ACCESS_TOKEN_SECRET,
    options={
        'handle_rate_limit': True,
        'retry_max': 3,
        'retry_delay': 5000,
        'retry_on_status': [404, 500, 503]
    })

account = client.accounts(ACCOUNT_ID)

data = TargetingCriteria.locations(account, count=1)

for i in data:
    print(i['targeting_value'])
    print("rate_limit_remaining: {}".format(data.rate_limit_remaining))

• log

...
rate_limit_remaining: 5
c937f4e8dee2cd30
rate_limit_remaining: 4
2a477551c9b5ffa9
rate_limit_remaining: 3
2cfd899c9c6c548c
rate_limit_remaining: 2
15d53cbe4ca76f3d
rate_limit_remaining: 1
6fa38f9c8283c034
rate_limit_remaining: 0
Request reached Rate Limit: resume in 806 seconds

10c4b91189877846
rate_limit_remaining: 399
f6069e863e29e322
rate_limit_remaining: 398
84e681a9fb76656f
rate_limit_remaining: 397
03ee27c08cf99bea
rate_limit_remaining: 396
f39bb92424252cee
rate_limit_remaining: 395
8f88713a5e8554ff

[smaeda-ks]

monkeypatch to stub time.sleep() in http.py

[tushdante]

LGTM!