Search code examples
phppodiorate-limiting

Podio RateLimit on Fileserver

We're having an issue with the Podio Api or there is something, Podio hasn't documented.

We're making a request to the Api to get an Item with the function ->getItemById(). This item has file fields and attached files. To edit this files on our server we need the rawData of these files, so we're calling ->getFile() and ->getRawData().

It all works fine. After ~15-20 requests in about ~10 minutes (time and request count varies on every test) We're getting errors from Podio Server. We're getting errors even if we put short delays between the requests (tested to 30 seconds), so I think it's not a "batching request limit". The errors are with the number 420 which is described as RateLimit Error. After a few minutes waiting the error has gone and the Server is responding the correct answer. But we absolutely don't have any plan when this is going to happen.

We made a function to check if we're hitting the RateLimit but there are still remaining, both for the normal request and for the rate limited request. As you can see in the ScreenShots above.

Please don't answer with the link to the Podio Rate Limit Documentation. I read this section multiple times and I know how this limits are set. We're only having our problem when we are calling the file server (files.podio.com) and not when we're making other api requests.

Has anyone had a similar experience with our case?

Functions to call from the PHP Api:

/**
 *  get Item by Public ID (in url, starts with 1)
 */
public function getItemById($item_id) {
    try {
        return \PodioItem::get_by_app_item_id($this->app_id, $item_id );
    } catch (Exception $e) {
        throw new Exception($e->body['error_description']);
    }
}


public function getFile($file_id) {
    try {
        return \PodioFile::get( $file_id );
    } catch (Exception $e) {
        throw new Exception($e->body['error_description']);
    }
}

public function getRawData($file) {
    try {
        return $file->get_raw();
    } catch (Exception $e) {
        throw new Exception("Status: " . $e->status . " Message: " . (empty($e->body['error_description']) ? '-' : $e->body['error_description']));
    }
}

Call to the Podio Server with the RateLimit call:

$podio = new \Podio\Api\Api('APP_ID', 'APP_TOKEN');

for($i = 0; $i < 10; $i++) {

    echo "Rate Limit: " . $podio->getRateLimitRemaining() . '<br>';

    $item = $podio->getItemById(500);
    echo "Rate Limit: " . $podio->getRateLimitRemaining() . '<br>';

    $file = $podio->getFile(668548372);
    $raw_file = $podio->getRawData($file);

}

Browser Output. As you can see the RateLimit is not hit, but we're getting a 420 directly after calling getRawData()

enter image description here

Podio Support told me, they don't give support to private people despite of we are a company who uses podio on a daily basis and this is in my eyes a bug on their server (or it isn't documented anywhere).

Update:
I made a second function to trigger this down:

// This function is used to test Podio Fileserver for Rate Limits
public function podioAction() {

    $podio = new \Podio\Api\Api('APP_ID', 'APP_TOKEN');     

    for($i = 0; $i < 100; $i++) {

        try {

            $item = $podio->getItemById(500);
            echo date("H:i:s") . " Rate Limit Remaining: " . $podio->getRateLimitRemaining() . ', EndPoint URL: /app/{$app_id}/item/{$app_item_id} <br>';

            $podio->updateItem($item->item_id, array('titel-intern' => 'API Test'), array(), array('silent' => true));
            echo date("H:i:s") . " Rate Limit Remaining: " . $podio->getRateLimitRemaining() . ', EndPoint URL: /item/{$item_id} <br>';

            $file = $podio->getFile(668548372);
            echo date("H:i:s") . " Rate Limit Remaining: " . $podio->getRateLimitRemaining() . ', EndPoint URL: /file/{$file_id} <br>';

            $raw_file = $podio->getRawData($file);
            echo date("H:i:s") . " File received, EndPoint URL: https://files.podio.com/668548372 " . '<br>';

        } catch(Exception $e) {

            echo date("H:i:s") . " Error: " . $e->getMessage() . " <br>";
            exit;

        }

    }

    exit;

}

This gives me to following Browseroutput (the first few lines are not included):

15:46:38 Rate Limit Remaining: 4753, EndPoint URL: /app/{$app_id}/item/{$app_item_id} 
15:46:38 Rate Limit Remaining: 8, EndPoint URL: /item/{$item_id} 
15:46:38 Rate Limit Remaining: 4752, EndPoint URL: /file/{$file_id} 
15:46:38 File received, EndPoint URL: https://files.podio.com/668548372 
15:46:39 Rate Limit Remaining: 4751, EndPoint URL: /app/{$app_id}/item/{$app_item_id} 
15:46:39 Rate Limit Remaining: 6, EndPoint URL: /item/{$item_id} 
15:46:39 Rate Limit Remaining: 4750, EndPoint URL: /file/{$file_id} 
15:46:39 File received, EndPoint URL: https://files.podio.com/668548372 
15:46:39 Rate Limit Remaining: 4749, EndPoint URL: /app/{$app_id}/item/{$app_item_id} 
15:46:40 Rate Limit Remaining: 4, EndPoint URL: /item/{$item_id} 
15:46:40 Rate Limit Remaining: 4748, EndPoint URL: /file/{$file_id} 
15:46:40 File received, EndPoint URL: https://files.podio.com/668548372 
15:46:40 Rate Limit Remaining: 4747, EndPoint URL: /app/{$app_id}/item/{$app_item_id} 
15:46:40 Rate Limit Remaining: 2, EndPoint URL: /item/{$item_id} 
15:46:40 Rate Limit Remaining: 4746, EndPoint URL: /file/{$file_id} 
15:46:41 File received, EndPoint URL: https://files.podio.com/668548372 
15:46:41 Rate Limit Remaining: 4745, EndPoint URL: /app/{$app_id}/item/{$app_item_id} 
15:46:41 Rate Limit Remaining: 0, EndPoint URL: /item/{$item_id} 
15:46:41 Rate Limit Remaining: 4744, EndPoint URL: /file/{$file_id} 
15:46:41 Error: Status: 420 Message: -

Today in this test everything seems normal. It seems like the files are also RateLimited Requests and are throwing errors when these Limits are off. But in the Screenshot above I had a different result. If it is always like today, there is not a problem and I can track this. But this was not the case in the last few days.

I'm posting again if this happens as it was at the time asking this question, but for now everything seems to work fine.

Solution

  • ->getRawData() method is marked as internal and it is not listed on https://developers.podio.com/doc/files as available method. You are not supposed to use it :)

    Please use link attribute of file object from ->getFile() method. And send the regular Authorization header to that endpoint to authenticate for the download. Download is also subject to lower rate limit (as is ->getRawData() method)