Pulling data from historical API frequently raises errors

**noatech** · 03-02-2023, 12:46 PM

Hi,

last line of your filter should also be in square brackets:

'fileTypeCollection': ['M']

It is "a collection" with two possible values E (events) and M (markets) so you should put it in an array of strings.

Also, there are some additional filters such as "EventId" and "CountriesCollection" for which I am not sure if they can be excluded like in your case.

Anyhow, this is a frequent error message when there is something wrong with file filter.

Hope this helps you.
BR

**davidmarco** · 08-08-2023, 10:56 AM

Originally posted by RobWH View Post

Hi,

I've been trying to pull historic basketball odds data using the historic data service for the time periods of 2020-01-01 -> 2020-01-31 (free data) and 2022-10-01 -> 2022-12-31 (paid data), but am frequently hitting various errors raised by the API when trying to retrieve the list of historic files (1) and then later download files from that list (2)

(1):
using `bf_api_client.historic.get_file_list` (where the api client is instantiated using `betfairlightweight.APIClient` I get frequent InvalidResponse and APIErrors. For an example of the parameters supplied to a query:

Code:

Params:
{'sport': 'Other Sports',
'plan': 'Advanced Plan',
'fromDay': 1,
'fromMonth': 1,
'fromYear': 2020,
'toDay': 1,
'toMonth': 3,
'toYear': 2020,
'eventName': 'Memphis Grizzlies',
'marketTypesCollection': ['COMBINED_TOTAL', 'HANDICAP', 'MATCH_ODDS', 'TOTAL_POINTS'],
'fileTypeCollection': 'M'}

For two examples of returned errors messages:

(edited down due to errors trying to post here)

Code:

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width" />
<title>ngErrorRedirect</title>
</head>
<body>
<div>
Error
</div>

and

Code:

Exception: HTTPSConnectionPool(host='historicdata.betfair.com', port=443): Read timed out. (read timeout=16)

My internet is otherwise stable (as tested by running a background ping to betfair.com) with roughly 50-100mbps download speed (easily enough to pull a list of files measuring a few kb at most).

(2) Similarly, if I try to pull a given file from the server using: drift hunters

Code:

bf_api_client.historic.download_file(file_path=file_name, store_directory=file_directory)

I tend to run into more readtimeout errors.

Code:

HTTPSConnectionPool(host='historicdata.betfair.com', port=443): Read timed out. (read timeout=16)

For both the above the error does not seem related to the query as retrying the same query upon error generally pulls request data on the second (and usually max third) request.

I saw the post https://forum.developer.betfair.com/...download-speed which seems to have similar issues. Unfortunately, our problems appear to be worse since the last post in that thread last Friday.

Please advise how to continue. Can share more details upon request though preferentially in private.

Thank you,

Make sure you are using a valid session token - this is required for accessing any historical data. The token needs to come from a prior login() call.

Pulling data from historical API frequently raises errors

Pulling data from historical API frequently raises errors

Comment

Comment