Parsing JSON data

[strangecarr]

Hi there,

I've managed to download bz2 files of historical data. I'm having a little trouble extracting the data into JSON format - I can get the data into string format using a python script, but can't convert the string into anything useful at the moment (using json.loads() on the data raises an error). Does anyone have any advice?

[strangecarr]

Managed to fix it so now have the data in JSON. Can anyone tell me how to relate the published time to a date_timestamp? The spec says published time is milliseconds since epoch. What does this mean?

[andrap]

its milliseconds since the UNIX epoch (January 1, 1970 00:00:00 UTC). In c# the converter would looklike something similar yourDateTime.ToUniversalTime().Subtract( new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc) ).TotalMilliseconds

[LiamP]

Managed to fix it so now have the data in JSON. Can anyone tell me how to relate the published time to a date_timestamp? The spec says published time is milliseconds since epoch. What does this mean?

Let betfairlightweight do the hard work for you:

https://github.com/liampauling/betfair

https://github.com/liampauling/betfa...ghistorical.py

[strangecarr]

Hi Liam thanks a lot, I've been using your module on a few other scripts and have only just got to the historical data part. It really is a great module.

On the streaming historical data script, should the directory contain the bz2 files, or should they be unpacked first? I haven't managed to get the script working just yet (hence looking at the files directly myself to try to understand what data is contained in them).

[LiamP]

You need to unzip them first but you can use a python script for that as well.

[strangecarr]

Thanks, I unzipped using bzip2 in terminal. But the bz2 files are just numbered with no file extension (e.g. 1.12345678) - I can read them using a text editor but doesn't seem to work on betfairlightweight. Sorry if quite a simple question, what format should they be in for betfairlightweight to read them using the historical streaming script?

[yiyerre]

They must first be unzipped, although you can also do it with a Python script.



happy wheels unblocked