Announcement

Collapse
No announcement yet.

Parsing JSON data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Parsing JSON data

    Hi there,

    I've managed to download bz2 files of historical data. I'm having a little trouble extracting the data into JSON format - I can get the data into string format using a python script, but can't convert the string into anything useful at the moment (using json.loads() on the data raises an error). Does anyone have any advice?

  • #2
    Managed to fix it so now have the data in JSON. Can anyone tell me how to relate the published time to a date_timestamp? The spec says published time is milliseconds since epoch. What does this mean?

    Comment


    • #3

      its milliseconds since the UNIX epoch (January 1, 1970 00:00:00 UTC). In c# the converter would looklike something similar yourDateTime.ToUniversalTime().Subtract( new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc) ).TotalMilliseconds

      Comment


      • #4
        Originally posted by strangecarr View Post
        Managed to fix it so now have the data in JSON. Can anyone tell me how to relate the published time to a date_timestamp? The spec says published time is milliseconds since epoch. What does this mean?
        Let betfairlightweight do the hard work for you:

        https://github.com/liampauling/betfair

        https://github.com/liampauling/betfa...ghistorical.py

        Comment


        • #5
          Hi Liam thanks a lot, I've been using your module on a few other scripts and have only just got to the historical data part. It really is a great module.

          On the streaming historical data script, should the directory contain the bz2 files, or should they be unpacked first? I haven't managed to get the script working just yet (hence looking at the files directly myself to try to understand what data is contained in them).

          Comment


          • #6
            You need to unzip them first but you can use a python script for that as well.

            Comment


            • #7
              Thanks, I unzipped using bzip2 in terminal. But the bz2 files are just numbered with no file extension (e.g. 1.12345678) - I can read them using a text editor but doesn't seem to work on betfairlightweight. Sorry if quite a simple question, what format should they be in for betfairlightweight to read them using the historical streaming script?

              Comment


              • #8
                They must first be unzipped, although you can also do it with a Python script.



                happy wheels unblocked

                Comment

                Working...
                X