Announcement

Collapse
No announcement yet.

Trying to get odds data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Trying to get odds data

    Hi, i can't scrap odds data from betfair's website. I'm using PHP. I'm getting this error all the time:

    Error:
    Code:
    response: {"faultcode":"Client","faultstring":"DSC-0008","detail":{}}

    Could anyone tell me what's wrong with my code?

    Code:
    $url = "https://www.betfair.com/www/sports/n...wYMrh&alt=json";
    
    $ch = curl_init();
    
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array(
        #'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36',
        'Accept: application/json',
        'Content-Type: application/json',
        'Referer: https://www.betfair.com/exchange/plus/football'));
    
    $postData = '[{ "params": {"filter": {}} }]';
    
    
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
    
    # execute
    
    $output = curl_exec($ch);
    curl_close($ch);

  • #2
    When my prog sends curls, everything I want to be considered a string on receipt at the destination has double quotes round it. If you could show us the final curl string you send, that would make things massively easier.

    I've never seen any code on here from anyone attempting to scrape data from Betfair. Everyone else has been using an API.

    Comment


    • #3
      Hi jabe, thank you for your response. Here is my final curl string:

      Code:
      final curl string: Array
      (
      [url] => https://www.betfair.com/www/sports/navigation/facet/v1/search?_ak=nzIFcwyWhrlwYMrh&alt=json
      [content_type] =>
      [http_code] => 0
      [header_size] => 0
      [request_size] => 0
      [filetime] => -1
      [ssl_verify_result] => 0
      [redirect_count] => 0
      [total_time] => 0
      [namelookup_time] => 0
      [connect_time] => 0
      [pretransfer_time] => 0
      [size_upload] => 0
      [size_download] => 0
      [speed_download] => 0
      [speed_upload] => 0
      [download_content_length] => -1
      [upload_content_length] => -1
      [starttransfer_time] => 0
      [redirect_time] => 0
      [redirect_url] =>
      [primary_ip] =>
      [certinfo] => Array
      (
      )
      
      [primary_port] => 0
      [local_ip] =>
      [local_port] => 0
      [http_version] => 0
      [protocol] => 0
      [ssl_verifyresult] => 0
      [scheme] =>
      [appconnect_time_us] => 0
      [connect_time_us] => 0
      [namelookup_time_us] => 0
      [pretransfer_time_us] => 0
      [redirect_time_us] => 0
      [starttransfer_time_us] => 0
      [total_time_us] => 0
      )
      
      http status: 400
      
      response: {"faultcode":"Client","faultstring":"DSC-0008","detail":{}}
      
      type of response: string
      Process finished with exit code 0
      I was inspired by this solution: https://www.betangel.com/forum/viewtopic.php?t=18742
      Last edited by elo; 20-05-2020, 07:22 PM.

      Comment


      • #4
        The DSC-0008 means there's some sort of syntax error in the code you're using to request data.

        It's been a long time since I used PHP. Could you show me what's in these?
        $ch, CURLOPT_POSTFIELDS, $postData

        Comment


        • #5
          Hi jabe, all my code is in first post. Could you precise please what should i show you?

          Comment


          • #6
            Ah, okay. The JSON coding isn't right. Everything that's a string needs double quotes round it.

            Where you've got a string containing

            Accept: application/json

            It needs to be

            "Accept": "application/json"

            The name of the parameter has to be in quotes. The parameter value can be without quotes if it's numeric (but it'll be accepted with quotes in most cases). Always wrap marketIds in quotes; they look numeric, but they're not, and if you omit quotes, you could lose a final zero.


            These might help with JSON:

            https://www.w3schools.com/js/js_json_intro.asp

            https://jsonlint.com/?code=

            Comment


            • #7
              Thank you jabe. I changed format of my request string but now I'm getting this kind of error:

              Code:
              response: {"faultcode":"Server","faultstring":"DSC-0002","detail":{}}
              I couldn't find what that error means on the official betfair api website. I've just found that it means "FrameworkError" on some page on the github (https://github.com/betfair/cougar-do...t_Reporting.md).

              Except that "DSC-0002" i'm getting HTTP 500 eror. Sorry for many edits.


              Here is the modified request:

              Code:
              
              $url = "https://www.betfair.com/www/sports/navigation/facet/v1/search?_ak=nzIFcwyWhrlwYMrh&alt=json";
              $ch = curl_init();
              curl_setopt($ch, CURLOPT_URL, $url);
              curl_setopt($ch, CURLOPT_POST, 1);
              curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
              curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
              curl_setopt($ch, CURLINFO_HEADER_OUT, true);
              curl_setopt($ch, CURLOPT_HTTPHEADER, array(
              '"Accept":"application/json",
              "Content-Type":"application/json",
              "Referer":"https://www.betfair.com/exchange/plus/football"' ));
              
              $postData = '[{"filter": {} ]';

              Here is the response:

              Code:
              final curl string: Array
              (
              [url] => https://www.betfair.com/www/sports/navigation/facet/v1/search?_ak=nzIFcwyWhrlwYMrh&alt=json
              [content_type] =>
              [http_code] => 0
              [header_size] => 0
              [request_size] => 0
              [filetime] => -1
              [ssl_verify_result] => 0
              [redirect_count] => 0
              [total_time] => 0
              [namelookup_time] => 0
              [connect_time] => 0
              [pretransfer_time] => 0
              [size_upload] => 0
              [size_download] => 0
              [speed_download] => 0
              [speed_upload] => 0
              [download_content_length] => -1
              [upload_content_length] => -1
              [starttransfer_time] => 0
              [redirect_time] => 0
              [redirect_url] =>
              [primary_ip] =>
              [certinfo] => Array
              (
              )
              
              [primary_port] => 0
              [local_ip] =>
              [local_port] => 0
              [http_version] => 0
              [protocol] => 0
              [ssl_verifyresult] => 0
              [scheme] =>
              [appconnect_time_us] => 0
              [connect_time_us] => 0
              [namelookup_time_us] => 0
              [pretransfer_time_us] => 0
              [redirect_time_us] => 0
              [starttransfer_time_us] => 0
              [total_time_us] => 0
              )
              
              http status: 500
              
              response: {"faultcode":"Server","faultstring":"DSC-0002","detail":{}}
              
              type of response: string
              Process finished with exit code 0
              Last edited by elo; 21-05-2020, 04:39 PM.

              Comment


              • #8
                I can't find a description for DSC-0002, but an http error 500 means something is wrong at the web
                Last edited by jabe; 23-05-2020, 06:34 PM.

                Comment


                • #9
                  After i added "curl_setopt($ch,CURLOPT_HEADER, true);" option to my header, I'm getting additional info about error. I've used Google Chrome developer tools to check how looks request header. Maybe link to send post request is wrong? I mean:
                  Code:
                  https://www.betfair.com/www/sports/navigation/facet/v1/search?_ak=nzIFcwyWhrlwYMrh&alt=json
                  Code:
                  HTTP/1.1 500 Server Error
                  Date: Fri, 22 May 2020 12:49:25 GMT
                  Cache-Control: no-cache
                  Content-Type: application/json;charset=utf-8
                  Content-Length: 59
                  Content-Security-Policy-Report-Only: default-src 'unsafe-inline' 'unsafe-eval' data: https: wss:; report-uri https://shk.betfair.com/csp
                  X-XSS-Protection: 1; mode=block
                  
                  {"faultcode":"Server","faultstring":"DSC-0002","detail":{}}
                  Last edited by elo; 22-05-2020, 06:38 PM.

                  Comment


                  • #10
                    We do have another problem; I understand what the ultimate purpose of your code is, but I'm not familiar with the method or how it works.

                    Comment


                    • #11
                      Ok, I understand. Thank you for your help jabe. Is it even possible to scrap odds data from Betfair without using API? As I mentioned above, I've used some info from this thread: https://www.betangel.com/forum/viewtopic.php?t=18742. That guy didn't use API and probably, as he wrote, he succeeded in scrap odds.
                      Last edited by elo; 22-05-2020, 09:39 PM.

                      Comment


                      • #12
                        I don't know, I'm afraid. I think the person who posted originally on the BetAngel site made a typo - I'm pretty sure taking data from websites in this manner is called screen scraping, so he should have written scrape rather than scrap. However, from the links you provided, it appears that the software you're using was written by Betfair themselves, so it should be usable.

                        In the past I have "borrowed" data from websites by downloading an entire webpage and saving it as a file so that I could later write a program to read each file and extract the specific data that I wanted from it. (I used PHP, as it was able to read a webpage into a string, so I could then save it as a file. This was around 20 years ago, however!)

                        I've looked at the links you provided. Perhaps it's something I should be taking a closer look at myself. Unfortunately things are very busy at present and are likely to remain that way, so I don't know when I'd be able to find the time to do it.

                        If I find time to try it out for myself and can get it to work, I'll return to this page and add whatever information seems important. In the meantime, I wish you luck.

                        Comment


                        • #13
                          Thank you again jabe! You are very helpful! I will try "to fight" with this 500 error and I will wait for your so valuable information

                          Comment


                          • #14
                            Originally posted by elo View Post
                            Thank you again jabe! You are very helpful! I will try "to fight" with this 500 error and I will wait for your so valuable information
                            You are trying to access not public web api of betfair web site, what could be tolerated to some level.

                            You should use betfair api.

                            What data are you actually trying to parse from betfair web site?
                            Betfair BOT SDK

                            Comment


                            • #15
                              Originally posted by bfexplorer View Post

                              You are trying to access not public web api of betfair web site, what could be tolerated to some level.

                              You should use betfair api.

                              What data are you actually trying to parse from betfair web site?
                              But using something called Cougar, which was written by Betfair.

                              Comment

                              Working...
                              X