Terrible Response Times (5300ms) :(

[FitzPattons]

As the title suggests, I am currently (and consistently) getting awful response times from the betfair server. I was hesitant to submit this to the forum since I don't see anyone else with a similar problem (which suggests that I'm doing something daft), but I am completely stumped so I am hoping someone can help.

Below is an example of the json-rpc request I'm sending:

	common_headers = {
		"X-Application": application_key_live,
		"X-Authentication":  session_token,
		"content-type": "application/json",
		"Accept-Encoding": "gzip"
	}

	mrktBook_req = '\
	{\
		"jsonrpc": "2.0",\
		"method": "SportsAPING/v1.0/listMarketBook",\
		"params": {\
			"marketIds": ["' + market_id + '"], \
			"priceProjection":{\
				"priceData": ["EX_BEST_OFFERS"]\
			}\
		},\
		"id": 1\
	}'

I've tried using my delayed app key, and my live app key; with and without gzip compression requested and connecting to this endpoint: https://api.betfair.com/exchange/betting/json-rpc/v1

I have sent hundreds of requests one after the other whilst timing the responses and I consistently get response times of ~5.30 seconds. I have tried on multiple machines, multiple operating systems, routers in my house and my python code is as trimmed down as can be (a function to log in and return a session code, which is then used to send further requests).

What's odd is that using the same machine, I get ~50ms response times from various other servers (which is a typical time to expect) and the ping and download speed of the internet at my house is great, so I can only narrow it down to communication with the betfair servers!!

The API is unusable at this speed, does anyone have any suggestion at all as to what it might be, I'm very much hoping that's it's something I can do on my end and not just the BF servers.

Thanks,
Pat

[Franklin1]

1 - how do you do with the API visualizer?
If that is slow:
2 - how do you do over a different internet connection (did you try your phone as a wifi hotspot)?

[geoffw123]

Hi

That all sounds pretty strange. The Betfair Server certainly isn't that slow, so the problem must lie elsewhere.

I suggest you try using Fiddler to capture your comms packets to and from the Server. http://www.telerik.com/download/fiddler

Its are very nice free App that shows packets and can decode/ display zipped JSON packets. It will have timestamps on the packets which will at least prove who the culprit is for your apparent slow response.

Geoff

[FitzPattons]

Thanks for the responses guys,
The response times I get from the visualizer are fine (fast), and after some more investigation using various other wifi points (thanks MacDonalds) the response times I get using the exact same code are the kind of speeds I expected, in the milliseconds. So it seems that there is something fishy going on with my router that is allowing normal browsing, and communication via http to servers to run fine, however connecting to the betfair servers over https to run a bit slower. I haven't tried using Fiddler yet as I'm on a Mac right now, so I am in the process of finding and configuring an alternative.

Thanks again for the responses and I will update the thread with what I find in case it helps anyone else in future (you never know).

Pat

[gus]

One thing you might try if you're currently using your router as a WiFi is to plug your pc (or Mac) directly into the router so you're using it just as a Modem.

I don't know a helluva lot about routers, but with my VirginMedia router I can just connect my pc to the router with an ethernet cable and then use the Router's web page control panel to switch from Router (WiFi) to Modem mode. Probably you can do something similar with yours.

In practice switching modes doesn't make much difference for me with the API... response times are good with either, but for some other programs (notably using ftp), in router (WiFi) mode i can't make any connection at all, whereas in Modem mode there's no problem, so there's obviously something in the router settings that makes a difference under certain circumstances or certain protocols.

[BFairUser]

same delay problem

I have the same problem
From yesterday evening (about 19:30) i have not normal delay.
theese are some of the results for normal call listMarketBook

06:49:06 0,031
06:49:06 0,343
06:49:07 0,328
06:49:09 0,234
06:49:09 0,312
06:49:10 0,406
06:49:12 0,936
06:49:12 0,593
06:49:13 0,515
06:49:14 0,53
06:49:15 0,312
06:49:16 0,374
06:49:18 0,016
06:49:18 0,39
06:49:20 0,874
06:49:21 0,624
06:49:21 0,39
06:49:23 0,109
06:49:23 0,328
06:49:25 0,733
06:49:26 0,546

I have just now contact Virgin Media (My Provider) and seems to be that the problem is not in my provider, seems to be that the data obtained are with various delay, from 0.01 to 1,1 seconds, with an average more than 0.6 seconds...
I have check with a normal vodafone sim connection(in hotspot mode), and put the results on a chart:
this is from yesterday about 19:30.

http://postimg.org/image/i4jfnrofl/

now the situation is the same, not normal delay.

P.S. The previous results are with 'clean' listMarketBook calls, with a normal call (all the runners datas), for only 1 market, these are the delays:
13:26:59 5,46
13:27:02 4,462
13:27:07 5,351
13:27:11 5,491
13:27:14 4,477
13:27:18 3,931
13:27:21 3,37

So an average of 4/5 seconds instead of few milliseconds.
Hope i can continue to bet.
If not, i will do something else.

P.P.S. Five minuts ago (14:11) for some minutes the situation was back to normality but now there is still delay...
Seems that the BDP team is trying to solve the problem...
Hope that soon they can find the solution...

[gus]

The only problem I've seen today, both from my Home (VirginMedia) and vps connections is that the Menu of events is very slow to load ( i use the gzipped Menu in my app):

https://api.developer.betfair.com/se...pageId=4392234

but the actual BF website seems to be having the same problem, so hopefully it will be fixed soon.

Other than that, I record all calls to listMarketBook that take more than 1 second, and my log shows only 7 such delays this afternoon (between about 13.00 and now (16.20).

[BFairUser]

Found the problem

I investigated the problem more accurately.
Although I called Virgin Media, and a 'technician' by phone have tried in every way to reset my connection, changing the frequencies ... ??? ...
The problem remained.
The Virgin Media technician has blamed the Betfair Server
But I run the same program at the same time with Virgin Media Connection and a dedicated Tagadab vps.
With the other connection everything worked normally (fast), while Virgin Media is always in abnormal delayed.
The problem is in the Virgin Media Connection.
Despite the 'technician' who answer the phone insist that the VirginMedia connection is ok.
The strange thing is that are three months that I use this broadband, and everything has always worked well, and suddenly there was such a drastic change, and the callCenter operator tell me that there is nothing wrong in my connection.
I think the call center operator mistake.

[gus]

That's interesting BFUser because your setup is almost identical to mine.

My vps is also Tagadab and my home Connection is VirginMedia (150mbps), so, like you, my tagadab connection is fine (average 28 - 32 millisecs for my API apps doing listMarketBook during InPlay horse races), but my VirginMedia connection is also ok usually average 45 - 65 millis for listMarketBook.

My VirginMedia connection was slightly slower today ... about 70-80 millis (for InPlay tennis matches), but that's not really outside the kind of range I'd expect, so it does look like a problem with your particular VirginMedia connection ... have you tried doing a Ping Plot to see where the delay is happening?

[BFairUser]

fixing the issue

Thanks Gus
i've try to take a look to the trace route of the connection.
Please, i'm not a technician and i've not much knowledge in matter, so if i mistake something, forgive me.
If you try to take a quick look to the ping to api.betfair.com, opening the dos prompt and typing 'ping api.betfair.com' you haven't results because betfair have disabled this functionality (probably for obvious reasons), but you can trace a route of your request, typing 'tracert api.betfair.com')
so you can quickly and easy see every step of your request, and you can see 'if' and 'where' there are issues.
You can also verify if and where the web request is readdressed
i've notice that in my connection, besides high delay, there are some strange, expired request between IP 86.28.83.109 (VirginMedia, Hook) and the IP 62.254.42.126 (VirginMedia, Hook).
So i contacted the call center, and found a very good operator that opened a remote connection with my pc, verifing the issue
He have fixed an appointment for tomorrow morning, and a technical will check whats wrong in my connection.
I'll post what will happen.

P.S. FitzPattons, you can easily check tracert in dosPrompt.
And you can check the settings of ipv4, setting ip and dns to on obtain automatically, sometimes the provider delay the personals settings

[davecon]

Thanks Gus

If you try to take a quick look to the ping to api.betfair.com, opening the dos prompt and typing 'ping api.betfair.com' you haven't results because betfair have disabled this functionality

I ping www.api.betfair.com to test my bot connection is this not the same?
This is using Virgin Media 50mb although I have no idea what I am looking at lol

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\Davec>ping www.api.betfair.com

Pinging www.api.betfair.com [81.200.64.50] with 32 bytes of data:
Reply from 81.200.64.50: bytes=32 time=16ms TTL=51
Reply from 81.200.64.50: bytes=32 time=19ms TTL=51
Reply from 81.200.64.50: bytes=32 time=19ms TTL=51
Reply from 81.200.64.50: bytes=32 time=18ms TTL=51

Ping statistics for 81.200.64.50:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 16ms, Maximum = 19ms, Average = 18ms

Cheers
Dave

[BFairUser]

Yes, you're right, I was wrong to write.

'ping api.betfair.com' gives you a result.
But I think that 'this result' is not the Betfair server's IP, is the IP of the server of your provider which tries to find the domain api.betfair.com and failed in the search.
I mean that if you try to 'ping' the betfair server, you get no results, because (I think) Betfair has turned off the ping response.
If you select the IP address that you are getting, does not seem to be the Betfair server (try 'tracert 81.200.64.50' and tries to investigate what you can find)
I think you have to take a look at the IP to which you send the web request, and then try to ping that IP address.
I think (then again, maybe I'm wrong) that if you run your program that performs the 'webRequest' (which establish a connection with the Bf server), and check in dosPrompt with the command 'netstat -ano' the established connections, you'll find the IP address with which you are 'talking'.
If we try to do the 'ping' of that address will get no any answer (what I meant).
But if you make a request 'tracert ... (ipAddress)' you find the last IP address that belongs to the server you are looking for.
Thank you.
P.S. Sorry for my english.
P.P.S. Thanks DaveCon.

[gus]

It seems that a lot of people with VirginMedia connections have been having problems.

I just happened to notice this thread on the Horse Racing part of the Betfair Website's Forum:

API issues for Virgin Media users
Betfair Customer Services

We are aware of the current issues with the API and are looking into this ASAP.

Apologies for any inconvenience caused.

---------------------------
Betfair Customer Services
05 Feb 15 16:36

Please note these issues are for Virgin media clients only and and unfortunately out of our control.

There's more but you can find it for yourselves!

Like I said, I'm not experiencing these problems, but it's not unusual for some geographical locations to be affected and others to be OK.

[davecon]

Hi BFairUser

Thanks for that now I know Ive been wasting my time pinging that www before Login as I looked into this and indeed it seems they have blocked it for some reason
Anyway it does not affect my bot so I'm still happy
Thanks again
And your English is fine
Cheers
Dave

But I think that 'this result' is not the Betfair server's IP, is the IP of the server of your provider which tries to find the domain api.betfair.com and failed in the search.
I mean that if you try to 'ping' the betfair server, you get no results, because (I think) Betfair has turned off the ping response.

[davecon]

Have you tried this one

pathping api.betfair.com

(Ive no idea what it is but somebody reccomends)

C:\Users\Davec>pathping api.betfair.com

Tracing route to api.betfair.com [84.20.200.10]
over a maximum of 30 hops:
0 Dave [192.168.0.2]
1 routerlogin.net [192.168.0.1]
2 cpc4-roch7-2-0-gw.10-1.cable.virginm.net [86.10.100.1]
3 manc-core-2b-xe-003-0.network.virginmedia.net [82.8.208.69]
4 * * *
Computing statistics for 75 seconds...
Source to Here This Node/Link
Hop RTT Lost/Sent = Pct Lost/Sent = Pct Address
0 Dave [192.168.0.2]
0/ 100 = 0% |
1 2ms 0/ 100 = 0% 0/ 100 = 0% routerlogin.net [192.168.0.1]
0/ 100 = 0% |
2 16ms 0/ 100 = 0% 0/ 100 = 0% cpc4-roch7-2-0-gw.10-1.cable.virgi
nm.net [86.10.100.1]
0/ 100 = 0% |
3 12ms 0/ 100 = 0% 0/ 100 = 0% manc-core-2b-xe-003-0.network.virg
inmedia.net [82.8.208.69]

Trace complete.

Dave