Getting abysmal speed with API-NG

[geoffw123]

Hi

I noticed today (Fri 15th Aug) that the response times I am getting from the Betfair API-NG Server are very varied and abysmally slow.

I am using a slightly hacked version of the Betfair C# example code. I haven't long got this working so not sure if it is just something that is bad today or has been like this all along ? I suspect it wasn't this bad yesterday though.

I have spent most of today tinkering with this but no joy in fixing it. I am simply requesting listMarketBook once per second and seeing response times on average 800 mS for 2500 bytes. The variation is all over the place though, as low as 100 mS to 3 or 4 seconds sometimes.

I cant see a problem with my code that would cause this. I have been reading some older similar posts here that mention 2 relevant factors.

1) Getting the Server to return Gzip responses to speed things up. After much struggle I managed to add this to my code. Result was no measurable improvement

2) Tell the Server to keep the connection alive. I haven't been able to figure out how to do this as I am using the JSON-RPC link which uses WebRequest not HttpWebRequest, the later has a keepalive property but the former doesn't.
Does anyone know how to persuade the JSON-RPC code to keep the connection alive ?

I have just discovered a neat tool call Fiddler which gives me some visibility of the connection between me and Betfair.

For a request that was very slow 2 seconds or so, here are the stats

ACTUAL PERFORMANCE
--------------
ClientConnected:	21:33:04.701
ClientBeginRequest:	21:38:36.987
GotRequestHeaders:	21:38:36.987
ClientDoneRequest:	21:38:36.987
Determine Gateway:	0ms
DNS Lookup: 		0ms
TCP/IP Connect:         0ms
HTTPS Handshake:	0ms
ServerConnected:	21:33:07.876
FiddlerBeginRequest:	21:38:36.987
ServerGotRequest:	21:38:36.987
ServerBeginResponse:	21:38:39.212
GotResponseHeaders:	21:38:39.212
ServerDoneResponse:	21:38:39.212
ClientBeginResponse:	21:38:39.213
ClientDoneResponse:	21:38:39.213

Overall Elapsed:	0:00:02.226

The culprit here seems pretty obvious unless I am missing something subtle, the time between ServerGotRequest and ServerBeginResponse is over 2 seconds !!!!

Can any networking Gurus throw some light on my struggles here please ? Hello Betfair Developer Support some thoughts from you guys would be appreciated too !

Regards Geoff

[doctormike]

Slow response times

Hi Geoff
I've just migrated my arbitrage app to API-NG and, like you, am getting slow responses to listMarketBook requests (about 60% slower than old API). It takes about 1-2 seconds for including API response and the other tasks relating to getting bookie prices, compared to less than 1 second with my original app. Maybe Betfair's development team might want to respond to this/address the issue.
With regards
Mike

[geoffw123]

Hi Mike

Thanks for the reply.

If you try it now (Monday 18th) its speeded up vastly, I have changed nothing of significance in my code and I am no longer seeing the huge delays where it was taking seconds to get the data.

I have left my App. logging data while I am typing this post, I am now seeing typically 80 mS response time with an average time of 78 mS over 1100 messages.

Its pretty annoying the level of support we get from Betfair here. I wasted hours struggling with this problem, with no word whatsoever from Betfair. No pre-warning of any potential Server problem, no response to my forum post, no message from them apologising and letting us know they have now fixed the issue.

Is there a Betfair Support section that reads these messages ? It makes you wonder !

[doctormike]

API problems again this morning

Hi Geoff
Real problems again with API this morning (Tuesday, 7am). Not just API-NG either. Login delays via app, and I'm getting timeouts when I make getMarketCatalogue requests. The problem is more widespread, as I'm getting similar from the old API. Maybe BF team are doing maintenance. They'll have to sort it out pronto, as there are many commercial apps out there, and York's coming up!
With regards
Mike

[doctormike]

Re slow response times

Geoff, Ignore my previous post - the delays I was experiencing in my apps this morning were not down to the BF APIs - rather they were due to problems on the odds comparison site I scrape bookmaker odds from.
With regards
Mike

[doctormike]

listMarketBook still consistently slow

Hi Geoff
listMarketBook remains slow. I thought it was exclusively down to the odds comparison site I use being slow but it appears not. I've been calling once per second, asking for EX_OFFERS for one race in each call. Next I'm going to try getting prices for 5 races per single call then batch process them - not ideal but may suffice for my purposes if it speeds up the iteration through races sufficiently. Are you using any async procedures?
With regards
Mike

[geoffw123]

Hi Mike

What do you define as "slow" ? Over 2000 calls I am averaging an 82ms Response time. This has been pretty typical for the last few days. I am happy with the speed for now, so no I haven't used async calls to get prices. I have been experimenting with async calls yesterday to get the Betfair graphs. Just about got that working now, not totally sure I have implemented it properly though.

Interesting to hear you are working on an Arbitrage App. When I had a brief look at that strategy it seemed to be very difficult to make it profitable. I could only see small price variations which were soon swallowed up by the 5% Betfair commission.
Has it been a workable strategy for you ?

Regards Geoff

[khughes]

Hi Geoff

I ran some timing tests out of curiosity to get a ball park idea for the kind of response times I am getting, and also to compare the two APIs.

The first test I ran was on API-NG listMarketBook. The test comprised making 60 calls with a 1 second interval between each call. The time measured was fairly simple, it was just the time taken for the line of code in my program that makes the API call to return with the JSON response string (so it is not the pure API response time but shouldn't be too far off). The calls were made on a UK horse racing market. The parameters were set to request just the SP_AVAILABLE and EX_ALL_OFFERS prices (no orders or matches). The responses were GZipped.

Results (all times are in milliseconds):

Pre-event: Min 218 Max 328 Avg 253
In-play: Min 218 Max 375 Avg 264


I then carried out a similar test using API6 making calls to getMarketPricesCompressed with the following times:

Pre-event: Min 171 Max 265 Avg 191
In-play: Min 171 Max 312 Avg 207


So the new API-NG is a little slower than API6 which was surprising. And therefore while the two APIs are available, it is not a level playing field with users of the older version enjoying a slight advantage.
I don't see the huge response times that you observed so not sure what the cause might be there.

Hope the info is useful anyway.
Regards
Kevin

[geoffw123]

Todays Timings (Sunday 31st)

Hi

The problem I was seeing with 2 sec or longer responses was definitely a Betfair issue. It lasted for a day or so then suddenly went away, and it has been pretty fast for days now.

I just re-checked the figures for the new api-ng today.

I am seeing a bit of variance today in response times, some as large as 400 mS, but typically I am seeing a 50 mS response time, with some as low as 35 mS. Average is about 100 mS.
I don't have any figures for the old V6 comms to compare against.

My figures seem to be about twice as fast as yours, I don't have a clue why that is. I am using pretty standard code from the Betfair example they published. (If its relevant, I am in the UK with a pretty good home cable connection )

Regards Geoff