Understanding whats happening in the Sample Files

[Lopiner]

I have been messing around with the sample files and i have some questions about it, things that i cannot figure out. If anyone from the BDP team can help clarify i would be very appreciated.

Note: Neil just clarified that the Image field isn't relevant in this post.

I'm analyzing the football-adv-sample.bz2 file and i was checking a random market and selection, in this case the market ID "1.131162806" which is a Match Odds Market for Middlesbrough v Man City and i choose the "47999" selection ID which is Man City.

The first message for this Market ID and Selection ID looks like this:



First question that arises is that this is clearly not the first 50ms market cycle because we already have some considerable matched volume and what seems like a stable market although with thin liquidity. Will this happen in the paid files, is this expected?

The second and third message are ok, there were two trades, the first one of 10£ at 1.35. The traded volume rises for 20£ as expected and 10£ disappear from the market depth. The second trade is a small 3.12£ traded at 1.34 and everything is also as expected:



Message number 4 seems like some sort of correction, it's a full image where some of the market depth amounts rise by some cents and the traded volumes also change by some cents (some of them decrease which is very hard to understand):



I don't know what this means. My first thought was that some of the market depth levels were changed by some cents and this amounts didn't trigger the threshold needed for a message to be created (if there is one). This isn't possible because the traded volumes change above the best back/lay. It's also improbable that the people/person who were occupying the 1.34, 1.35, 1.36 and 1.37 all updated their current bets at the same time for a few cents.

Message number 5 is what appears to be another correction, in this case more understandable. It happens exactly one hour after the previous message and there are no decreases in traded volume although the traded volumes changes in areas outside best back/lay.



Message 6 is another mystery, it seems like a full image but it isn't, it has some of the data, mostly unchanged:



I cannot make sense of the data, i understand that it can be unreliable, but what behavior is expected, what is unexpected or noise? Thanks

[jptrader]

I suspect that messages 4 and 5 are due to currency updates. Every hour, Betfair updates the currency rates of GBP vs other currencies. So every order in the queue with a non-GBP currency will change by some small number. This may also explain a decreasing traded volume, as the numbers reported in the historical data are all GBP.

[Lopiner]

I suspect that messages 4 and 5 are due to currency updates. Every hour, Betfair updates the currency rates of GBP vs other currencies. So every order in the queue with a non-GBP currency will change by some small number. This may also explain a decreasing traded volume, as the numbers reported in the historical data are all GBP.

Thanks JPTRADER.
I thought of that but i find it very strange to update traded volumes.
That may explain those two messages, meaning that the money waiting at 1.34, 1.35, 1.36 and 1.37 was in part or in full non GBP and thats why it was updated on message 4 and 5.

In the traded volume the reasoning makes less sense because if in fact there were matched trades in non GBP in message 4 in 1.34, 1.35, 1.36, 1.38, 1.41 and 1.43 in message 5 the same values would be updated unless the change was less than one cent. In fact in message 5 only the traded volumes of 1.35, 1.36, 1.38 and 1.40 changes.

[BetfairDeveloperProgram]

I suspect that messages 4 and 5 are due to currency updates. Every hour, Betfair updates the currency rates of GBP vs other currencies. So every order in the queue with a non-GBP currency will change by some small number. This may also explain a decreasing traded volume, as the numbers reported in the historical data are all GBP.

Hi Lopiner/JP Trader.

I can confirm the above is correct regarding the currency updates.

First question that arises is that this is clearly not the first 50ms market cycle because we already have some considerable matched volume and what seems like a stable market although with thin liquidity. Will this happen in the paid files, is this expected?

No, this won't happen in the PRO files which will show every update. This appears to be the ADVANCED file.

Thanks

Neil

[Lopiner]

Hi Lopiner/JP Trader.

I can confirm the above is correct regarding the currency updates.



No, this won't happen in the PRO files which will show every update. This appears to be the ADVANCED file.

Thanks

Neil

Thanks Neil.

I incorrectly named the ADVANCED file but i was analyzing the PRO.
Indeed this is the first message for that market:

{"op":"mcm","clk":"3199907563","pt":1493129993642,"mc":[{"id":"1.131162806","marketDefinition":{"bspMarket":true,"turnInPlayEnabled":true,"persistenceEnabled":true,"marketBaseRate":5.0,"eventId":"28202626","eventTypeId":"1","numberOfWinners":1,"bettingType":"ODDS","marketType":"MATCH_ODDS","marketTime":"2017-04-30T13:05:00.000Z","suspendTime":"2017-04-30T13:05:00.000Z","bspReconciled":false,"complete":true,"inPlay":false,"crossMatching":true,"runnersVoidable":false,"numberOfActiveRunners":3,"betDelay":0,"status":"OPEN","runners":[{"status":"ACTIVE","sortPriority":1,"id":63907,"name":"Middlesbrough"},{"status":"ACTIVE","sortPriority":2,"id":47999,"name":"Man City"},{"status":"ACTIVE","sortPriority":3,"id":58805,"name":"The Draw"}],"regulators":["MR_INT"],"countryCode":"GB","discountAllowed":true,"timezone":"Europe/London","openDate":"2017-04-30T13:05:00.000Z","version":1629124022,"name":"Match Odds","eventName":"Middlesbrough v Man City"},"rc":[{"atb":[[8,6],[1.01,45176],[10,54.67],[1.03,20000],[10.5,30],[11,62.19],[1.02,28000],[1.05,5000],[1.04,19000]],"id":63907},{"trd":[[12.5,3.12],[10.5,44.54],[11,260.45],[11.5,2.83],[12,168.08]],"ltp":12.5,"tv":479.02,"id":63907},{"atl":[[12,18.7],[14,5],[490,1.78],[1000,0.1]],"id":63907},{"spn":11.5,"spf":"NaN","id":63907},{"atb":[[1.01,45176],[1.03,25000],[1.02,28000],[1.33,200.1],[1.32,300],[1.3,20000],[1.29,20000],[1.31,20000],[1.34,107.3],[1.05,5000],[1.04,24000]],"id":47999},{"trd":[[1.35,108.85],[1.4,10.48],[1.41,20.45],[1.36,204.81],[1.37,90],[1.43,1.56],[1.42,200],[1.34,71.74],[1.38,71.88]],"ltp":1.35,"tv":779.77,"id":47999},{"atl":[[1.36,188.8],[1.37,112.75],[1.5,0.1],[490,1.78],[1.35,32.65],[1000,0.1]],"id":47999},{"spn":1.34,"spf":"NaN","id":47999},{"atb":[[1.28,1.7],[1.01,26],[5.8,54.97],[1.26,1.7],[3.5,20],[5.6,50],[5.1,50],[1.27,1.7],[5,50.78],[1.25,2],[5.5,94.04],[1.29,1.7],[5.7,16.15]],"id":58805},{"trd":[[5.8,10.73],[5.6,28.31],[5.5,5.96],[6,49.12],[5.9,31.98]],"ltp":5.8,"tv":126.1,"id":58805},{"atl":[[6.4,5],[6.2,57.82],[950,2.2],[240,1.78],[1000,0.1]],"id":58805},{"spn":6.0,"spf":"NaN","id":58805}],"con":true,"img":true,"tv":1464.66}]}

It includes a bunch of traded volume, clearly this inst the first message. I trust that the paid files wont have this problem so lets forget that issue.

The currency issue is understood.
What about message 6, is it also currency related?

Thanks,

[StefanBelo.]

In spec you say there are batb, batl data presented, but in sample file there are no such data.

I downloaded horse racing sample file.

[BetfairDeveloperProgram]

In spec you say there are batb, batl data presented, but in sample file there are no such data.

Hi StefanBelo

The batb and batl is provided in the ADVANCED file only, the PRO file contains atb and atl data which is as per the data spec.

Kind Regards

Neil

[BetfairDeveloperProgram]

The currency issue is understood.
What about message 6, is it also currency related?

This indicates a change in Traded Volume which could be caused by an currency fx update (for bets matched at non GBP currencies).

Thanks

Neil

[Lopiner]

Hi Neil,

I think its better if we point out specific examples of things that we don't understand or find strange instead of asking how is the data arranged.

In the "football-pro-sample" i'm looking at this two messages:

{"op":"mcm","clk":"3232194336","pt":1493526539865,"mc":[{"id":"1.131162793","rc":[{"atb":[[2.2,0]],"id":6660614},{"trd":[[2.2,15.14]],"ltp":2.2,"tv":15.14,"id":6660614}],"con":true,"img":false,"tv":40.32}]}

and

{"op":"mcm","clk":"3232194460","pt":1493526542507,"mc":[{"id":"1.131162758","rc":[{"atb":[[2.18,0.11]],"id":63907,"hc":1.5},{"trd":[[2.18,311.82],[2.2,0.01]],"ltp":2.18,"tv":311.83,"id":63907,"hc":1.5}],"con":true,"img":false,"tv":3194.8},{"id":"1.131162793","rc":[{"atb":[[2.2,0]],"id":6660614},{"trd":[[2.2,15.14]],"ltp":2.2,"tv":15.14,"id":6660614}],"con":true,"img":false,"tv":40.33}]}

This two messages have information relating Market ID "1.131162793" and Selection ID "6660614". The first one says that a trade was made at 2.20 for the amount of 15.14 which left the market depth of 2.2 (Available to Back) with no money. So there is no money at 2.2 right now.

The second message comes after the first, there are no updates for that market between them.
It says that a trade was made at 2.2 which inst possible unless the two bets (the lay and the back) were sent exactly at the same time, in the same market cycle. This would be plausible (but improbable) but traded value remains basically the same (increases by one cent) so i think it's a duplicate or a currency update. If this is a currency update it's very difficult to identify as such.

Could you please check this out and maybe explain? Thanks