Stream API Questions (contd)

[geoffw123]

Hi

I am finding it pretty tough to understand the new Stream API Spec. and the sample code on Github, could anyone help with this please ?

Lets say we have an App. where we have 10 markets that we want to get the best 3 Back and Lay prices, but we also have a ladder grid open for an 11th market where we want the full market depth.

This was easy using the old API-NG, for each market price request we tagged the sent data with either EX_BEST_OFFERS or EX_ALL_OFFERS.

For the new stream subscription system I cant figure out how this could be done ?

The MarketSubscriptionMessage class contains these 2 member variables.

public MarketDataFilter MarketDataFilter { get; set; }
public MarketFilter MarketFilter { get; set; }

where the MarketFilter contains the list of the mktIds to subscribe to.
the MarketDataFilter contains the variable that allows us to pick either "EX_BEST_OFFERS" or "EX_ALL_OFFERS".

So to get our 10 markets to "best offers" depth of 3, we could do that easily enough setting the MarketDataFilter and MarketFilter appropriately.

But how do we now get the 11th market with full market depth.

We cant send another marketSubscription msg with just the 11th market to full depth, as that would cancel the other 10 market subscriptions.

I must be missing something obvious here ?

Thanks for any help

Regards Geoff

[betdynamics]

You could set up two separate streams - one for the markets with a depth of 3; the other for the market with full depth.

Alternatively, just set up a single stream with full depth and filter the number of prices within your application.

The second one is probably easier to do if you are using the sample code.

[geoffw123]

Hi BetDynamics

Thanks for the reply, I guess you are agreeing with my understanding of this then, there isn't any simple way to configure it efficiently.

Option 1) Set up two streams, that sounds complex and a bit of a clunky solution. One stream is complex enough for my liking.

Option 2) Just subscribe to all required markets at full depth. This is nice and simple, but it kind of defeats the purpose of the new Stream API. That was designed to reduce the traffic that the BF Servers have to send out. Now I am having to ask for a lot of price data I don't really need due to the way the protocol has been designed.

Geoff

[betdynamics]

Geoff

I'm not sure it defeats the object of the new API. The API is set up so that data is only sent when data on the market has changed.

It depends on how frequently you used to poll for data as against how often the data on the market changes.

For example:

Imagine someone was polling market data every 200ms for a bunch of largely static markets (I know! A bit unlikely, but I'm trying to illustrate a point).

In the polling scenario, a bunch of data is being sent for all of the markets, 5 times per second.

In the streaming example, only very little data is being sent back.

It's horses for courses, really. The streaming API does not replace polling and there will always be cases where polling is the sensible way forward.

[geoffw123]

hi

Yes I see your point and agree.

It just seemed odd that didn't design the filter so you have to subscribe to a list of tuples of <marketId, depth required> this would give the benefit you describe and also prevent having to ask for more price data than needed per marketId.

Digressing slightly, just reading the spec of this now. It states

If no marketIds passed user will be subscribed to all markets

Really ? There must be a zillion Betfair markets, surely they don't mean you will get every price change for every open market ? Excuse me a moment while I go and get a fire extinguisher for my poor old Internet connection !!

That must be a horrendous amount of data most of the time ?

Geoff

[betdynamics]

My understanding is that "firehose" applications require special approval from Betfair.

I don't think Betfair are too worried about the amount of information being sent out. Instead, I think they are looking to cut down the number of requests for data that they receive.

These sort of streaming APIs are common in the banking sector.

[geoffw123]

Some feedback for BF and some more questions

Hi

My Stream API document pdf I downloaded a while ago looks out of date to me (it mentions Aus exchange which is now defunct). I was trying to find a link to a newer pdf version today but couldn't find one. Could anyone point me at the latest download link please ?

Some feedback for BF on this document
1) Please add a document revison number in the pdf so we can at least ask what the latest version is !!

2) This document is not as easy to understand or as comprehensive as the API-NG pdf file. It could do with some extra work.

For example it mentions a marketDefinition data structure is sent, but doesn't define exactly what this contains. From perusing the code, this is a large data structure of over 30 variables. I think its a stream specific structure, so its not defined in the API-NG pdf either. Please add this definition to the next document update.

Questions for BF (or anyone else)
3) The stream pdf doc says

The Exchange Stream API supports GBP currency only.

Errrm, why ? Is this just a temporary issue that you are planning to fix shortly ? Or is this the final design intent?

It seems odd that ListMarketBook() in the API-NG gives us prices in Euros for example, but the equivalent data in the Stream API is GBP only. Having to do the conversion on the client side, seems a recipe for bugs and inconsistences. For example listCurrentOrders() will still be giving results back in Euros.


4) The Stream doc says nothing at all about unsubscribing from markets. It would appear that there is no function to unsubscribe, so I am pondering what I should be doing here ?

Presumably Betfair are OK with the fact that I just keep adding to my subscription list when needed. This might be ok from their point of view as markets will naturally close and the Server wont then be sending any data for that market.

It would be helpful to get a few words from Betfair on what they consider is best practice here to keep the data throughput down to acceptable levels.

[LiamP]

The latest documentation is on the home page:

http://docs.developer.betfair.com/do...nge+Stream+API

[geoffw123]

Hi Liam

Thanks for the reply, but I was looking for a pdf file to download. That's harder to find.

Geoff

[jabe]

Hi Liam

Thanks for the reply, but I was looking for a pdf file to download. That's harder to find.

Geoff

It certainly is with that Atlassian banner wasting so much screen space. The documentation has been reduced to about 40% of the page, which is absurd.

The PDF versions of the documentation seem to have dried up some time back.

[BetfairDeveloperProgram]

Hi Geoffw.

1) Please add a document revison number in the pdf so we can at least ask what the latest version is !!

I've added a link to the offline version of the documentation via http://docs.developer.betfair.com/do...eDocumentation

2) This document is not as easy to understand or as comprehensive as the API-NG pdf file. It could do with some extra work.

For example it mentions a marketDefinition data structure is sent, but doesn't define exactly what this contains. From perusing the code, this is a large data structure of over 30 variables. I think its a stream specific structure, so its not defined in the API-NG pdf either. Please add this definition to the next document update.

The full specification is documented via the Swagger Definition link (http://docs.developer.betfair.com/do...ggerDefinition)

The field definitions & description are the same as those documented for the Exchange API via http://docs.developer.betfair.com/do...ons-MarketBook and http://docs.developer.betfair.com/do...arketCatalogue

Questions for BF (or anyone else)

3) The stream pdf doc says
Quote:
The Exchange Stream API supports GBP currency only.
Errrm, why ? Is this just a temporary issue that you are planning to fix shortly ? Or is this the final design intent?

This is intentional and is the final design. There are various complexities around maintaining multi currency streams so a decision was made to provide the feed in the base currency (GBP) and document currency conversion for those who require volume data in other currencies.

It seems odd that ListMarketBook() in the API-NG gives us prices in Euros for example, but the equivalent data in the Stream API is GBP only. Having to do the conversion on the client side, seems a recipe for bugs and inconsistences. For example listCurrentOrders() will still be giving results back in Euros.

Betting data is explicitly tied to the account currency.

4) The Stream doc says nothing at all about unsubscribing from markets. It would appear that there is no function to unsubscribe, so I am pondering what I should be doing here ?

Presumably Betfair are OK with the fact that I just keep adding to my subscription list when needed. This might be ok from their point of view as markets will naturally close and the Server wont then be sending any data for that market.

There is no explicit way to unsubscribe from specific markets other than to remove them from the subscription message. This isn't a problem from our perspective and is how we expect customers to use the service. See http://docs.developer.betfair.com/do...Considerations regarding Stream API performance considerations.

Kind Regards

Neil

[geoffw123]

Hello Neil

Thanks very much for the replies, it helped a lot.

The currency being only in GBP is a bit of a pain, bearing in mind that there is no way to change your currency for testing purposes once you have created a Betfair account.

This means that if I want to support multiple currencies to test it properly I would have to create a new account using a different currency. I wonder if it lets you create a 2nd account though with the same address and Credit Card number but just with a different currency ? I suspect it would reject that ?

I will probably just hack the low level code to make it look like the Server sent me a non GBP account setting and test it that way.

Regards Geoff

[LiamP]

It certainly is with that Atlassian banner wasting so much screen space. The documentation has been reduced to about 40% of the page, which is absurd.

+1, mentioned this before to Neil, makes the site so hard to use!

[geoffw123]

+2 Agree totally, the layout of that web page is dreadful.

Whoever designed that page must have a liking for sitting outside his house and reading his favourite book through the letter box from 10 yards away.

[geoffw123]

Next questions on the Stream API

Still slowly plodding my way through the Stream code....
The next thing I am puzzled by that the docs don't explain relate to the assortment of prices returned.

In the sample code I have marketData.Snap.MarketRunners.Prices
below that I have 6 lists of prices

AvailableToBack
AvailableToLay
BestAvailableToBack
BestAvailableToLay
BestDisplayAvailableToBack
BestDisplayAvailableToLay

From what I can figure out the variables that have the word "Display" in them means the list is inclusive of virtual prices. Virtual here means that extra prices are potentially present due to the cross matching Betfair are doing internally.

The other 4 variables don't have virtual prices.

Two questions spring to mind here.

Q1) Why would an App not want to take advantage of extra advantageous prices provided by the cross matching. So why are non virtual prices returned at all ?

Q2) I was expecting the virtual prices to be returned to full market depth, but as the variable name implies they are only for "Best". I.e if you look at the list BestDisplayAvailableToBack you see just 3 entries (3 is what is defined in the MarketDataFilter for the "Best" depth)

Am I correct in saying that cross matched prices are only in the "best" depth ladders ?

Anyone have a clearer understanding of this ?

Thanks Geoff