C# The beginnings of a bot

[The Geeks Toy]

I agree polling is better, but I think it needs taking one step further in that especially for GCMPC & GMTVC you should combine the calls into one & use an incremental update process between client & server instead of returning the entire market on each call.

Same timestamp thoery, but only returning data lines that have changed since the last timestamp, with the client feeding back the last returned timestamp on the next call.

It's a bit more work at both ends, however seeing as the new generation of apps are utilising these calls more, it would cut down the bandwidth requirement to a fraction of that currently used.

[vossie]

I agree polling is better, but I think it needs taking one step further in that especially for GCMPC & GMTVC you should combine the calls into one & use an incremental update process between client & server instead of returning the entire market on each call.

Same timestamp thoery, but only returning data lines that have changed since the last timestamp, with the client feeding back the last returned timestamp on the next call.

It's a bit more work at both ends, however seeing as the new generation of apps are utilising these calls more, it would cut down the bandwidth requirement to a fraction of that currently used.

Hi Maverick,

Good to see you back here, hope all is well I have been looking at the possibilities of incremental updates and even thow the theory behind it is solid and undeniable you do have a certain level of head ache that comes with it.

• The first is complexity, I know the BF APIs is not what you would describe as simple but doing this would add another layer of complexity which we would prefer to move away from (guess we could ship a betfair.dal.dll for .NET clients)
• Secondly you need to store change points and then how long do you cache them for 1, 2 or 3 seconds and in the first that case we are stuck with say 5+1 cache objects rather than 1 multiply number of active markets.
• blah blah blah (tired of my own voice now)

I think the short and simple would be that yes I would like to see incremental updates for price and bet data probably cached for 2 change ticks and if you are out by more than 2 ticks you get the whole lot. Note: Changes get pushed up to the server in batches, so it would be 2 change pushes for incremental increases. On busy markets this would be sub-second.
(This is being researched by the way)

[The Geeks Toy]

Hi Maverick, Good to see you back here, hope all is well

Things are fine. Been a little busy in recent times with a product launch.

Re change points. You can hold the full history on the server, if you hold a timestamp against each data row. { And this is the really simplified one server , one market, one selection, 4 price, solution }

The incremental server { IS } holds a snapshot of the market data along with a timestamp against each row. EG

10:00 AM 1.01 £10
10:00 AM 1.02 £0
10:00 AM 1.03 £20
10:00 AM 1.04 £30

First client request for the market uses a timestamp of DateTime.Min. Data returned is... {NB You don't need to return the timestamp for each row to the client, just the highest one on the IS. Just included below for explanation clarity.}

10:00 AM 1.01 £10
10:00 AM 1.03 £20
10:00 AM 1.04 £30

Client now holds the above, and a timestamp of 10:00AM
================================================== =============

New data is pushed to the IS from the market server @ 10:01

10:01 AM 1.01 £10
10:01 AM 1.02 £0
10:01 AM 1.03 £20
10:01 AM 1.04 £40

When merged, the resulting IS snapshot is

10:00 AM 1.01 £10
10:00 AM 1.02 £0
10:00 AM 1.03 £20
10:01 AM 1.04 £40

Client makes incremental call with the 10:00 timestamp, and is returned

10:01 AM 1.04 £40

Client now holds

10:00 AM 1.01 £10
10:00 AM 1.03 £20
10:01 AM 1.04 £40

and a timestamp of 10:01AM
================================================== =============

New data is pushed to the IS from the market server @ 10:02

10:02 AM 1.01 £0
10:02 AM 1.02 £0
10:02 AM 1.03 £30
10:02 AM 1.04 £40

When merged, the resulting in the IS snapshot is

10:02 AM 1.01 £0
10:00 AM 1.02 £0
10:02 AM 1.03 £30
10:01 AM 1.04 £40

Client makes incremental call with the 10:01 timestamp, and is returned

10:02 AM 1.01 £0
10:02 AM 1.03 £30

Client now holds
10:02 AM 1.03 £30
10:01 AM 1.04 £40

and a timestamp of 10:02AM
================================================== ==============

Another client can make a call with a 10:00 timestamp & get

10:02 AM 1.01 £0
10:02 AM 1.03 £30
10:01 AM 1.04 £40

Yada, Yada, Yada. Any client can call with any timestamp & get just the changes. As long as you have a single consistent timestamp / changepoint from the market data source, it should scale easily to multiple API servers.

Only other thing I can think of off the top of my head is the IS data for each market needs to be locked when being updated by the Market Server, however I doubt this would be much of an issue.

Anyway, long story short, you have only a single cache object for each market which covers all changepoints, which you merge new market data into instead of replacing it.

[vossie]

that looks good, but like I said theory is great until you stick it on an enterprise environment. How large cluster do you need to make the IS scalable and by the time you have built and configured this what benefit have you gained? a couple of gigs worth of bandwith but you have the overhead of the new clusters, software/hardware maintenance, increased complexity etc.

[The Geeks Toy]

Not my field of expertiese so I have no idea of the hardware requirements & bandwidth costings. I'll shall leave that to the real geniuses to work out. I just offered up a possible solution for the change points.

In the meantime I'll go crawl back under my stone for another few months.

[vossie]

In the meantime I'll go crawl back under my stone for another few months.

Please don't do that

[Macberrypro]

Don't use date/time stamps. Moreover use an increasing integer based stamp. For enterprise solutions, this will scale both horizontally and vertically

[vossie]

Don't use date/time stamps. Moreover use an increasing integer based stamp. For enterprise solutions, this will scale both horizontally and vertically

A timestamp would come from the data source so they would be accurate and of more relevance to the end user i.e. us. On a F.I.X. system int would probably make sense because you can ask for retransmits but on a sports exchange it has little value.

I really don't see how knowing the number of changes you missed help and what happens if you have a market that is on the site for a year? Any thoughts.

Unix timestamp: 125852735
Rolling int after 24 hours+/-: 432000
Rolling int after 300 days +/-:129600000

Not really enough of a difference to affect performance

[Monairda]

Good morning

I downloaded the application, but do not get any market or price.
I follow the first post but I do not get information like the sample image samples. We have to make a step further?



Sorry for the English,
Thank you very much

[vossie]

Click play/start and post a screen shot

[Monairda]

Thank you very much,

As you can see the odds are empty. And I get the error GetMarketPricesCompressed:HEADER_ERROR_EXCEEDED_TH ROTTLE:BODY_ERROR=API_ERROR

I guess it will exceed the refresh for free api. Where can modify that?

Also object of obtaining the message:
AutoMarketLoader:Going to sleep until 07/12/2009 8:13:49

Greetings

[vossie]

Thank you very much,

As you can see the odds are empty. And I get the error GetMarketPricesCompressed:HEADER_ERROR_EXCEEDED_TH ROTTLE:BODY_ERROR=API_ERROR

I guess it will exceed the refresh for free api. Where can modify that?

Also object of obtaining the message:
AutoMarketLoader:Going to sleep until 07/12/2009 8:13:49

Greetings

Hi Monairda,

If you wait for a while it will create all the markets even with a free API subscription it just takes a while because you have to wait for the "throttle" to reset. Just wait for it to load.

The auto market loader is the library that looks for new markets at set intervals based on the "Strategy Load Patern" you configure. To add/edit/modify the query go to
Edit > Main Configuration > Strategy Load Paterns

[Monairda]

Thank you very much

I waited for several minutes and this was the result.
Markets appear to show you in the picture. Odds are charged only in two markets, the rest is left empty.
The reason is because the free API can not load it?

the second image belongs to the market odds 09-11-27 19:20
Odds displays no decimal places instead of 2.4 € shows 24€. Do I have to configure anything for the show with decimals?

In the third image shows no odds and and I waited for several minutes

Best Regards

[vossie]

What region is your computer set as?
"Start > Settings > Control Panel > regional and Language Options"
Try changing it to English (United Kingdom)

I suspect it is a problem with locales. Also note that the market view user control rounds to the nearest whole number like the site does, it does not mean the data is wrong in the backend.

[Monairda]

Hi vossie

Region => Spain

At 21.00 hours we have three markets:

\GB\Wolv 27 Nov\ To Be Placed 09-11-27 21:20
\GB\Wolv 27 Nov\ To Be Placed 09-11-27 21:50
\GB\Wolv 27 Nov\ To Be Placed 09-11-27 22:20

And continues to show only the odds of one market, is it normal?

Best Regards