Is MarketDescription and MarketCatalogue a 1:1 or 1:many relationship?

[JohnFromCairnsAustralia]

Hello there,

I am working through understanding the API. My current dilemma is the MarketCatalogue and MarketDescription objects provided by the API.

On my apps end, I initially thought I could merge these two into one object thinking they were 1:1... I think that was a mistake. My recent understanding of these two objects is as follows;

MarketDescription is a description of a market. That description never really changes.
MarketCatalogue is a record of whats goes on for that market over time. There can be multiple of these for each market description?

For example; I could have a horse race that runs once per year. That would have a MarketDescription that remains the same, but a market catalogue for each year the race is run?

or alternatively does Betfair ALWAYS have 1:1 making a new Market Description of each race. The Market Start Time and Market Suspend Time would probably indicate that it IS a 1:1 relationship...

Thank you kindly for any comments
John

[jabe]

When I read your post initially I thought you were asking about the hierarchy of the various pieces of data that can be retrieved with API calls, but on reading it again, I think I was mistaken.

The marketDescription contains several pieces of data contained within a single marketCatalogue. Both are one-offs because both contain date-time fields. For apparently sensible reasons, Betfair's data designers decided that there are subsets of data that it makes sense to group together. One of the first things I did was to create classes for all pieces of data. Many of these classes included data contained within further classes or sub-classes. For the most part it made sense to group data in this way, not least to make it clear that the elements within were connected in some way. This process helped me see where pieces of data were held. I used to be in IT, so I know about system design, and Betfair's data design makes a lot of sense. That said, it was several years ago.

I'm not aware of any piece of data that you could use to determine recurring events such as the World Cup or the Grand National, other than from the marketName in the marketCatalogue or the eventName in the Event class. While there may be some commonality in the names from year to year (or whatever the interval), changes may include the actual year, not to mention names of new sponsors.

If I've misinterpreted your question, let me know.

[JohnFromCairnsAustralia]

Hi Jabe, thanks heaps for your response and thoughts,

I take your point that there is no real data to determine recurring events. Also that the time fields would indicate they occur only once and are thus unique.

My main concern was to wonder why Betfair chose to separate MarketCatalogue.Description from MarketCatalogue? If it really is a 1:1 relationship, why not just merge the two?

For example. From the C# sample code there are two objects ; MarketCatalogue references MarketDescription;

public class MarketCatalogue {
[JsonProperty(PropertyName = "competition")]
public Competition Competition { get; set; }
[JsonProperty(PropertyName = "description")]
public MarketDescription Description { get; set; }
[JsonProperty(PropertyName = "event")]
public Event Event { get; set; }
[JsonProperty(PropertyName = "eventType")]
public EventType EventType { get; set; }
[JsonProperty(PropertyName = "isMarketDataDelayed")]
public bool IsMarketDataDelayed { get; set; }
//etc

and...
public class MarketDescription {
[JsonProperty(PropertyName = "bettingType")]
public MarketBettingType BettingType { get; set; }
[JsonProperty(PropertyName = "clarifications")]
public string Clarifications { get; set; }
[JsonProperty(PropertyName = "bspMarket")]
public bool IsBspMarket { get; set; }
[JsonProperty(PropertyName = "discountAllowed")]
public bool IsDiscountAllowed { get; set; }
[JsonProperty(PropertyName = "persistenceEnabled")]
public bool IsPersistenceEnabled { get; set; }
//etc

One might also note that betfair have named their Runner and RunnerDescription classes similarly...In that instance, RunnerDescription is static data about for example, a horse, or sports team...but the runner object is for dynamic data. There is a definite 1 to Many relation there.....I was concerned they perhaps may have used the same pattern...

It is possible I am concerned about nothing important
Thank you kindly for your thoughts tho

[jabe]

There are options with some of the calls to only request certain parts of the data. I note that marketCatalogue contains a totalMatched figure for the market. The most obvious place for that is on the marketBook, and it is there too. It would be possible to ask listMarketCatalogue not to bother with data you already have or which you don't need, just for efficiency. This is done on the listMarketCatalogue using the marketProjection parameters. It gives a user more options. When deciding what data to retrieve, it may be that the MARKET_DESCRIPTION (the parameter to request it) is not relevant to what you need to know at that time; the same may apply to RUNNER_DESCRIPTION and especially to RUNNER_METADATA*.

Additionally, when looking at code it so easy for it to appear as one huge messy mass. Dividing it into smaller but sensibly divided chunks makes it easier to read. As an ex-IT developer, I reckon that if a section or function or procedure or subroutine is more than a screen long, it's probably time to look at splitting it one way or another.

That said, one of the things I particularly enjoyed, other than investigation, was trying to improve the efficiency of a program. Time, money, and lack of seriously badly performing programs (and worries held by owners of critical systems) generally did not permit this, but spectacular results were possible with problem programs.

I think that you may be worrying over not much, but I do believe that it can be helpful to hear what other people think about these things. I have never used object oriented programming in a professional capacity and was concerned that if I tried to teach myself using online sources or books that I wouldn't get to see the "industry standard" way of doing things. I once went for a job interview for a post as a trainee Visual Basic.NET developer. They expected me to know far more than I did. I expected them to teach me how to do it properly. It turned out that what they meant by trainee was someone who knew nothing about the development cycle, rather than someone who knew that but didn't know quite so much about VB.NET. In the end, the main barrier between us was that I hadn't used VB for database programming. They invited me to reapply when I'd learnt how to do it, but I never got round to it. So I still don't know the industry standard for OO and there are still aspects of it I've never used.



* my main program adds up the amount of data retrieved on every call and displays it. In order to save time if restarting the program, much of the data is saved as files and loaded into the program to save it having to retrieve another ton of data. A typical daily amount so far is under 20MB, so not very much. My local ISP has installed fibre all over the place, so I now have a very fast connection, so the total data download is pretty much irrelevant now.

[JohnFromCairnsAustralia]

I think you are correct that I am probably worrying about nothing. I will continue to treat it as a 1:1 relation. The motive for them splitting it may be along the lines of efficiency with sending the data unnecessary over the wire... 10 years back not sure how ubiquitous fibre or high speed internet was...

Well, as an aside. I am sort of in a similar boat with my own career. I have worked professional as a developer but only for a period until that company went bankrupt I can't say I learned all that many good practices there... Pluralsight is what has taught me the most. It's a long road tho and full of pitfalls. You could if you so chose spend years and years learning 'best practice'...from my perspective it's all a huge trade off to getting to a goal quickly or not. I have bitten into the whole Domain Driven Design thing. It's great and all...but its a far longer path than a data driven design (if you can accept that you have dependencies that aren't easily changed).... I enjoy writing neat maintainable, consistent code, but it's not necessarily the most profitable thing to do

I use an orm and MVVM patterns for my app, its been a learning curve for me, and in retrospect would probably ditch the user interface for a consol app... UI's come with a lot of complication...and I guess so do databases and orms....

[jabe]

I was at my first IT place for ten years and they initially had a hierarchically structured database. They decided to replace it with an SQL compatible one and at about that time a chap called WIlliam Reynolds, who had been important at IBM, came in and gave us an enjoyable and memorable two-week course in entities and processes. It's stuck with me ever since. Part of it that had promise was the idea of having start and end dates for data, which can be very useful.

I'm afraid it was all before the internet, so I've had to look up several of the items and acronyms you mentioned. I've often found after completing a program that one realises too late that there may have been a better way to do something in it. My current VB.NET program, because I was having to come to terms with so many new things - VB, to an extent, objects, the API, communicating across the internet, async actions, etc - it's the most complicated program I've tried to write by a long chalk. The design was perhaps a little over-ambitious too. It remains incomplete and bits of it feel messy but unentanglable. I might rewrite it in C# (another learning curve) and see if it has newer options that allow me to improve parts of it. I gather Microsoft have no plans to develop VB.NET at all. A rewrite would allow me to re-familiarise myself with what's there already and hopefully get me over the hill and to its end.

[JohnFromCairnsAustralia]

I'm in my 40's so never had to deal with hierarchical databases. It all changes so much, but one hopes things slow down a bit from here on maybe.....its tough on the individual, but I guess change brings opportunity...

I started out with MS-Access, then VB.Net...but always preferred curly bracket languages, essentially, vb isn't all that different to c#...just more verbose...a lot of work to change over, but I don't think you'd find it too strenuous...

I have found myself that it's all the design patterns that entail the biggest effort ... MVVM, DDD, ORMs, Repositories, CQRS, onion architecture, SOLID, etc....all that stuff you can sink years into and every man and their dog has an opinion on how to architecture things and depending on who you talk to they use different concepts and terms interchangeable...so googling things is mildly dangerous...(you may have discovered that after googling MVVM)

...so once you get there all that stuff is great...but takes a few years of learning...and I would argue you may be better off without it depending on your goals... (it's fun learning it all tho...like doing a giant jigsaw puzzle)...

[Guest]

Great reading fellas. I am also in my 40's John and a local just down the road (Brizzie) I use VB.Net and quite happy with it and with my Betbot . The only issue now is trying to find/make my BetBot rules/strategies profitable. I think my Bot placed >300 bets on one day last week for a net loss of $12 (including Betfair's commission). Thank goodness the bot was only placing $1 bets

I actually came out here to read up on Stream API and to work out whether what advantages Stream offers over what I am doing now.

[JohnFromCairnsAustralia]

Heya Troy! I grew up in Brisbane... dislike the traffic a bit now....Job prospects are much better there than in Cairns tho (A good part of the reason I am into trading)... a net loss of $12 is probably better than most so keep trying...

I don't yet use the Stream API...one day...if you want to load historical data, you'll want to know it...all that stuff comes in stream API json message format...

I may not be the best person to give advice about it, but I would say the main advantage is if you are relying on highspeed/low latency for your algorithms... in terms of work load and labours, you still have to understand the regular pull API anyway... on top of that you'll want to learn about .net reactive extensions I would say...

The guy that did the C# sample code has some stream stuff in it, but I haven't had a good look at it for 12 months tho...
https://github.com/joelpob/betfairng

...at that time reactive extensions was just a bit too much overload for my brain There's a good course or two on it on pluralsight if you can afford a subscription....I've skim watched them but forgot what I learned....I've always pushed it down on my list of things to know....one day tho