The Next Sports Exchange API is Coming

[AlgoTrader]

The new API will allow you to make a request that retrieves the complete market depth, your current position (i.e., getMUBets), and the traded volume of the market in a single response, for 1 or more markets.

It really looks too fantastic to update all the markets of interest with all the data there in a single request. Well, if I ask for 20 markets with all the data bundled, won't it hurt latency? If the response is one megabyte, the response download time will kill the benefits of bundling 20 markets together.

It should be gzipped 4-8 times at least though. getAllMarkes currenlty gzipped in 10 times (lots of repeating strings), while login only 2 times.

Look forward to see anything real :-)

[shuba]

I like Java and so I like XML, I like SOAP :-) And I try to defend it.
Somebody, please explain me hysteria around JSON.
What the significant difference between Betfair XML and Betfair JSON for the Betfair developer?

1. Traffic difference? But I forgot for traffic since I had turned on gZip encoding on my software.
2. More simplest parsers? I dont think so. JSON and SOAP parsers already made for any programming language/framework. You never need to write you own one. May be just for misteriuos GetCompleteMarketPricesCompressed raw string. But after implementing Betfair JSON you need to rewrite you Betfair framework. Again.
3. New features? I dont need it. I implemented all features I need by manual.

BDP! All of my software is STABLE now. I don need neither JSON nor new features. I made workarounds and fixes for all Betfair bugs. I DONT NEED to suffer from new bugs, to write new workarounds, to stick to my wall near PC new "Betfair API defects" sheet and whole night to look inside debug window.

BDP! Please keep XML/SOAP as long as possible or dont reject it at all!

[AlgoTrader]

I like Java and so I like XML, I like SOAP :-) And I try to defend it.
Somebody, please explain me hysteria around JSON.

1. JSON is much shorter
2. JSON is much simpler, no namespaces, WSDL, xsi/xds types
3. JSON is easily mapped to most languages native types without any libs
4. JSON is human eye friendly
5. JSON parse is hell of fast

What the significant difference between Betfair XML and Betfair JSON for the Betfair developer?

No need for ugly SOAP libs. Less trafic (you have limist for traffic if you use VPS). Easy to implement.

BDP! Please keep XML/SOAP as long as possible or dont reject it at all!

I understand you try to protect your investments in SOAP. But as soon as people switch to JSON you will be underdog. The less network traffic and parsing latency decrease (just several milliseconds, but faster) will make SOAP users in great disadvantage. There will be much less traffic because of much less invocations to server, BF people tell us we will need just 1 JSON invocation to update market.

[Toy]

I like Java and so I like XML, I like SOAP :-) And I try to defend it.
Somebody, please explain me hysteria around JSON.

You gotta move with times, this will be a major leap forward. Plus you will have 18 months+ to migrate.

[shuba]

1. JSON is much shorter
2. JSON is much simpler, no namespaces, WSDL, xsi/xds types
3. JSON is easily mapped to most languages native types without any libs
4. JSON is human eye friendly
5. JSON parse is hell of fast
t.

I think JSON shorter, faster and simpler on 10Mb server response. But not on 1-2K Bet fair response. I think JSON lib and "ugly" SOAP lib will parse 1K for identical time.

To my mind XML more human eye friendly then JSON hieroglyphs
:-)

[AlgoTrader]

To my mind XML more human eye friendly then JSON hieroglyphs
:-)

Lets compare:

<n:loginResponse xmlns:n="http://www.betfair.com/publicapi/v3/BFGlobalService/">
    <n:Result xsi:type="n2:LoginResp">
        <header xsi:type="n2:APIResponseHeader">
            <errorCode xsi:type="n2:APIErrorEnum">OK</errorCode>
            <minorErrorCode xsi:nil="1" />
            <sessionToken xsi:type="xsd:string">PKdvAW+ruIAg0s769DeM+vxY=</sessionToken>
            <timestamp xsi:type="xsd:dateTime">2007-08-23T11:25:05.285Z</timestamp>
        </header>
        <currency xsi:type="xsd:string">GBP</currency>
        <errorCode xsi:type="n2:LoginErrorEnum">OK</errorCode>
        <minorErrorCode xsi:nil="1" />
        <validUntil xsi:type="xsd:dateTime">0001-01-01T00:00:00.000Z</validUntil>
    </n:Result>
</n:loginResponse>

JSON:

{
    "header": {
        "errorCode": "OK",
        "minorErrorCode": null,
        "sessionToken": "pirdDtkE/8R3LkVHBeTZEnmB31jzZsdC3E6s95inAvI=",
        "timestamp": "2012-08-24T07:13:43.324Z"
    },
    "currency": "USD",
    "errorCode": "OK",
    "minorErrorCode": null,
    "validUntil": "0001-01-01T00:00:00.000Z"
}

And tell me honest which one is easier to read? Please don't try to convince me namespaces, xsi:type attributes, closing tags help you better undertand things. Can you please explain me, how XML's '<' and '>' make things much better than JSON's '{' and '}'?

PS. The session code is not valid one

[AlgoTrader]

I think JSON shorter, faster and simpler on 10Mb server response. But not on 1-2K Bet fair response. I think JSON lib and "ugly" SOAP lib will parse 1K for identical time.

You have to compare 50kb JSON vs 150kb XML. JSON has no namespaces, xsi attributes, closing tags, SOAP stuff. See the post above, the same response is in times shorter.

In many languages you do not need external libs to parse JSON.
PHP: json_decode function parse json to PHP var
JavaScript: JSON.parse method parses json into native object
C#: System.Runtime.Serialization.Json

SOAP is legacy now, fortunately, Betfair is smart enough to innovate and make things modern.

JSON response message will be parsed before XML SOAP response is fully received. The only exception is "compressed" calls, for example getAllMarkets has just one tag <marketData>... 100kb of text</marketData>. I would suggest make getAllMarkets exception and have it still text-encoded.

[shuba]

xsi:type="n2:LoginErrorEnum"

This is cost of strict typification which is advantage of SOAP and weakness of Jason where typification is absent

[shuba]

You have to compare 50kb JSON vs 150kb XML. JSON has no namespaces, xsi attributes, closing tags,

Hm.... 50 and 150K? What kind of Bet fair response you mean ?

[AlgoTrader]

xsi:type="n2:LoginErrorEnum"

This is cost of strict typification which is advantage of SOAP and weakness of Jason where typification is absent

There is no Jason it JSON.
JSON has types. Everything in "" is string, everything without "" is number, everything in {} is object, everything in [] is array.

That's enough. I code like this

var response = getAllMarkets();
if(response.errorCode!='OK') {
    // invalid response
    return;
}
// valid response

What we have with XML? DOM parser with ugly API or SAX parser that is not much better? SOAP libs that generates megabytes of complex code by generating hundreds of classes and enums? My current XML parser and serializer is some 7kb:

https://github.com/AlgoTrader/betfai...air_encoder.js
https://github.com/AlgoTrader/betfai...air_decoder.js

SOAP libs generated me megabytes of code

If you do validation of every response in app (I never seen anybody who does such validation by applying WSDL scheme) then I like your way - you increase your processing latency and make things slow.

Hm.... 50 and 150K? What kind of Bet fair response you mean ?

The part of login XML response in the example above is 756 bytes. The same in JSON is 337 bytes. It is just after removing closing tags, namesapaces, attributes, etc. It includes formatting, after removing spaces XML is 652 bytes, JSON is 281 bytes. JSON is 43% of XML in this case, in more important calls the difference will be bigger.

I faced JSON several years ago and I was wondering why we need it because we already have XML? Three years later most of our modern web apps are JSON based. The benefits are so clear and convincing. I certainly prefer simplicity and speed to namespaces, schemas, WDSL, validation, types, SOAP enverlops and other stuff.

[wotsisname]

I think the easiest solution is for Betfair to introduce the JSON format API (which is what the majority of developers want) and if some other customer wants XML, she can always use a standard JSON to XML parser.

[AlgoTrader]

There is no reasonable JSON->XML converter. I tried several online tools just for fun:

http://jsontoxml.utilities-online.info/

Json:
{
   "name" : "John Smith",
    "age" : 40,
    "children" : [
        { "name": "Maria" },
        { "name": "Ann" }
    ]
}
XML:
<?xml version="1.0" encoding="UTF-8" ?>
	<name>John Smith</name>
	<age>40</age>
	<children>
		<name>Maria</name>
	</children>
	<children>
		<name>Ann</name>
	</children>

The XML is not valid

http://json.online-toolz.com/tools/x...-convertor.php

JSON:
{
   "name" : "John Smith",
    "age" : 40,
    "children" : [
        { "name": "Maria" },
        { "name": "Ann" }
    ]
}
XML:
<?xml version="1.0" encoding="UTF-8"?>
<json>
<name>John Smith</name>
<age>40</age>
<children>
<name>Maria</name>
</children>
<children>
<name>Ann</name>
</children>
</json>

The XML is valid but is complete shit.

The XML->JSON converters exist and work good, although JSON contains lots of garbage - attributes, namespaces, xsi:types are still in JSON with weird encoding rules. The approach is proved to be very poor (I used it in my lib)

The JSON->XML is just complete nonsense. JSON arrays are completely destroyed while encoded in XML, XML has no "array" concept.

I will forget about XML as soon as JSON API is in usable state

[Fred77]

I too am happy enough with SOAP/XML - JSON is fine too, but I don't see JSON as some massive leap forward, though I do welcome an overhaul of the API.

I was hoping the new API might feature something more exciting such as streaming / push technology, e.g. the client subscribes to markets of interest and live updates (price changes/bet matchings etc) are sent to the client as they happen, rather than the client having to poll for changes. Something like what Smarkets are attempting - but done properly!

[AlgoTrader]

I too am happy enough with SOAP/XML - JSON is fine too, but I don't see JSON as some massive leap forward, though I do welcome an overhaul of the API.

Let's compare:
xml:

<request>
  <header>
  <clientStamp>0</clientStamp>
    <sessionToken>eA6cUxo+2IC7BYKZFOMqomfaPwVSZfT+1tQn66q15zM=</sessionToken>
    </header>
    <bets xsi:type="n2:ArrayOfCancelBets">
      <CancelBets  xsi:type="n2:CancelBets">
        <betId xsi:type="xsd:long">21772814452</betId>
      </CancelBets>
      <CancelBets xsi:type="n2:CancelBets">
        <betId xsi:type="xsd:long">21772814453</betId>
      </CancelBets>
      <CancelBets  xsi:type="n2:CancelBets">
        <betId xsi:type="xsd:long">21772814454</betId>
    </CancelBets>
  </bets>
</request>

json:

{
    "header": {
        "sessionToken": "eA6cUxo+2IC7BYKZFOMqomfaPwVSZfT+1tQn66q15zM=",
        "clientStamp": 0
    },
    bets: [ "21772814452", "21772814453", "21772814454",]   
}

It's not the secret Betfair heavily uses arrays but there is no "array" concept in XML. Array of ints in JSON is just "[1, 2, 3]", the same in XML:

<array xsi:type="ns:ArrayOfInts">
  <int xsi:type="xsd:integer">1</int>
  <int xsi:type="xsd:integer">2</int>
  <int xsi:type="xsd:integer">3</int>
</array>

ns:ArrayOfInts cannot be omited, the parser needs to know "array" tag is array container. Then every <int> is element container. In fact, int's tag type can be omited but BF never drops types in responses.

Closing tags, attributes, namespaces, weird array encodings - everything makes SOAP XML so garbaged, so Betfair decided to implement "compressed" calls for the most critical things. getAllMarkets has smart text encoding, otherwise responses can easily reach multiple megabytes. We need to use complex SOAP libs or make our own intellectual parsers, after that, we need to decode "compression" by applying post-SOAP processing. Why so much complexity to do quite simple things?

Decoding JSON is rarely something more complex than

response = json_parse(data);

JSON is easily mapped into native types of most of languages, for example in C++/Qt

// create a parser instance
QJson::Parser parser;
bool ok;

// json is a QString containing the data to convert
QVariant result = parser.parse (json, &ok);

Saying farewell to DOM or SAX XML parsers or mutimegabytes SOAP generated stuff, parsing "compression" and other woodoo stuff is really majour leap forward

[Regi Keyz]

To BetFair Team:

Hi,

I'm looking forward to your new API, to be released soon hopefully

May I say something?

I've studied the rise of Japan from broke to becoming a major economic power over the last half century.
One of the many 'secrets' of their success, was because at lunch times in 60's, in the major manufacturing plants, the blue collar workers would eat at the same table as the white collar managers/directors (in UK and US, the white collars didnt want to sit with the blue collars). The white collar Japs knew the guys at the bottom often had essential real life infomation that could massively improve their production process.. information usually ignored by the people at the top.. history tells us the result, their manufacturing overtook USA to become ultimately the best in the world. Of course there were other factors at play too, but..
..with this in mind, please would you develop this API in close association with some of the more experienced guys on this forum eg wotsisname and co.. the last API had lots of issues, it would be a shame to put in all this work and not get full benefits from it!
Especially now other competition is on the rise, if you can get your API spanking, it will part a part in keeping you at pole position as the top exchange out there.

Just my thought
Thanks for listening!