Tweet
After a break from botting I'm back and currently having a go at consuming the stream api with PHP.
Just wondering how other people have gone about it, or how you think you'd go about it. Interested in hearing any PHP waffle.
I've reached the stage where I can connect and reliably start streaming using PHPs stream functions in a reasonably asyncronous manner and build simple market representations. Seeing the data in full flow for the first time was like viewing the matrix! I have puzzled over how to interface this code with bot code and what the overall structure / program flow would be like.
My streaming code is based on a stream_select() loop, whereas my existing APING code uses a curl_multi_select() loop where necessary. These cannot be easily combined (without an ugly hack) as phpcurl cannot currently import/expose underlying sockets, phpcurl doesn't support streaming either.
My current idea and goal is rather than integrate streaming code into existing bots, is to keep my streaming code as a separate process, like a local server that can distribute data to multiple strategy scripts. I think that will leave my options reasonably open, it can serve strategies that want constant updates as well as ones that only need slower updates.
To build my local server I think an event loop is the way to go, so that I can asynchronously:
Starting to sound complicated! I believe the above would be relatively straightforward if one was familiar with the right extensions/libraries/packages. I don't know which library would do it all and do it well.
I have been experimenting and have found it would be possible to do everything in vanilla php using non-blocking stream functions and stream_select() rather than an event loop, but there's a few wheels that need reinventing in order to get desired performance/efficiency.
SSL is a complication too. When it's working it's fine, but establishing connections, slow connections, and closing connections, are problem areas for the stream functions. Establishing an SSL connection can be realised asynchronously with a slight workaround. With non-blocking streams slow connections result in a lot of "" empty reads between valid reads, even though stream_select indicates data. Closed connections also result in "" empty reads too, no easy way to differentiate between slow & closed. I made an ugly hack to extract the underlying FD from a stream, which fopen would DUP so that it could then be checked for EOF, this seems to work.
I'm currently experimenting with pecl event extension, it has an event loop and it offers an SSL integration that seems to work better, I haven't tried it in earnest yet though.
Just wondering how other people have gone about it, or how you think you'd go about it. Interested in hearing any PHP waffle.
I've reached the stage where I can connect and reliably start streaming using PHPs stream functions in a reasonably asyncronous manner and build simple market representations. Seeing the data in full flow for the first time was like viewing the matrix! I have puzzled over how to interface this code with bot code and what the overall structure / program flow would be like.
My streaming code is based on a stream_select() loop, whereas my existing APING code uses a curl_multi_select() loop where necessary. These cannot be easily combined (without an ugly hack) as phpcurl cannot currently import/expose underlying sockets, phpcurl doesn't support streaming either.
My current idea and goal is rather than integrate streaming code into existing bots, is to keep my streaming code as a separate process, like a local server that can distribute data to multiple strategy scripts. I think that will leave my options reasonably open, it can serve strategies that want constant updates as well as ones that only need slower updates.
To build my local server I think an event loop is the way to go, so that I can asynchronously:
- resolve dns (not essential as betfair rarely changes IP addresses)
- access the stream
- access session functions login/keepalive
- perhaps access APING functions, needed to get any missing information, e.g. runner names
- serve local bots using IPC, probably unix sockets
- log what's going on
- store market data for testing purposes
Starting to sound complicated! I believe the above would be relatively straightforward if one was familiar with the right extensions/libraries/packages. I don't know which library would do it all and do it well.
I have been experimenting and have found it would be possible to do everything in vanilla php using non-blocking stream functions and stream_select() rather than an event loop, but there's a few wheels that need reinventing in order to get desired performance/efficiency.
SSL is a complication too. When it's working it's fine, but establishing connections, slow connections, and closing connections, are problem areas for the stream functions. Establishing an SSL connection can be realised asynchronously with a slight workaround. With non-blocking streams slow connections result in a lot of "" empty reads between valid reads, even though stream_select indicates data. Closed connections also result in "" empty reads too, no easy way to differentiate between slow & closed. I made an ugly hack to extract the underlying FD from a stream, which fopen would DUP so that it could then be checked for EOF, this seems to work.
I'm currently experimenting with pecl event extension, it has an event loop and it offers an SSL integration that seems to work better, I haven't tried it in earnest yet though.
Comment