JSON to database

[JayBee]

Has anyone looked into getting JSON strings into a database or some sort of half-way store?

For time series collection you just want to store the JSON strings and not process them at that stage as that slows the system down. You just want to concentrate on getting the data.

At a later data you would want to get the JSON strings into some sort of RDBM.

Has anyone looked into this already?

[smilerdude]

If you're just looking to store the JSON string data for later processing, the simplest/quickest method is to write it out to a text flat file sequentially for later data batch processing.

Reading your question, it appears that you've already got the data from the JSON to make your betting decisions on. Why not
use a ASYNC process to write that data into a DB at the same time of when your programme is making decisions with that data?

The reason why I mention that is because since you've already got an object built up with data, you might as well write it out immediately to a DB of your choice. Rather than having waste time/effort to extract the same data at later date.

So you can either write out the data to

1)a RDBM - MySQL/MariaDB, ProgreSQL - you need to sort out the data relationships

2) a document store - MangoDB - writes out the JSON object in a key/pair relationship.

Personally, I'm using MariaDB and I've written a Node.js programme to handle the ASYNC DB reads/writes, to handle JSON calls and betting decisions.

The reason is that I'm sticking with MariaDB (originally MySQL 5.1) is that I want to keep historic betting data that I've got over the last few years.

Even if this setup doesn't work well eventually, I've gained enough experience over the past 20 years working with DB's that I can transform data between any DB formats with relative ease.

[JayBee]

Well, it looks like you don't have any speed issues with serializing and async saving of the data to a db in realtime.

I just wanted to make a specific time series bot that saved the JSON for later processing so that I could specifically grab data for every race in any one day.

I've looked at document dbs but I think I will just save the JSON in plaintext and process it afterwards.