Hello All,
This is just a generic question on updating data. We’re currently ingesting data into CrateDB as straight inserts and never updating it, with several hundred rows going in with each bulk insert (with >18 million rows going in a day)
We’re now in a position where we are offering to ingest 3rd parties data into our platform. This means we will want to allow 3rd parties to insert and update data and so for every insert we’ll be doing an INSERT INTO … ON CONFLICT () DO UPDATE SET…
as we will not know if its an insert or an update.
Is this an expensive operation for the Crate engine?
I know Crate is very capable, but not really an RDBMS, so I’m wary of trying to update on each failed insert.
Or am I just being paranoid ?
Many thanks
David.