How to tier partitions onto cheaper disks

CrateDB Configuration
1

Hi, new to Crate, apologies if this is covered in the docs, but I couldn’t find it.

I want to partition my data by date so that very old shards live on slower, much cheaper disks, on all the nodes. Is this possible? Or is it only possible to specify certain nodes as “slow”?

Thank you.

2

Hi there, just posting a reply in the hope it re-raises visibility to somebody who knows. Many thanks for any pointers.

3

Hi,
While you can configure multiple paths for data on each node, these are treated equally, so to distinguish between classes of storage you will need multiple nodes, but nothing prevents you from running multiple CrateDB logical nodes on the same “physical” machine.
Once you got CrateDB nodes with different classes of storage you can do something along the lines of Building a hot and cold storage data retention policy in CrateDB with Apache Airflow - Tutorials - CrateDB Community
I hope this helps.

1 Like
4

Thanks very much, super helpful