# Index size limit

**URL:** <https://community.cratedb.com/t/index-size-limit/260>\
**Category:** CrateDB\
**Created:** [July 30, 2019, 12:48pm UTC](https://community.cratedb.com/t/index-size-limit/260 "2019-07-30T12:48:06Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![julianmar](https://avatars.discourse-cdn.com/v4/letter/j/ed655f/32.png) [@julianmar](https://community.cratedb.com/u/julianmar)\
**Post date:** [July 30, 2019, 12:48pm UTC](https://community.cratedb.com/t/index-size-limit/260/1 "2019-07-30T12:48:06Z")

</div>

Hi,

I have a Table with a size of about 40gbs and this table will only grow. So about this amount in a year.  
So in 2 years we would have about 120gbs of data.

Can Crate handle this kind of Data and will I still be able to query over this amount of data?  
Are there some settings I can change to improve performance?  
More Sharding and maybe more Nodes could help?

Thanks for every answer 🙂

---

<div class="post-metadata">

**Author:** ![smu](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/smu/32/60_2.png) [@smu](https://community.cratedb.com/u/smu)\
**Post date:** [October 24, 2019, 7:34am UTC](https://community.cratedb.com/t/index-size-limit/260/2 "2019-10-24T07:34:04Z")

</div>

Yes CrateDB can handle such load. Adding nodes will help as the query load will be spread across them if sharding is set up properly. Shard size should stay in certain limits, [https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster) is a good read related to that and as CrateDB uses the sharding logic of Elasticsearch it also applies to CrateDB. The bigger a shard the more bytes need to be transferred on reallocation/recovery. Also the maximum number of docs in a shard is 2^31, which is a lucene hard limit.

For time-based datasets, using a partitioned table can help a lot with read performances as queries can be limited to hit only a low number of partitions (e.g. limit to latest records only).  
See [https://crate.io/docs/crate/reference/en/latest/general/ddl/partitioned-tables.html](https://crate.io/docs/crate/reference/en/latest/general/ddl/partitioned-tables.html).