# Tutorials **URL:** https://community.cratedb.com/c/tutorials/28.md?page=2 [Latest](https://community.cratedb.com/latest.md) · [Categories](https://community.cratedb.com/categories.md) · [Tags](https://community.cratedb.com/tags.md) **Page:** 3 --- ## [Fundamentals of eventual consistency in CrateDB](https://community.cratedb.com/t/fundamentals-of-eventual-consistency-in-cratedb/1235)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [September 30, 2022, 6:23am UTC](https://community.cratedb.com/t/fundamentals-of-eventual-consistency-in-cratedb/1235 "2022-09-30T06:23:43Z")
CrateDB has certain characteristics when describing its transactional guarantees: Atomicity and eventual consistency. Atomicity on record-level guarantees that if a record is written to CrateDB, it either gets written… --- ## [User-Defined Functions in CrateDB](https://community.cratedb.com/t/user-defined-functions-in-cratedb/1218)
**Author:** [@rafaelasantana](https://community.cratedb.com/u/rafaelasantana)\ **Replies:** 0\ **Last updated:** [September 19, 2022, 8:16am UTC](https://community.cratedb.com/t/user-defined-functions-in-cratedb/1218 "2022-09-19T08:16:35Z")
Watch the tutorial on our Youtube channel! In this tutorial, I will show you how to build your own User-Defined Function step-by-step. If you are struggling with finding the proper function for your database task, t… --- ## [Fundamentals of the COPY FROM statement](https://community.cratedb.com/t/fundamentals-of-the-copy-from-statement/1178)
**Author:** [@Marija](https://community.cratedb.com/u/Marija)\ **Replies:** 0\ **Last updated:** [July 26, 2022, 9:16am UTC](https://community.cratedb.com/t/fundamentals-of-the-copy-from-statement/1178 "2022-07-26T09:16:35Z")
COPY FROM is a handy command: it allows you to connect CrateDB with a filesystem and import data from local or remote files to a specified table. The syntax is fairly simple: often, you just need to specify the destinati… --- ## [Set up an Apache Superset development sandbox with CrateDB](https://community.cratedb.com/t/set-up-an-apache-superset-development-sandbox-with-cratedb/1163)
**Author:** [@amotl](https://community.cratedb.com/u/amotl)\ **Replies:** 0\ **Last updated:** [June 29, 2022, 8:25pm UTC](https://community.cratedb.com/t/set-up-an-apache-superset-development-sandbox-with-cratedb/1163 "2022-06-29T20:25:45Z")
:spiral\_notepad: Introduction This is a little walkthrough about how to quickly spawn a development sandbox with Apache Superset, in order to work on the CrateDB Python driver with live code reloading. :battery: Prere… --- ## [Quickly starting CrateDB with 2.5M records of the NYC Yellowcab dataset](https://community.cratedb.com/t/quickly-starting-cratedb-with-2-5m-records-of-the-nyc-yellowcab-dataset/1162)
**Author:** [@amotl](https://community.cratedb.com/u/amotl)\ **Replies:** 0\ **Last updated:** [June 29, 2022, 4:20pm UTC](https://community.cratedb.com/t/quickly-starting-cratedb-with-2-5m-records-of-the-nyc-yellowcab-dataset/1162 "2022-06-29T16:20:15Z")
:spiral\_notepad: Introduction This is a little note on how to quickly spawn a single-node CrateDB instance on your workstation using Docker and load it with a subset of the NYC Yellowcab dataset. The intention is to ha… --- ## [Migrating to a different table schema](https://community.cratedb.com/t/migrating-to-a-different-table-schema/1155)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [June 23, 2022, 8:49am UTC](https://community.cratedb.com/t/migrating-to-a-different-table-schema/1155 "2022-06-23T08:49:13Z")
Let’s assume you have a table with some data inserted already and you what like to organize your data with a different table structure. Maybe your first thought is to write a simple script that reads the data from the ol… --- ## [Optimizing storage for historic time-series data](https://community.cratedb.com/t/optimizing-storage-for-historic-time-series-data/762)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 10\ **Last updated:** [June 22, 2022, 9:31am UTC](https://community.cratedb.com/t/optimizing-storage-for-historic-time-series-data/762 "2022-06-22T09:31:04Z")
When dealing with time-series data, performance is crucial. Data that gets ingested should be available quickly for querying. To enable fast decision-making, analytical queries often need to return results in a fractio… --- ## [Best practices for transferring data from one CrateDB cluster to another](https://community.cratedb.com/t/best-practices-for-transferring-data-from-one-cratedb-cluster-to-another/1148)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [June 9, 2022, 10:09am UTC](https://community.cratedb.com/t/best-practices-for-transferring-data-from-one-cratedb-cluster-to-another/1148 "2022-06-09T10:09:51Z")
When operating multiple CrateDB clusters, you might need to copy certain data from one cluster to another. There can be several reasons why to run multiple CrateDB clusters, such as having different stages (development… --- ## [Storing and querying hierarchical data in CrateDB](https://community.cratedb.com/t/storing-and-querying-hierarchical-data-in-cratedb/1147)
**Author:** [@Marija](https://community.cratedb.com/u/Marija)\ **Replies:** 0\ **Last updated:** [June 8, 2022, 8:58am UTC](https://community.cratedb.com/t/storing-and-querying-hierarchical-data-in-cratedb/1147 "2022-06-08T08:58:38Z")
CrateDB provides a great way to store and query large amounts of data quickly and efficiently. But, how do you store data with hierarchical links between records, such as a file system? In this article, we will look at … --- ## [Guide to decrease the number of shards in your cluster](https://community.cratedb.com/t/guide-to-decrease-the-number-of-shards-in-your-cluster/873)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 3\ **Last updated:** [June 3, 2022, 3:39pm UTC](https://community.cratedb.com/t/guide-to-decrease-the-number-of-shards-in-your-cluster/873 "2022-06-03T15:39:39Z")
Potentially you created your tables with too many shards and want to rectify this situation. Some indications for too many shards can be: you have many small shards with typical shard sizes being below 5GB, or you ha… --- ## [Building a hot and cold storage data retention policy in CrateDB with Apache Airflow](https://community.cratedb.com/t/building-a-hot-and-cold-storage-data-retention-policy-in-cratedb-with-apache-airflow/934)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 4\ **Last updated:** [May 11, 2022, 12:47pm UTC](https://community.cratedb.com/t/building-a-hot-and-cold-storage-data-retention-policy-in-cratedb-with-apache-airflow/934 "2022-05-11T12:47:55Z")
In this fourth article on automating recurrent CrateDB queries with Apache Airflow, we will present a second strategy for implementing a data retention policy. Previously, we shared the Data Retention Delete DAG, which… --- ## [Storing long-term metrics with Prometheus in CrateDB](https://community.cratedb.com/t/storing-long-term-metrics-with-prometheus-in-cratedb/1012)
**Author:** [@rafaelasantana](https://community.cratedb.com/u/rafaelasantana)\ **Replies:** 0\ **Last updated:** [February 17, 2022, 6:39pm UTC](https://community.cratedb.com/t/storing-long-term-metrics-with-prometheus-in-cratedb/1012 "2022-02-17T18:39:16Z")
This tutorial was written in collaboration with Johannes Faigle In this tutorial, I show how to Set up Docker Compose to run CrateDB, Prometheus, and the CrateDB Prometheus Adapter Run the applications with Docker … --- ## [Interpolating missing time-series values](https://community.cratedb.com/t/interpolating-missing-time-series-values/1010)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [February 11, 2022, 8:55am UTC](https://community.cratedb.com/t/interpolating-missing-time-series-values/1010 "2022-02-11T08:55:02Z")
This article requires CrateDB \>= 4.7.0. For older CrateDB versions, please refer to a previous version of this article. When dealing with time-series data, you will likely at some point run into the question of how … --- ## [Building a data retention policy for CrateDB with Apache Airflow](https://community.cratedb.com/t/building-a-data-retention-policy-for-cratedb-with-apache-airflow/1001)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [February 2, 2022, 10:19am UTC](https://community.cratedb.com/t/building-a-data-retention-policy-for-cratedb-with-apache-airflow/1001 "2022-02-02T10:19:08Z")
In time-series use cases, a significant amount of data can accumulate over time. Making efficient use of available storage can be a key factor for the economic success of a database solution. Keeping full historical d… --- ## [Connecting to CrateDB in Trino](https://community.cratedb.com/t/connecting-to-cratedb-in-trino/993)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 3:48pm UTC](https://community.cratedb.com/t/connecting-to-cratedb-in-trino/993 "2022-01-19T15:48:32Z")
Trino (formerly known as Presto SQL) is a distributed query engine, that allows running analytical queries across different data sources via SQL. One of those data sources can be CrateDB and this article is going to look… --- ## [Automating stock data collection and storage with CrateDB and Apache Airflow](https://community.cratedb.com/t/automating-stock-data-collection-and-storage-with-cratedb-and-apache-airflow/990)
**Author:** [@rafaelasantana](https://community.cratedb.com/u/rafaelasantana)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 9:27am UTC](https://community.cratedb.com/t/automating-stock-data-collection-and-storage-with-cratedb-and-apache-airflow/990 "2022-01-19T09:27:48Z")
There is an updated version of this tutorial on Updating stock market data automatically with CrateDB and Apache Airflow Motivation Previously, on the Automating financial data collection and storage in CrateDB with P… --- ## [Workaround for connecting CrateDB and Apache Superset with HTTPS](https://community.cratedb.com/t/workaround-for-connecting-cratedb-and-apache-superset-with-https/969)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [January 4, 2022, 10:20am UTC](https://community.cratedb.com/t/workaround-for-connecting-cratedb-and-apache-superset-with-https/969 "2022-01-04T10:20:19Z")
:partying\_face: crate-python version 0.27.0 with a fix for https has been released It is now possible to connect to tls/ssl secured clusters with: crate://foo:bar@cratedb.example.org:4200/?ssl=true :scroll: crate-pyth… --- ## [Setting MinIO as a CrateDB backup repository](https://community.cratedb.com/t/setting-minio-as-a-cratedb-backup-repository/833)
**Author:** [@romanas667](https://community.cratedb.com/u/romanas667)\ **Replies:** 3\ **Last updated:** [December 21, 2021, 1:58pm UTC](https://community.cratedb.com/t/setting-minio-as-a-cratedb-backup-repository/833 "2021-12-21T13:58:56Z")
MinIO is a popular s3-compatible storage server, that can also be used as a CrateDB backup repository. Configuring MinIO for your backups is generally as straightforward as AWS S3, except for a couple of small details. … --- ## [Exploring Blob storage in CrateDB](https://community.cratedb.com/t/exploring-blob-storage-in-cratedb/938)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [December 13, 2021, 10:13am UTC](https://community.cratedb.com/t/exploring-blob-storage-in-cratedb/938 "2021-12-13T10:13:00Z")
CrateDB can be used to store binary large objects (Blobs). This allows you to store binary data (e.g. product photos, PDFs, etc) in a table which will be automatically sharded and replicated across your cluster. In thi… --- ## [Implementing a data retention policy in CrateDB using Apache Airflow](https://community.cratedb.com/t/implementing-a-data-retention-policy-in-cratedb-using-apache-airflow/913)
**Author:** [@Marija](https://community.cratedb.com/u/Marija)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 10:17am UTC](https://community.cratedb.com/t/implementing-a-data-retention-policy-in-cratedb-using-apache-airflow/913 "2021-11-25T10:17:55Z")
Written by Niklas Schmidtmer and Marija Selakovic What is a Data Retention Policy? A data retention policy describes the practice of storing and managing data for a designated period of time. Once a data set completes… --- ## [Connecting to CrateDB from Apache NiFi](https://community.cratedb.com/t/connecting-to-cratedb-from-apache-nifi/647)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 8\ **Last updated:** [October 8, 2021, 12:32pm UTC](https://community.cratedb.com/t/connecting-to-cratedb-from-apache-nifi/647 "2021-10-08T12:32:40Z")
This article describes how to connect from Apache NiFi to CrateDB and ingest data from NiFi into CrateDB. Prerequisites To follow this article, you will need: A CrateDB cluster An Apache NiFi installation that can c… --- ## [Deploying CrateDB to the cloud via Terraform](https://community.cratedb.com/t/deploying-cratedb-to-the-cloud-via-terraform/849)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [October 7, 2021, 2:52pm UTC](https://community.cratedb.com/t/deploying-cratedb-to-the-cloud-via-terraform/849 "2021-10-07T14:52:45Z")
CrateDB already has a wide variety of deployment options, ranging from Docker Compose to cloud setups on AWS and Azure. To make cloud setups more manageable and predictable, we today add another option using Terraform,… --- ## [Ingesting MQTT messages into CrateDB using Node-RED](https://community.cratedb.com/t/ingesting-mqtt-messages-into-cratedb-using-node-red/803)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [September 13, 2021, 2:10pm UTC](https://community.cratedb.com/t/ingesting-mqtt-messages-into-cratedb-using-node-red/803 "2021-09-13T14:10:36Z")
Node-RED is a workflow automation tool allowing to orchestrate message flows and transformations via a comfortable web interface. In this article, we will demonstrate the usage of Node-RED with CrateDB at the example … --- ## [Automating recurrent CrateDB queries](https://community.cratedb.com/t/automating-recurrent-cratedb-queries/788)
**Author:** [@hammerhead](https://community.cratedb.com/u/hammerhead)\ **Replies:** 0\ **Last updated:** [September 9, 2021, 6:43am UTC](https://community.cratedb.com/t/automating-recurrent-cratedb-queries/788 "2021-09-09T06:43:42Z")
Certain database operations can require running identical queries recurringly, creating the wish to make use of a scheduling mechanism. This article will discuss different existing scheduling solutions and how to inte… --- ## [Connecting to CrateDB with Java and JDBC](https://community.cratedb.com/t/connecting-to-cratedb-with-java-and-jdbc/786)
**Author:** [@rafaelasantana](https://community.cratedb.com/u/rafaelasantana)\ **Replies:** 0\ **Last updated:** [September 8, 2021, 11:33am UTC](https://community.cratedb.com/t/connecting-to-cratedb-with-java-and-jdbc/786 "2021-09-08T11:33:02Z")
In this tutorial, we will Install CrateDB Install Maven and create a simple Maven project Connect to CrateDB using the PostgreSQL-JDBC Driver Test the CrateDB connection and make queries Make an alternative connection… --- ## [User-Defined Function collection](https://community.cratedb.com/t/user-defined-function-collection/773)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [August 9, 2021, 11:52am UTC](https://community.cratedb.com/t/user-defined-function-collection/773 "2021-08-09T11:52:22Z")
CrateDB supports User-Defined Functions to extend the functionality provided out of the box with custom functionality. Feel free to use these UDFs for your own use case or as a starting point for creating your own UDF… --- ## [Introduction to Azure Databricks with CrateDB](https://community.cratedb.com/t/introduction-to-azure-databricks-with-cratedb/764)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [August 3, 2021, 2:08pm UTC](https://community.cratedb.com/t/introduction-to-azure-databricks-with-cratedb/764 "2021-08-03T14:08:09Z")
This is a quick intro into getting started with Azure Databricks and CrateDB. Setup Azure Databricks Add a new Databricks service to your Azure Subscription Once this is done use “Launch Workspace” After you are sign… --- ## [Connecting to CrateDB with Node.js](https://community.cratedb.com/t/connecting-to-cratedb-with-node-js/751)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [July 25, 2021, 10:00pm UTC](https://community.cratedb.com/t/connecting-to-cratedb-with-node-js/751 "2021-07-25T22:00:00Z")
This article describes how to connect to CrateDB from Node.js using the node-postgres driver. Prerequisites Install the node package to use the driver npm install pg Connecting to CrateDB Launch a free trial cluster… --- ## [Installing CrateDB on a Raspberry Pi and OpenSuse](https://community.cratedb.com/t/installing-cratedb-on-a-raspberry-pi-and-opensuse/700)
**Author:** [@inviridi](https://community.cratedb.com/u/inviridi)\ **Replies:** 0\ **Last updated:** [June 5, 2021, 6:12am UTC](https://community.cratedb.com/t/installing-cratedb-on-a-raspberry-pi-and-opensuse/700 "2021-06-05T06:12:03Z")
I ran into some difficulties installing Crate on my favourite Linux distros and after solving them, @proddata kindly asked to write a short tutorial about it. It is really simple Download Crate for AMD64 curl https://c… --- ## [Handling duplicate records](https://community.cratedb.com/t/handling-duplicate-records/667)
**Author:** [@jayeff](https://community.cratedb.com/u/jayeff)\ **Replies:** 0\ **Last updated:** [May 10, 2021, 8:10am UTC](https://community.cratedb.com/t/handling-duplicate-records/667 "2021-05-10T08:10:04Z")
Prevent duplicates Your first option is to prevent it from happening in the first place by defining a PRIMARY KEY for your tables. If you try to insert data with the same PRIMARY KEY a DuplicateKeyException is thrown. E… [Previous page](https://community.cratedb.com/c/tutorials/28.md?page=1) [Next page](https://community.cratedb.com/c/tutorials/28.md?page=3)