Setting Azure blob storage as snapshot repository for CrateDB

Tutorials
,
1
  1. Create a new Storage Account in Microsoft Azure
  2. Create a new Container in the Storage Account
  3. Get you Access keys for the Container (Security + networking > Access Keys)
  4. In your CrateDB Cluster create a new repository
    • You will need:
      • Storage Account name (e.g. cratetest)
      • Container name (e.g. cratebackup)
      • Access Key (e.g. /YDcp3IIJz4DIgSgWEGA3T8NhfOOCrm6EaDLaeqycgyZEXOg==)
    • Create the repo:
    CREATE REPOSITORY azurerepo TYPE azure
WITH (account = 'cratetest',
      container = 'cratebackup',
      key ='/YDcp3IIJz4DIgSgWEGA3T8NhfOOCrm6EaDLaeqycgyZEXOg==');
  5. Create your first snapshot
    CREATE SNAPSHOT azurerepo.first_snapshot TABLE my_table;

For the full configuration options see CREATE REPOSITORY — CrateDB: Reference
For more information on snapshots and repositories see Snapshots — CrateDB: Reference

5 Likes
Unable to create a repository (CrateDB cluster running with docker containers) [SOLVED]
2

Hi. I wanted to explore this feature in a testbed against an Azurite instance to support the SNAPSHOT policy of CrateDB Toolkit’s retention and expiration subsystem.

It took me a while to figure out that I must actually use IP-style URI addressing and a few additional cycles how to map this to the right incantation of CrateDB’s CREATE REPOSITORY command, specifically its endpoint parameter.

First, create a storage container called example, for example using azure-cli [1].

export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite:10000/devstoreaccount1;QueueEndpoint=http://azurite:10001/devstoreaccount1;TableEndpoint=http://azurite:10002/devstoreaccount1;"
uv run --with='azure-cli==2.81' az storage container create --public-access=container --name=example

Then, create/register the repository in CrateDB, for example using crash.

CREATE REPOSITORY
    testrepo
TYPE
    azure
WITH (
    endpoint   = 'http://azurite:10000/devstoreaccount1',
    account    = 'devstoreaccount1',
    key        = 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==',
    container  = 'example'
);

The whole setup is wrapped up into the attached compose.yaml file, to be invoked with Docker Compose or Podman Compose.

compose.yaml (2.8 KB)

After downloading the file to your machine, invoke the service definition.

export BUILDKIT_PROGRESS=plain
docker compose up

Then, invoke the CREATE REPOSITORY command against CrateDB.

docker compose run --rm repository

  1. As of 2026-02-12, please use azure-cli 2.81 with current Azurite ~3.35. ↩︎