Google Cloud Storage repository plugin

CrateDB
1

Hello, I’m running a CrateDB server on GCP and I would like to know if there is a plugin to make snaptshots in Google Cloud Storage. From what I could see Elasticsearch has one but I doubt it works for Crate.

Thanks

2

Hi @lascarayf,

Thanks for your question. Unfortunately we do not support snapshots to Google Cloud Storage. CrateDB uses a different plugin mechanism than Elasticsearch.

However, you could of course try to adapt the Elasticsearch plugin and make it a community contribution!

3

I found the way to make CrateDB snapshots work with Google Cloud Platform Storage buckets.

There is s3 snapshots plugin by default installed in CrateDB. GCP s3 implementation is not fully compatible with it and if fails on the verification step when adding a new repository. To bypass this you can add it by Elastic Search API with a skipped verification flag on.

Steps:

  1. Start CrateDB with parameter: -Ces.api.enabled=true

  2. Add repository with API call:

    curl -X PUT “localhost:4200/_snapshot/repositoryname?pretty&verify=false” -H ‘Content-Type: application/json’ -d’
    {
    “type”: “s3”,
    “settings”: {
    “access_key”: “GOOG”,
    “secret_key”: “secret”,
    “endpoint”: “https://storage.googleapis.com”,
    “bucket”: “bucket_name”,
    “base_path”: “base_path”
    }
    }
    ’

  3. CREATE SNAPSHOT repositoryname.snapshot_name ALL;

  4. RESTORE SNAPSHOT repositoryname.snapshot_name ALL;

  5. access_key and secret_key you can obtain in Google Cloud Platform console in Storage > Settings > Interoperability > Access keys for service accounts