Setting MinIO as a CrateDB backup repository

Tutorials
,
1

MinIO is a popular s3-compatible storage server, that can also be used as a CrateDB backup repository.

Configuring MinIO for your backups is generally as straightforward as AWS S3, except for a couple of small details.

  1. First, follow the official MinIO tutorial to install it on the platform of your choice.
  2. Create a tenant, a bucket and a user that can access your bucket. Take note of the credentials.
  3. For CrateDB to access MinIO correctly, you must have bucketDns enabled (your bucket must be reachable via DNS). If that is not possible in your setup, you can fall back and use the IP address of your MinIO server instead.
  4. Create a new repository in your CrateDB Cluster:
    CREATE REPOSITORY minio TYPE s3 WITH (
  access_key = 'test', 
  secret_key = 'testsecret', 
  endpoint   = '10.1.3.7', 
  protocol   = 'http', 
  bucket     = 'test-bucket', 
  base_path  = 'backups'
);
  5. CrateDB will perform a connectivity test and tell you if the bucket is reachable.
  6. Create a new snapshot by running:
    CREATE SNAPSHOT minio.testbackup ALL WITH (wait_for_completion = true);

Note: if you want to use https to perform backups, then bucketDns must be enabled and your MinIO server must have a valid SSL certificate.

4 Likes
2

Hello @romanas667

thank you for your tutorual.

I try to set up a minio repo over https.

CREATE REPOSITORY minio_1 TYPE s3 WITH (
  access_key = 'MI0hgdReslC2PPZAKQV6K1VI6I',
  secret_key = '...',
  endpoint   = 'api.minio.mydomain.org',
  protocol   = 'https',
  bucket     = 'staging',
  base_path  = 'backups'
);

But it fails with the error

RepositoryVerificationException[[minio_1] Unable to verify the repository, [minio_1] is not accessible on master node: SdkClientException 'Unable to execute HTTP request: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target']

The certificate is issued by let’s encrypt authority and seems to be valid

image
image540×621 36.4 KB

Do you have any idea why connections can’t be established to my minio s3 repo?

3

Hi @Max_Becker,

Can you verify you can curl/wget the minio address over HTTPS from the same host that your CrateDB is running? If that is not working either, you might be missing CA certificates.

4

@romanas667 thank you for quick response.

curl can access the minio address from cratedb host. So far so good.

Just like you I suspect that cratedb cannot resolve the certificate chain.
I checked /crate/jdk/lib/security/cacerts within docker. It only has X1 from Let’s Encrypt. My issued certificate was signed by R3. I think it must be added to /crate/jdk/lib/security/cacerts too.

I’ll try it.