How to create a Crate Cluster from two node

CrateDB Configuration
1

Hi, Guys.

I know this is supposed to be simple, but I’m having trouble with it. (backstory) we have a crate node on one server and we want to create a second node on a different server to create a cluster of 2 nodes. The reason for 2 notes is that we use crate to process and temporarily store data for the day, which is used by other platforms for calculations, once they are done, other scripts run to clear the tables for the next day. so far we are happy with one node, but we would like to run another one a redundancy. and daily records are around 20-30 million in 60 tables.

1 Like
2

Hi,
Please check the configuration file (typically /etc/crate/crate.yml) of the existing CrateDB server, if the node was started without specifying the initial_master_nodes setting (the default configuration), or with discovery_type set to single-node, it would have been started as a standalone instance and there is not clean way to scale it into a cluster.
If you are in that situation I would advise setting up the new cluster in parallel, taking a snapshot and then restoring it in the new cluster.

If initial_master_nodes was set, then you can add nodes to that setup.
There are many settings you can configure, but the basic steps are to prepare a new node installing CrateDB but making sure it does not auto-boostrap as a single-node instance, you may want to either create /etc/crate/crate.yml using this a template in advance or use a mechanism like policy-rcd-declarative.
Set discovery.seed_hosts in the configuration file to the full list of nodes, including the new one you are adding, and start CrateDB, if it can reach the other node on TCP port 4300 it will join the cluster.

Please note that once a cluster has 2 nodes it cannot operate with a single node as it could not know for sure if the other node is alive in a network partition and the data has moved forward, if you are looking for high availability we advise to have a minimum of 3 nodes.

I hope this helps.

2 Likes
3

Hi @hernanc ,

I have a cratedb server running as a docker container.
The server was created as a single-node,
the command used in docker compose is :
command: ["crate","-Cdiscovery.type=single-node"]

It’s not clear to me how I can scale to two or more servers from this setup.

Would it be possible to have an example of settings to use in two separate docker composes (on two different hosts)?

Thx.

Luca

4

Hi @cocoa

A cluster started with discovery.type=single-node unfortunately will never join a multi-node cluster.

Would it be possible to have an example of settings to use in two separate docker composes (on two different hosts)?

Are we talking about a Docker swarm setup or just two independent hosts?
A docker swarm example can be found here:

5

Hi @proddata,

my mistake in translating the answer about setting up with single-node.

Thanks for the docker swarm example.

At this point I would need to understand if it is possible to start with a single host that is able to scale later or if it is necessary to start from the beginning, for a scalable solution, with three CrateDB servers on three different hosts.

Out of curiosity, it is possible to start with an installation with three cratedb servers on single hosts as the docker compose example here and then migrate two of these services on two different hosts?

Thx,

Luca

6

At this point I would need to understand if it is possible to start with a single host that is able to scale later or if it is necessary to start from the beginning, for a scalable solution, with three CrateDB servers on three different hosts.

Yes, you can start with a single CrateDB node and add more nodes later.

To quickly try it:

  • Create a network:

    docker network create cratenet

  • Start one node (with an initial master node):

    docker run --name crate-host-01 --net cratenet -p 4201:4200 crate -Cnode.name=node-01 -Ccluster.initial_master_nodes=node-01

  • Start a 2nd node (with the initial node in the seed_hosts):

    docker run --name crate-host-02 --net cratenet -p 4202:4200 crate -Cnode.name=node-02 - Cdiscovery.seed_hosts=crate-host-01

Out of curiosity, it is possible to start with an installation with three cratedb servers on single hosts as the docker compose example here and then migrate two of these services on two different hosts?

In theory, yes. The CrateDB nodes need to see each other on the transport port (default 4300)

1 Like
7

Thanks for the hint, this is the solution to my question.

Luca

8

Please note that once a cluster has 2 nodes it cannot operate with a single node as it could not know for sure if the other node is alive in a network partition and the data has moved forward, if you are looking for high availability we advise to have a minimum of 3 nodes.

Am setting up my first cluster, have 2 nodes defined, but due to networking issues currently neither can see each other. The above would explain why I’m not able to perform any operations (e.g. crate a new user etc) on the master node I’ve defined, or do I have another issue?

9

have 2 nodes defined, but due to networking issues currently neither can see each other. The above would explain why I’m not able to perform any operations (e.g. crate a new user etc) on the master node I’ve defined, or do I have another issue?

If there were only 2 nodes and they cannot see each other anymore on the network they will stop serving requests. With 3 nodes it would be different. To make sure the issue is a network partition and not something else I recommend you check the logs, you can use docker logs for this if you are running CrateDB on containers.
If the messages you see on the logs are not clear feel free to share them here and we can comment on those.

10

Great, thanks for confirming.