# I'm trying to set up a 3-node CrateDB cluster and now I can't even connect with crash **URL:** \ **Category:** CrateDB\ **Created:** [October 1, 2021, 10:22am UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839 "2021-10-01T10:22:47Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 1, 2021, 10:22am UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/1 "2021-10-01T10:22:47Z")
Hello, I’m trying to set up a 3-node CrateDB cluster and now I can’t even connect with `crash` now on Node 001. Yesterday, I started with the first node and I was able to connect to it via `crash`, and then I configured all of the nodes as the following: **Node 001** : ```auto administrator@crate-dn-001:~$ grep -v "^#" /etc/crate/crate.yml | sed '/^[[:space:]]*$/d' path.data: /data gateway.expected_nodes: 3 gateway.recover_after_nodes: 3 network.host: _site_, _local_ discovery.seed_hosts: - 192.168.239.30:4300 - 192.168.239.31:4300 - 192.168.239.32:4300 cluster.initial_master_nodes: - 192.168.239.30 - 192.168.239.31 - 192.168.239.32 auth.host_based.enabled: true auth: host_based: config: 0: user: crate address: _local_ method: trust 99: method: password node.name: crate-dn-001 ``` **Node 002** : ```auto administrator@crate-dn-002:~$ grep -v "^#" /etc/crate/crate.yml | sed '/^[[:space:]]*$/d' path.data: /data gateway.expected_nodes: 3 gateway.recover_after_nodes: 3 network.host: _site_, _local_ discovery.seed_hosts: - 192.168.239.30:4300 - 192.168.239.31:4300 - 192.168.239.32:4300 cluster.initial_master_nodes: - 192.168.239.30 - 192.168.239.31 - 192.168.239.32 auth.host_based.enabled: true auth: host_based: config: 0: user: crate address: _local_ method: trust 99: method: password node.name: crate-dn-002 ``` **Node 003** : ```auto administrator@crate-dn-003:~$ grep -v "^#" /etc/crate/crate.yml | sed '/^[[:space:]]*$/d' path.data: /data gateway.expected_nodes: 3 gateway.recover_after_nodes: 3 network.host: _site_, _local_ discovery.seed_hosts: - 192.168.239.30:4300 - 192.168.239.31:4300 - 192.168.239.32:4300 cluster.initial_master_nodes: - 192.168.239.30 - 192.168.239.31 - 192.168.239.32 auth.host_based.enabled: true auth: host_based: config: 0: user: crate address: _local_ method: trust 99: method: password node.name: crate-dn-003 ``` When I restart the nodes, and check the status, I get the following: **Node 001** : ```auto administrator@crate-dn-001:~$ sudo systemctl restart crate administrator@crate-dn-001:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:14:28 UTC; 3s ago Docs: https://crate.io/docs/ Main PID: 12421 (java) Tasks: 43 (limit: 36863) Memory: 1.0G CGroup: /system.slice/crate.service └─12421 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccu Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [io.crate.plugin. Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [io.crate.plugin. Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [io.crate.udc.plu Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [org.elasticsearc Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [org.elasticsearc Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [org.elasticsearc Oct 01 10:14:30 crate-dn-001 crate[12421]: [2021-10-01T10:14:30,425][INFO][o.e.p.PluginsService] [crate-dn-001] loaded plugin [org.elasticsearc Oct 01 10:14:31 crate-dn-001 crate[12421]: [2021-10-01T10:14:31,219][INFO][o.e.d.DiscoveryModule] [crate-dn-001] using discovery type [zen] and Oct 01 10:14:31 crate-dn-001 crate[12421]: [2021-10-01T10:14:31,857][INFO][psql] [crate-dn-001] PSQL SSL support is disabled. Oct 01 10:14:31 crate-dn-001 crate[12421]: [2021-10-01T10:14:31,955][INFO][i.c.p.PipelineRegistry] [crate-dn-001] HTTP SSL support is disabled. administrator@crate-dn-001:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:14:28 UTC; 1min 5s ago Docs: https://crate.io/docs/ Main PID: 12421 (java) Tasks: 61 (limit: 36863) Memory: 1.7G CGroup: /system.slice/crate.service └─12421 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccu Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,162][INFO][o.e.n.Node] [crate-dn-001] initialized Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,162][INFO][o.e.n.Node] [crate-dn-001] starting ... Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,292][INFO][psql] [crate-dn-001] publish_address {192.168.239.30 Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,304][INFO][o.e.h.n.Netty4HttpServerTransport] [crate-dn-001] publish_address {192.16 Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,315][INFO][o.e.t.TransportService] [crate-dn-001] publish_address {192.168.239.30 Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,458][INFO][o.e.b.BootstrapChecks] [crate-dn-001] bound or publishing to a non-lo Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,461][INFO][o.e.c.c.Coordinator] [crate-dn-001] cluster UUID [iXbkexKsQZyDNjMVU Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,577][INFO][o.e.c.s.MasterService] [crate-dn-001] elected-as-master ([1] nodes jo Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,624][INFO][o.e.c.s.ClusterApplierService] [crate-dn-001] master node changed {previo Oct 01 10:14:32 crate-dn-001 crate[12421]: [2021-10-01T10:14:32,634][INFO][o.e.n.Node] [crate-dn-001] started ``` **Node 002** : ```auto administrator@crate-dn-002:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:16:37 UTC; 39s ago Docs: https://crate.io/docs/ Main PID: 4317 (java) Tasks: 66 (limit: 36863) Memory: 1.4G CGroup: /system.slice/crate.service └─4317 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Xlog: Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,348][INFO][o.e.n.Node] [crate-dn-002] initialized Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,348][INFO][o.e.n.Node] [crate-dn-002] starting ... Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,488][INFO][psql] [crate-dn-002] publish_address {192.168.239.31:5432}, bound_addresse Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,502][INFO][o.e.h.n.Netty4HttpServerTransport] [crate-dn-002] publish_address {192.168.239.31:4200}, bound_ Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,517][INFO][o.e.t.TransportService] [crate-dn-002] publish_address {192.168.239.31:4300}, bound_addresse Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,705][INFO][o.e.b.BootstrapChecks] [crate-dn-002] bound or publishing to a non-loopback address, enforc Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,708][INFO][o.e.c.c.Coordinator] [crate-dn-002] cluster UUID [Eri8ChLNRz23zCUfJOGneA] Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,818][INFO][o.e.c.s.MasterService] [crate-dn-002] elected-as-master ([1] nodes joined)[{crate-dn-002}{z Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,861][INFO][o.e.c.s.ClusterApplierService] [crate-dn-002] master node changed {previous [], current [{crate Oct 01 10:16:41 crate-dn-002 crate[4317]: [2021-10-01T10:16:41,870][INFO][o.e.n.Node] [crate-dn-002] started ``` **Node 003** : ```auto administrator@crate-dn-003:~$ sudo systemctl restart crate administrator@crate-dn-003:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:18:04 UTC; 6s ago Docs: https://crate.io/docs/ Main PID: 4387 (java) Tasks: 67 (limit: 36863) Memory: 1.1G CGroup: /system.slice/crate.service └─4387 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Xlog:gc*,gc+age= Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,695][INFO][o.e.n.Node] [crate-dn-003] initialized Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,696][INFO][o.e.n.Node] [crate-dn-003] starting ... Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,823][INFO][psql] [crate-dn-003] publish_address {192.168.239.32:5432}, bound_addresses {[::1]:54 Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,835][INFO][o.e.h.n.Netty4HttpServerTransport] [crate-dn-003] publish_address {192.168.239.32:4200}, bound_addresses { Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,846][INFO][o.e.t.TransportService] [crate-dn-003] publish_address {192.168.239.32:4300}, bound_addresses {[::1]:43 Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,995][INFO][o.e.b.BootstrapChecks] [crate-dn-003] bound or publishing to a non-loopback address, enforcing bootstr Oct 01 10:18:07 crate-dn-003 crate[4387]: [2021-10-01T10:18:07,997][INFO][o.e.c.c.Coordinator] [crate-dn-003] cluster UUID [x7zTdQYtRKG9jOG6Cf4Jdw] Oct 01 10:18:08 crate-dn-003 crate[4387]: [2021-10-01T10:18:08,058][INFO][o.e.c.s.MasterService] [crate-dn-003] elected-as-master ([1] nodes joined)[{crate-dn-003}{tPM64bSrTeu8 Oct 01 10:18:08 crate-dn-003 crate[4387]: [2021-10-01T10:18:08,104][INFO][o.e.c.s.ClusterApplierService] [crate-dn-003] master node changed {previous [], current [{crate-dn-003}{tP Oct 01 10:18:08 crate-dn-003 crate[4387]: [2021-10-01T10:18:08,115][INFO][o.e.n.Node] [crate-dn-003] started ``` As far as I can see, it seems like each node is selecting **itself as the master** , is that correct? If so, then **why**? Also, on Node 001, I try to connect with `crash`, but unfortunately: ```auto administrator@crate-dn-001:~$ source /home/administrator/venv/cratedb/bin/activate (cratedb) administrator@crate-dn-001:~$ crash CONNECT ERROR (cratedb) administrator@crate-dn-001:~$ crash --verbose +-----------------------+-----------+---------+-----------+---------------------------------------+ | server_url | node_name | version | connected | message | +-----------------------+-----------+---------+-----------+---------------------------------------+ | http://localhost:4200 | NULL | 0.0.0 | FALSE | 503 Server Error: Service Unavailable | +-----------------------+-----------+---------+-----------+---------------------------------------+ CONNECT ERROR ``` Any ideas why `crash` stopped being able to connect locally? And is the cluster formed correctly with 3 nodes? Am I missing some configuration settings? ---
**Author:** ![proddata](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/proddata/32/1379_2.png) [@proddata](https://community.cratedb.com/u/proddata)\ **Post date:** [October 1, 2021, 10:32am UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/2 "2021-10-01T10:32:10Z")
> [@Emre\_Sevinc](#): > > As far as I can see, it seems like each node is selecting **itself as the master** , is that correct? If so, then **why**? You installed CrateDB using the **apt** packages, correct? If you do that, CrateDB starts automatically ins `single-node` mode which also prevents adding the nodes to any sort of cluster. This is intentional in a way, that apt packages should start without any additional confiuguration. There are basically two ways to fix this - using the `crate-node` tool or (easier) - deleting the cluster state by … - `service crate stop` - `rm -rf /usr/share/crate/data` - `service crate start` ---
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 1, 2021, 11:02am UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/3 "2021-10-01T11:02:21Z")
Yes, I’ve installed using apt on all of the 3 Debian GNU/Linux servers. Based on your recommendation I stopped crate service on all 3 servers and on all of them I ran: ```auto sudo rm -rf /usr/share/crate/data ``` Then I started the crate service on first, second and third node respectively, and this is what I get now: **Node 001** : ```auto administrator@crate-dn-001:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:54:33 UTC; 2min 5s ago Docs: https://crate.io/docs/ Main PID: 12692 (java) Tasks: 61 (limit: 36863) Memory: 1.9G CGroup: /system.slice/crate.service └─12692 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Xlog:gc*,gc+age=trac Oct 01 10:54:37 crate-dn-001 crate[12692]: [2021-10-01T10:54:37,922][INFO][o.e.n.Node] [crate-dn-001] initialized Oct 01 10:54:37 crate-dn-001 crate[12692]: [2021-10-01T10:54:37,923][INFO][o.e.n.Node] [crate-dn-001] starting ... Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,070][INFO][psql] [crate-dn-001] publish_address {192.168.239.30:5432}, bound_addresses {[::1]:5432}, Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,082][INFO][o.e.h.n.Netty4HttpServerTransport] [crate-dn-001] publish_address {192.168.239.30:4200}, bound_addresses {[::1 Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,096][INFO][o.e.t.TransportService] [crate-dn-001] publish_address {192.168.239.30:4300}, bound_addresses {[::1]:4300}, Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,269][INFO][o.e.b.BootstrapChecks] [crate-dn-001] bound or publishing to a non-loopback address, enforcing bootstrap c Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,270][INFO][o.e.c.c.Coordinator] [crate-dn-001] cluster UUID [iXbkexKsQZyDNjMVU6GD1A] Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,323][INFO][o.e.c.s.MasterService] [crate-dn-001] elected-as-master ([1] nodes joined)[{crate-dn-001}{pp1q4bZ5Tk6m88xu Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,371][INFO][o.e.c.s.ClusterApplierService] [crate-dn-001] master node changed {previous [], current [{crate-dn-001}{pp1q4b Oct 01 10:54:38 crate-dn-001 crate[12692]: [2021-10-01T10:54:38,381][INFO][o.e.n.Node] [crate-dn-001] started ``` **Node 002** : ```auto administrator@crate-dn-002:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:54:40 UTC; 5min ago Docs: https://crate.io/docs/ Main PID: 4538 (java) Tasks: 61 (limit: 36863) Memory: 3.5G CGroup: /system.slice/crate.service └─4538 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Xlog:gc*,gc+ Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,441][INFO][o.e.n.Node] [crate-dn-002] initialized Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,441][INFO][o.e.n.Node] [crate-dn-002] starting ... Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,563][INFO][psql] [crate-dn-002] publish_address {192.168.239.31:5432}, bound_addresses {[::1 Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,575][INFO][o.e.h.n.Netty4HttpServerTransport] [crate-dn-002] publish_address {192.168.239.31:4200}, bound_address Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,588][INFO][o.e.t.TransportService] [crate-dn-002] publish_address {192.168.239.31:4300}, bound_addresses {[::1 Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,746][INFO][o.e.b.BootstrapChecks] [crate-dn-002] bound or publishing to a non-loopback address, enforcing boo Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,748][INFO][o.e.c.c.Coordinator] [crate-dn-002] cluster UUID [Eri8ChLNRz23zCUfJOGneA] Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,819][INFO][o.e.c.s.MasterService] [crate-dn-002] elected-as-master ([1] nodes joined)[{crate-dn-002}{zaiVJzVU Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,864][INFO][o.e.c.s.ClusterApplierService] [crate-dn-002] master node changed {previous [], current [{crate-dn-002 Oct 01 10:54:44 crate-dn-002 crate[4538]: [2021-10-01T10:54:44,874][INFO][o.e.n.Node] [crate-dn-002] started ``` **Node 003** : ```auto administrator@crate-dn-003:~$ sudo systemctl status crate ● crate.service - CrateDB Server Loaded: loaded (/lib/systemd/system/crate.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2021-10-01 10:54:46 UTC; 6min ago Docs: https://crate.io/docs/ Main PID: 4606 (java) Tasks: 62 (limit: 36863) Memory: 5.0G CGroup: /system.slice/crate.service └─4606 /usr/share/crate/jdk/bin/java -Xms16G -Xmx16G -Djava.awt.headless=true -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Xlog:gc*,gc+age=trac Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,033][INFO][o.e.n.Node] [crate-dn-003] initialized Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,033][INFO][o.e.n.Node] [crate-dn-003] starting ... Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,174][INFO][psql] [crate-dn-003] publish_address {192.168.239.32:5432}, bound_addresses {[::1]:5432}, Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,188][INFO][o.e.h.n.Netty4HttpServerTransport] [crate-dn-003] publish_address {192.168.239.32:4200}, bound_addresses {[::1 Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,201][INFO][o.e.t.TransportService] [crate-dn-003] publish_address {192.168.239.32:4300}, bound_addresses {[::1]:4300}, Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,372][INFO][o.e.b.BootstrapChecks] [crate-dn-003] bound or publishing to a non-loopback address, enforcing bootstrap c Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,374][INFO][o.e.c.c.Coordinator] [crate-dn-003] cluster UUID [x7zTdQYtRKG9jOG6Cf4Jdw] Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,497][INFO][o.e.c.s.MasterService] [crate-dn-003] elected-as-master ([1] nodes joined)[{crate-dn-003}{tPM64bSrTeu8a0Jm Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,546][INFO][o.e.c.s.ClusterApplierService] [crate-dn-003] master node changed {previous [], current [{crate-dn-003}{tPM64b Oct 01 10:54:50 crate-dn-003 crate[4606]: [2021-10-01T10:54:50,555][INFO][o.e.n.Node] [crate-dn-003] started ``` ---
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 1, 2021, 11:15am UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/4 "2021-10-01T11:15:06Z")
And I still can’t connect with `crash` locally on the first node (I was able to connect locally before I started setting up configuration for 3-node cluster): ```auto administrator@crate-dn-001:~$ source /home/administrator/venv/cratedb/bin/activate (cratedb) administrator@crate-dn-001:~$ crash CONNECT ERROR (cratedb) administrator@crate-dn-001:~$ crash --verbose +-----------------------+-----------+---------+-----------+---------------------------------------+ | server_url | node_name | version | connected | message | +-----------------------+-----------+---------+-----------+---------------------------------------+ | http://localhost:4200 | NULL | 0.0.0 | FALSE | 503 Server Error: Service Unavailable | +-----------------------+-----------+---------+-----------+---------------------------------------+ CONNECT ERROR ``` The non-commented contents of `/etc/crate/crate.yml` on that node are: ```auto administrator@crate-dn-001:~$ grep -v "^#" /etc/crate/crate.yml | sed '/^[[:space:]]*$/d' path.data: /data gateway.expected_nodes: 3 gateway.recover_after_nodes: 3 network.host: _site_, _local_ discovery.seed_hosts: - 192.168.239.30:4300 - 192.168.239.31:4300 - 192.168.239.32:4300 cluster.initial_master_nodes: - 192.168.239.30 - 192.168.239.31 - 192.168.239.32 auth.host_based.enabled: true auth: host_based: config: 0: user: crate address: _local_ method: trust 99: method: password node.name: crate-dn-001 ``` ---
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 1, 2021, 12:49pm UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/5 "2021-10-01T12:49:27Z")
I don’t know how this affects forming of the cluster but this morning, I first set `cluster.name: crate-energytix` in `/etc/crate/crate.yml` and tried to bring up nodes this way. But then I commented it out, so the default cluster name should be `crate` again, right? It would be great if I could see a sample set of YAML files indicating the simplest 3-node cluster set up for an on-premise system that is installed via stable APT packages of CrateDB. Right now I have 3 powerful Debian GNU/Linux servers up and running, but whatever the configuration problems are with those YAML files on those servers, I end up with the following: - It seems like a 3-node CrateDB cluster is not formed. - I can’t connect with `crash` anymore (neither locally with the default `crate` user, nor remotely with the `energytix` user I created yesterday). - I’ve never been able to connect via a web browser to Admin UI. Therefore I don’t know what else I should try. Any ideas? ---
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 1, 2021, 1:58pm UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/6 "2021-10-01T13:58:41Z")
Apparently, I also had to delete whatever cluster state has been created under `/data` directory. After stopping the service, running `rm -rf *` in `/data`, and restarting the CrateDB service on all three nodes, they now see each other and form a cluster. I’m also able to connect locally with `crash`. I created the same `energytix` user again, because apparently it was deleted when I deleted the cluster state under `data` directories. I can also connect remotely via `crash`. The only remaining issue I have for now is that I still can’t connect to the Admin UI when I visit [http://192.168.239.30:4200](http://192.168.239.30:4200) from another machine (an MS Windows machine running Chrome browser, that machine can normally access 192.168.239.30). ---
**Author:** ![proddata](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/proddata/32/1379_2.png) [@proddata](https://community.cratedb.com/u/proddata)\ **Post date:** [October 1, 2021, 2:04pm UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/7 "2021-10-01T14:04:28Z")
Thanks for reporting back. > I can also connect remotely via `crash` . > The only remaining issue I have for now is that I still can’t connect to the Admin UI when I visit [http://192.168.239.30:4200](http://192.168.239.30:4200/) from another machine (an MS Windows machine running Chrome browser, that machine can normally access 192.168.239.30). If you can connect remotely via **crash** , then you should also be able to see the Admin UI **crash** uses the http-endpoint which serves the AdminUI `GET / ` → AdminUI `POST /_sql ` → sql http endpoint (used by both crash and the AdminUI) ---
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 1, 2021, 2:17pm UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/8 "2021-10-01T14:17:29Z")
Thanks for the feedback. To test this I installed `crash` on that MS Windows machine, and indeed, it seems like there’s some connectivity problem from that MS Windows machine to the CrateDB machine’s port **4200** because the following command ```auto (cratedb) C:\Users\esevinc>crash --verbose --hosts 192.168.239.30 --username energytix ``` just hangs without any output. Probably some sort of firewall issue, I’ll talk to our network team. ---
**Author:** ![Emre\_Sevinc](https://sea2.discourse-cdn.com/flex020/user_avatar/community.cratedb.com/emre_sevinc/32/375_2.png) [@Emre\_Sevinc](https://community.cratedb.com/u/Emre_Sevinc)\ **Post date:** [October 4, 2021, 10:19am UTC](https://community.cratedb.com/t/im-trying-to-set-up-a-3-node-cratedb-cluster-and-now-i-cant-even-connect-with-crash/839/9 "2021-10-04T10:19:36Z")
Connection to Admin UI issue has been solved as described at: [I can connect via crash remotely, but I can't access Admin UI via web browser, why? - #2 by Emre\_Sevinc](https://community.cratedb.com/t/i-can-connect-via-crash-remotely-but-i-cant-access-admin-ui-via-web-browser-why/838/2)