Translogs settings globally

CrateDB
1

Hi,

After a while I’ve found that the translogs are filling up free disk space, and eventually the disk.watermark.low/high gets triggered.
I see the index.translog.flush_threshold_size in the crate.yml file, and in the description it states:

In crate.yml:

You can set a number of options (such as replica options, mapping
or analyzer definitions, translog settings, …) for tables globally,
in this file.

But when I do that, I get the following exception on startup:

[2019-07-22T11:44:02,615][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [Aiguille du Fruit] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: node settings must not contain any index level settings
at org.elasticsearch.bootstrap.StartupExceptionProxy.<init>(StartupExceptionProxy.java:31) ~[crate-app.jar:3.3.4]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:159) ~[crate-app.jar:3.3.4]
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:135) ~[crate-app.jar:3.3.4]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:85) ~[crate-app.jar:3.3.4]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.5.1.jar:3.3.4]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.5.1.jar:3.3.4]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:88) ~[crate-app.jar:3.3.4]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:81) ~[crate-app.jar:3.3.4]
Caused by: java.lang.IllegalArgumentException: node settings must not contain any index level settings
at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:142) ~[crate-app.jar:3.3.4]
at org.elasticsearch.node.Node.<init>(Node.java:361) ~[crate-app.jar:3.3.4]
at io.crate.node.CrateNode.<init>(CrateNode.java:65) ~[crate-app.jar:3.3.4]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:193) ~[crate-app.jar:3.3.4]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:193) ~[crate-app.jar:3.3.4]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:264) ~[crate-app.jar:3.3.4]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) ~[crate-app.jar:3.3.4]
... 6 more

Ideally I wouldn’t want to have to change my “CREATE TABLE” scripts, so how can I set the translog limitations globally?

Thanks

2

Since CrateDB 2.0.0, it is no longer possible to set generic table properties via crate.yml or CLI arguments. Each property must be set on each concrete table using CREATE TABLE or ALTER TABLE commands.
Looks like we missed to remove these lines from the default crate.yml, we’ll follow up on this, thanks for mentioning!

3

Hi,

One more question plz, does the translog setting also limit the gc.log?
If not, is there a way to limit those too?

Thanks

4

Hi, apologies this answer comes in a bit late, but I just realise this question was unanswered and someone else could come across it in the future.
You can control GC logging with environment variables, take a look at Logging — CrateDB: Reference for details