Search code examples
dockerelasticsearchdocker-composeapple-silicon

Run Elasticsearch 5.2.0 on Docker Desktop

I am pretty new with Docker and I have an existing application that I needed to dockerize and run first locally.

The existing, non-docker, setup uses Ubuntu 18.04 (upgraded from 16.04) + Elasticsearch 5.2.0, MongoDB 4.2.8, Java 8, Tomcat (my-app is running) and works perfectly.

My goal is to dockerize all this and use 1 docker compose file only.

My application is using org.elasticsearch.elasticsearch library (5.2.0 also) and it should still be the same version as other older projects are using the same common library using version 5.2.0.

My problem is my docker-ES is not running properly and it seems to be compatibility issues (based on Elastic Support Matrix).

I tried 2 setups:

  1. On MacOS Ventura with MBP M1
  2. On Windows 11 on Core i7 9th Gen laptop

At first, I though M1 processor support (arm64) is the issue, but when I run the same docker compose yml file on Windows... I get the same issue.

I can see elasticsearch version 5.2.0 in hub docker site and it only supports platform linux/amd64. So I initially think that it will only work on Windows x64 and not on my mac M1 laptop. But it does not work on both.

Below are the error I got:

2023-08-08 16:22:57 [2023-08-08T08:22:57,087][INFO ][o.e.n.Node               ] [es01] initializing ...
2023-08-08 16:22:57 [2023-08-08T08:22:57,199][INFO ][o.e.e.NodeEnvironment    ] [es01] using [1] data paths, mounts [[/ (overlay)]], net usable_space [233gb], net total_space [250.9gb], spins? [possibly], types [overlay]
2023-08-08 16:22:57 [2023-08-08T08:22:57,199][INFO ][o.e.e.NodeEnvironment    ] [es01] heap size [1.9gb], compressed ordinary object pointers [true]
2023-08-08 16:22:57 [2023-08-08T08:22:57,202][INFO ][o.e.n.Node               ] [es01] node name [es01], node ID [6LH5fzjMRBSvg5c1JpC1Pw]
2023-08-08 16:22:57 [2023-08-08T08:22:57,206][INFO ][o.e.n.Node               ] [es01] version[5.2.0], pid[1], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/5.10.16.3-microsoft-standard-WSL2/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
2023-08-08 16:22:59 [2023-08-08T08:22:59,630][INFO ][o.e.p.PluginsService     ] [es01] loaded module [aggs-matrix-stats]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [ingest-common]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [lang-expression]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [lang-groovy]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [lang-mustache]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [lang-painless]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [percolator]
2023-08-08 16:22:59 [2023-08-08T08:22:59,631][INFO ][o.e.p.PluginsService     ] [es01] loaded module [reindex]
2023-08-08 16:22:59 [2023-08-08T08:22:59,632][INFO ][o.e.p.PluginsService     ] [es01] loaded module [transport-netty3]
2023-08-08 16:22:59 [2023-08-08T08:22:59,632][INFO ][o.e.p.PluginsService     ] [es01] loaded module [transport-netty4]
2023-08-08 16:22:59 [2023-08-08T08:22:59,633][INFO ][o.e.p.PluginsService     ] [es01] loaded plugin [x-pack]
2023-08-08 16:23:00 [2023-08-08T08:23:00,391][WARN ][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
2023-08-08 16:23:01 [2023-08-08T08:23:01,237][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [es01] uncaught exception in thread [main]
2023-08-08 16:23:01 org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: No match found
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01 Caused by: java.lang.IllegalStateException: No match found
2023-08-08 16:23:01     at java.util.regex.Matcher.group(Matcher.java:536) ~[?:1.8.0_92-internal]
2023-08-08 16:23:01     at org.elasticsearch.monitor.os.OsProbe.getControlGroups(OsProbe.java:216) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:414) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:466) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.monitor.os.OsService.<init>(OsService.java:45) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:45) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.node.Node.<init>(Node.java:345) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.node.Node.<init>(Node.java:232) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:241) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.2.0.jar:5.2.0]
2023-08-08 16:23:01     ... 6 more

Below is my docker-compose YAML file:

version: '3.8'
services:
  mongodb:
    image: 'mongo:4.2.8'
    container_name: 'my_db_container'
    ports:
      - 27017:27017
    entrypoint: [ "/usr/bin/mongod", "--bind_ip_all" ]
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.2.0
    platform: linux/amd64
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - discovery.type=single-node
      - node.name=es01
      - cluster.name=my-es-local
     # - bootstrap.system_call_filter=false
  app:
    build: .
    ports:
      - 8080:8080
      - 8000:8000
      - 9090:9090
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./target/order-0.0.1-SNAPSHOT/:/opt/apache-tomcat-8.5.82/webapps/ROOT/
    depends_on:
      - mongodb
      - elasticsearch
    environment:
      CATALINA_OPTS: "$CATALINA_OPTS -Dlogs.path.location=D:/my-logs/dev/logs/"
      JPDA_ADDRESS: 8000
      JPDA_TRANSPORT: dt_socket

With this YAML, my docker MongoDB runs fine, but my docker ES does not run and gets that error.

It seems my docker desktop should be running on ubuntu 16.04 lts so that docker elasticsearch 5.2.0 will run, but can I do this if I am using both Windows 11 x64 and MacOS Ventura M1?

After making my docker ES run, I would need to sync some of my DB data to ES in which I cannot proceed because of this error.

Update (Solution that works for me):
I used the image: docker.elastic.co/elasticsearch/elasticsearch:5.6.3 and it works using the same elasticsearch library (org.elasticsearch.elasticsearch v5.2.0) that my old app is using.

After ES docker was running and since my dev setup was only a single node and set a specific cluster name, I just set client.transport.sniff=false. After changing these, my application was able to connect to my docker ES.

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.6.3
    platform: linux/amd64
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - discovery.type=single-node
      - node.name=es01
      - cluster.name=my-es-local
      - xpack.security.enabled=false
      - http.publish_host=localhost
      - bootstrap.system_call_filter=false

This works both on my MBP M1 (MacOS Ventura) and Windows Laptop (Windows 11 64-bit)

Solution

  • Just Added this solution in-case someone have the same issue/case.

    I used the image: docker.elastic.co/elasticsearch/elasticsearch:5.6.3 and it works using the same elasticsearch library (org.elasticsearch.elasticsearch v5.2.0) that my old app is using.

    After ES docker was running and since my dev setup was only a single node and set a specific cluster name, I just set client.transport.sniff=false. After changing these, my application was able to connect to my docker ES.

      elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.6.3
    platform: linux/amd64
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - discovery.type=single-node
      - node.name=es01
      - cluster.name=my-es-local
      - xpack.security.enabled=false
      - http.publish_host=localhost
      - bootstrap.system_call_filter=false

    This works both on my MBP M1 (MacOS Ventura) and Windows Laptop (Windows 11 64-bit)