03 Installation of Elastic Search and Kibana

03 Installation: ElasticSearch and Kibana Installation #

在开始使用ElasticSearch和Kibana之前，我们首先需要将它们安装在我们的计算机上。

ElasticSearch和Kibana都可以在官方网站上下载到。你可以选择适合你操作系统的压缩包来进行安装。

ElasticSearch安装 #

  1. 访问ElasticSearch的官方网站： https://www.elastic.co/downloads/elasticsearch

  2. 下载与你操作系统对应的压缩包。

  3. 解压缩下载的压缩包至你想要安装ElasticSearch的位置。

  4. 打开终端或命令提示符窗口，导航至ElasticSearch的安装目录。

  5. 运行以下命令以启动ElasticSearch：

./bin/elasticsearch
  1. ElasticSearch会在默认端口9200上启动。

Kibana安装 #

  1. 访问Kibana的官方网站： https://www.elastic.co/downloads/kibana

  2. 下载与你操作系统对应的压缩包。

  3. 解压缩下载的压缩包至你想要安装Kibana的位置。

  4. 打开终端或命令提示符窗口，导航至Kibana的安装目录。

  5. 打开config/kibana.yml文件，并编辑以下设置：

elasticsearch.hosts: ["http://localhost:9200"]

确保elasticsearch.hosts的值与ElasticSearch的位置保持一致。

  1. 运行以下命令以启动Kibana：
./bin/kibana
  1. Kibana会在默认端口5601上启动。

现在，我们已经完成了ElasticSearch和Kibana的安装过程。在下一节中，我们将学习如何配置它们以使它们能够正常工作。

Installing ElasticSearch #

ElasticSearch is based on the Java platform, so Java needs to be installed first.

  • Platform Verification

Here I have prepared a CentOS 7 virtual machine. To facilitate the selection of the subsequent installation version, we need to check the system version information.

[root@VM-0-14-centos ~]# uname -a
Linux VM-0-14-centos 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Installing Java

Before installing Elasticsearch, you need to install a newer version of Java. The best choice is to get the latest official version of Java from www.java.com. After installation, confirm if it has been installed successfully.

[root@VM-0-14-centos ~]# java --version
openjdk 14.0.2 2020-07-14
OpenJDK Runtime Environment 20.3 (slowdebug build 14.0.2+12)
OpenJDK 64-Bit Server VM 20.3 (slowdebug build 14.0.2+12, mixed mode, sharing)
  • Downloading ElasticSearch

Download ElasticSearch from here.

For example, you can download it using curl.

[root@VM-0-14-centos opt]# curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  • Unpacking
[root@VM-0-14-centos opt]# tar zxvf /opt/elasticsearch-7.12.0-linux-x86_64.tar.gz
...
[root@VM-0-14-centos opt]# ll | grep elasticsearch
drwxr-xr-x  9 root root      4096 Mar 18 14:21 elasticsearch-7.12.0
-rw-r--r--  1 root root 327497331 Apr  5 21:05 elasticsearch-7.12.0-linux-x86_64.tar.gz
  • Adding elasticSearch User

You must create a non-root user to run ElasticSearch (starting from ElasticSearch 5, for security reasons, running as root is strictly forbidden).

If you use the root user to start ElasticSearch, you will receive the following error message:

[root@VM-0-14-centos opt]# cd elasticsearch-7.12.0/
[root@VM-0-14-centos elasticsearch-7.12.0]# ./bin/elasticsearch
[2021-04-05T21:36:46,510][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [VM-0-14-centos] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[elasticsearch-cli-7.12.0.jar:7.12.0]
        at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-7.12.0.jar:7.12.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-7.12.0.jar:7.12.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:101) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:168) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397) ~[elasticsearch-7.12.0.jar:7.12.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.12.0.jar:7.12.0]
        ... 6 more
uncaught exception in thread [main]
java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:101)
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:168)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397)
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
        at org.elasticsearch.cli.Command.main(Command.java:79)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)
For complete error details, refer to the log at /opt/elasticsearch-7.12.0/logs/elasticsearch.log
2021-04-05 13:36:46,979269 UTC [8846] INFO  Main.cc@106 Parent process died - ML controller exiting

So we add a separate Elasticsearch user to run as follows:

# Add elasticsearch user
[root@VM-0-14-centos elasticsearch-7.12.0]# useradd elasticsearch
[root@VM-0-14-centos elasticsearch-7.12.0]# passwd elasticsearch
Changing password for user elasticsearch.
New password: 
BAD PASSWORD: The password contains the user name in some form
Retype new password: 
passwd: all authentication tokens updated successfully.

# Modify directory permissions for the newly added elasticsearch user
[root@VM-0-14-centos elasticsearch-7.12.0]# chown -R elasticsearch /opt/elasticsearch-7.12.0
# Add storage areas for data and logs and grant permissions to the elasticsearch user
[root@VM-0-14-centos elasticsearch-7.12.0]# mkdir -p /data/es
[root@VM-0-14-centos elasticsearch-7.12.0]# chown -R elasticsearch /data/es
[root@VM-0-14-centos elasticsearch-7.12.0]# mkdir -p /var/log/es
[root@VM-0-14-centos elasticsearch-7.12.0]# chown -R elasticsearch /var/log/es

Then modify the data and log paths in the ‘vi /opt/elasticsearch-7.12.0/config/elasticsearch.yml’ file as follows:

# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data/es
#
# Path to log files:
#
path.logs: /var/log/es

  • Modify Linux system limits settings:

    1. Change the maximum number of files and other limitations on applications in the system. By default, Linux limits the maximum number of files an application can create to 65535. However, Elasticsearch requires at least 65536 file creation permissions.
    2. Change the maximum number of threads a user is allowed to start in the system. By default, Linux limits the number of threads a user can start to 1024 for non-root users, while root users can start an unlimited number of threads. It is necessary to change this limit to 4096 or more, as Elasticsearch requires at least 4096 thread pool reserves. Starting from Elasticsearch version 5.x, it is mandatory to run Elasticsearch as a non-root user in Linux.
    3. For low-version Linux kernels, the memory allocated for threads is 128K. Higher version kernels, such as the 4.x version, allocate more memory. If the virtual machine has only 1GB of memory, it can support a maximum of about 3000+ threads. It is recommended to allocate at least 1.5GB of memory to the virtual machine to accommodate more threads.

Modify the following configuration:

[root@VM-0-14-centos elasticsearch-7.12.0]# vi /etc/security/limits.conf

elasticsearch soft nofile 65536
elasticsearch hard nofile 65536
elasticsearch soft nproc 4096
elasticsearch hard nproc 4096
  • Start Elasticsearch:
[root@VM-0-14-centos elasticsearch-7.12.0]# su elasticsearch
[elasticsearch@VM-0-14-centos elasticsearch-7.12.0]$ ./bin/elasticsearch -d
[2021-04-05T22:03:38,332][INFO ][o.e.n.Node               ] [VM-0-14-centos] version[7.12.0], pid[13197], build[default/tar/78722783c38caa25a70982b5b042074cde5d3b3a/2021-03-18T06:17:15.410153305Z], OS[Linux/3.10.0-862.el7.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2021-04-05T22:03:38,348][INFO ][o.e.n.Node               ] [VM-0-14-centos] JVM home [/opt/elasticsearch-7.12.0/jdk], using bundled JDK [true]
[2021-04-05T22:03:38,348][INFO ][o.e.n.Node               ] [VM-0-14-centos] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-17264135248464897093, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms1894m, -Xmx1894m, -XX:MaxDirectMemorySize=993001472, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.path.home=/opt/elasticsearch-7.12.0, -Des.path.conf=/opt/elasticsearch-7.12.0/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
  • Check if the installation is successful:
[root@VM-0-14-centos ~]# netstat -ntlp | grep 9200
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      13549/java          
tcp6       0      0 ::1:9200                :::*                    LISTEN      13549/java 
[root@VM-0-14-centos ~]# curl 127.0.0.1:9200
{
  "name" : "VM-0-14-centos",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "ihttW8b2TfWSkwf_YgPH2Q",
  "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
    "build_date" : "2021-03-18T06:17:15.410153305Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Install Kibana #

Kibana is a user interface tool for querying data. When downloading, try to download a version that is compatible with ElasticSearch.

  • Download Kibana

Download Kibana from here.

  • Extract
[root@VM-0-14-centos opt]# tar -vxzf kibana-7.12.0-linux-x86_64.tar.gz
  • Use elasticsearch user privileges
[root@VM-0-14-centos opt]# chown -R elasticsearch /opt/kibana-7.12.0-linux-x86_64
# Configure remote access for Kibana
[root@VM-0-14-centos opt]# vi /opt/kibana-7.12.0-linux-x86_64/config/kibana.yml
server.host: 0.0.0.0
  • Start

Switch to the elasticsearch user.

[root@VM-0-14-centos opt]# su elasticsearch
[elasticsearch@VM-0-14-centos opt]$ cd /opt/kibana-7.12.0-linux-x86_64/
[elasticsearch@VM-0-14-centos kibana-7.12.0-linux-x86_64]$ ./bin/kibana
  log   [22:30:22.185] [info][plugins-service] Plugin "osquery" is disabled.
  log   [22:30:22.283] [warning][config][deprecation] Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0."
  log   [22:30:22.482] [info][plugins-system] Setting up [100] plugins: [taskManager,licensing,globalSearch,globalSearchProviders,banners,code,usageCollection,xpackLegacy,telemetryCollectionManager,telemetry,telemetryCollectionXpack,kibanaUsageCollection,securityOss,share,newsfeed,mapsLegacy,kibanaLegacy,translations,legacyExport,embeddable,uiActionsEnhanced,expressions,charts,esUiShared,bfetch,data,home,observability,console,consoleExtensions,apmOss,searchprofiler,painlessLab,grokdebugger,management,indexPatternManagement,advancedSettings,fileUpload,savedObjects,visualizations,visTypeVislib,visTypeVega,visTypeTimelion,features,licenseManagement,watcher,canvas,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeMarkdown,tileMap,regionMap,visTypeXy,graph,timelion,dashboard,dashboardEnhanced,visualize,visTypeTimeseries,inputControlVis,discover,discoverEnhanced,savedObjectsManagement,spaces,security,savedObjectsTagging,maps,lens,reporting,lists,encryptedSavedObjects,dashboardMode,dataEnhanced,cloud,upgradeAssistant,snapshotRestore,fleet,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,enterpriseSearch,beatsManagement,transform,ingestPipelines,eventLog,actions,alerts,triggersActionsUi,stackAlerts,ml,securitySolution,case,infra,monitoring,logstash,apm,uptime]
  log   [22:30:22.483] [info][plugins][taskManager] TaskManager is identified by the Kibana UUID: xxxxxx
  ...

If starting in the background:

[elasticsearch@VM-0-14-centos kibana-7.12.0-linux-x86_64]$ nohup ./bin/kibana &
  • Access the interface

img

Import simple data

img

View data

img

Configure Password Access #

When using the basic license, the Elasticsearch security features are disabled by default. However, in my testing environment, which is hosted on a public network, I need to set up password access. You can refer to the documentation for more information.

  1. Stop the Kibana and Elasticsearch services.
  2. Add the xpack.security.enabled setting to the ES_PATH_CONF/elasticsearch.yml file and set its value to true.
  3. Start Elasticsearch (./bin/elasticsearch -d).
  4. Run the following password setter command, ./bin/elasticsearch-setup-passwords interactive, to set passwords for each component.
  5. Add the elasticsearch.username setting to the KIB_PATH_CONF/kibana.yml file and set its value to the elastic user: elasticsearch.username: "elastic".
  6. Create the Kibana keystore with the command ./bin/kibana-keystore create.
  7. Add the password to the Kibana keystore using the command ./bin/kibana-keystore add elasticsearch.password.
  8. Restart the Kibana service using nohup ./bin/kibana &.

You can now use the password to log in:

img