06 Detailed Explanation of Index Templates ( Index Template)

06 Detailed Explanation of Index Templates (Index Template) #

索引模板是Elasticsearch中的一个重要概念，用于提供在创建索引时的默认配置。索引模板包含了一系列设置，如索引的分片数量、副本数量、字段的映射规则等。

与显式地为每个索引进行配置不同，使用索引模板可以自动为满足特定条件的索引应用相同的配置。这种方式可以大大简化索引创建过程，并确保所有的索引都符合相同的规范。

创建索引模板 #

要创建一个索引模板，可以使用Elasticsearch提供的API进行操作。以下是一个示例请求：

PUT _template/template_name
{
  "index_patterns": ["pattern*"],
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 2
  },
  "mappings": {
    "properties": {
      "field_name": {
        "type": "text"
      }
    }
  }
}

在这个示例中，我们创建了一个名为template_name的索引模板。索引模板会应用于所有以pattern开头的索引。在模板中，我们定义了分片数量为3，副本数量为2，并指定了一个名为field_name的文本字段。

匹配模板 #

当创建新的索引时，Elasticsearch会自动检查其名称是否与一个或多个索引模板的匹配模式相匹配。如果匹配成功，将会自动应用相应的模板配置。

除了通配符外，还可以使用其他匹配模式来定义索引模板。例如，可以使用正则表达式来匹配复杂的索引名称。

更新模板 #

已经存在的索引模板可以更新。只需再次使用相同名称进行PUT请求即可。新的设置和映射规则将覆盖旧的配置。

删除模板 #

要删除已有的索引模板，可以使用DELETE请求：

DELETE _template/template_name

删除后，将不再匹配该模板的索引会使用默认配置。

总结：索引模板是一个强大的工具，可以提供默认的索引配置，并在创建索引时自动应用。使用索引模板可以简化管理过程，并确保所有的索引都符合相同的规范。

Index Templates #

An index template is a method of configuring an index by telling Elasticsearch how to create the index.

  • Usage

Before creating an index, you can first configure a template. In this way, the template settings will serve as the basis for creating the index (either manually or by indexing documents).

Template Types #

There are two types of templates: index templates and component templates.

  1. Component templates are reusable building blocks used for configuring mappings, settings, and aliases. They are not directly applied to a set of indexes.
  2. Index templates can include collections of component templates, as well as directly specify settings, mappings, and aliases.

Priority in Index Templates #

  1. Composable templates take precedence over older templates. If no composable template matches a given index, an old-style template may still match and be applied.
  2. If an index is created with explicit settings and matches an index template as well, the settings in the create index request will take precedence over the settings specified in the index template and its component templates.
  3. If a new data stream or index matches multiple index templates, the one with the highest priority will be used.

Built-in Index Templates #

Elasticsearch has built-in index templates with a priority of 100, which apply to the following index patterns:

  1. logs-*-*
  2. metrics-*-*
  3. synthetics-*-*

So, when dealing with built-in index templates, it is important to avoid conflicts with the index patterns. For more information, please refer to here.

Example #

  • First, create two component templates:

    PUT _component_template/component_template1 { “template”: { “mappings”: { “properties”: { “@timestamp”: { “type”: “date” } } } } }

    PUT _component_template/runtime_component_template { “template”: { “mappings”: { “runtime”: { “day_of_week”: { “type”: “keyword”, “script”: { “source”: “emit(doc[’@timestamp’].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))” } } } } } }

The execution result is as follows:

img

  • Create an index template that uses the component templates:

    PUT _index_template/template_1 { “index_patterns”: [“bar*”], “template”: { “settings”: { “number_of_shards”: 1 }, “mappings”: { “_source”: { “enabled”: true }, “properties”: { “host_name”: { “type”: “keyword” }, “created_at”: { “type”: “date”, “format”: “EEE MMM dd HH:mm:ss Z yyyy” } } }, “aliases”: { “mydata”: { } } }, “priority”: 500, “composed_of”: [“component_template1”, “runtime_component_template”], “version”: 3, “_meta”: { “description”: “my custom” } }

The execution result is as follows:

img

  • Create an index bar-test that matches bar*:

    PUT /bar-test

Then get the mapping:

GET /bar-test/_mapping

The execution result is as follows:

img

Simulating Multi-component Templates #

Since templates can not only be composed of multiple component templates, but can also be composed of the index template itself, what will be the final index configuration? Elasticsearch designers have taken this into consideration and provided an API to configure the simulated combined template.

Simulating an Index Result for a Specific Template #

For example, with the template_1 mentioned above, instead of creating the bar* index (here simulating bar-pdai-test), we can simulate the calculated index configuration:

POST /_index_template/_simulate_index/bar-pdai-test

The execution result is as follows:

img

Simulating Component Template Results #

Of course, since template_1 is composed of two component templates, we can also simulate the index configuration after template_1 is combined:

POST /_index_template/_simulate/template_1

The execution result is as follows:

{
  "template": {
    "settings": {
      "index": {
        "number_of_shards": "1"
      }
    },
    "mappings": {
      "runtime": {
        "day_of_week": {
          "type": "keyword",
          "script": {
            "source": "emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))",
            "lang": "painless"
          }
        }
      },
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "created_at": {
          "type": "date",
          "format": "EEE MMM dd HH:mm:ss Z yyyy"
        },
        "host_name": {
          "type": "keyword"
        }
      }
    },
    "aliases": {
      "mydata": {}
    }
  },
  "overlapping": []
}

Simulating the Result of Combining Component Templates with the Template Itself #

  • Create two new templates:
PUT /_component_template/ct1
{
  "template": {
    "settings": {
      "index.number_of_shards": 2
    }
  }
}

PUT /_component_template/ct2
{
  "template": {
    "settings": {
      "index.number_of_replicas": 0
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        }
      }
    }
  }
}

Simulate adding the configuration of the template itself on top of the two component templates:

POST /_index_template/_simulate
{
  "index_patterns": ["my*"],
  "template": {
    "settings": {
      "index.number_of_shards": 3
    }
  },
  "composed_of": ["ct1", "ct2"]
}

The execution result is as follows:

{
  "template": {
    "settings": {
      "index": {
        "number_of_shards": "3",
        "number_of_replicas": "0"
      }
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        }
      }
    },
    "aliases": {}
  },
  "overlapping": []
}

img

Reference Articles #

https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/simulate-multi-component-templates.html