Introduction
As a best practice, it is recommended to maintain the shards' size between 10 and 50 GiB to strike a balance between having too many shards causing an overloaded cluster and having large shards that make cluster recovery difficult. In addition to this, keeping the shard size even and shard count a multiple of nodes would help with the even distribution of shards, reducing storage and performance skew.
Resolution
The primary shard count of an index can only be configured at the time of index creation and cannot be changed afterward. In order to change the sharding, you would have to create a new index with updated sharding and use _reindex API to copy all indices from existing indices to the new index.
For the purpose of this article, let us consider the following indices on an Elasticsearch cluster.
$ curl -XGET 'http://localhost:9200/_cat/indices?s=index&v=true'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open index-000001 8Dmwh8s-T2Csy8atxzK_8Q 5 1 6 0 32.7kb 9.5kb
green open index-000002 SadtrQ8ZRZWMmjhWrRoaUA 5 1 4 0 19.7kb 5kb
green open index-000003 wUw8StUKTEKExeaoOKSUEg 5 1 5 0 46.2kb 15.6kb
green open index-000004 yPXHeD9nQomucMedA6_Hzg 5 1 6 0 28.3kb 14.1kb
green open index-000005 J8aPYx9eRxCPseYvihz-Pg 5 1 8 0 27.7kb 18.5kb
Aggregating indices using index pattern
Often, indices are rolled-over/created on a daily basis to make it easily searchable. This could possibly result in many shards that are also small in size. These indices often have a pattern which can be used to aggregate these indices into less number of indices and thereby reducing shards. For example, the indices above have the pattern “index-*”. In such cases, you can create a new index as your aggregation destination. If you wish to change the sharding (primary shard count), you would have to do the same when creating a new index as below.
$ curl -XPUT 'http://localhost:9200/agg-index-1-10?pretty' -H 'Content-Type: application/json' -d '
> {
> "settings": {
> "index": {
> "number_of_shards": 3,
> "number_of_replicas": 1
> }
> }
> }'
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "agg-index-1-10"
}
Then, you can use the _reindex API to copy all documents from the existing indices to the newly created index.
$ curl -XPOST 'http://localhost:9200/_reindex?wait_for_completion=false&pretty' -H 'Content-Type: application/json' -d '
> {
> "source": {
> "index": "index-*"
> },
> "dest": {
> "index": "agg-index-1-10"
> }
> }'
{
"task" : "y2N1gjCZTcCUkYXaNTqqzQ:19969987"
}
You can monitor the above task using the _tasks API, until you notice “completed” : true.
$ curl -XGET 'http://localhost:9200/_tasks/sFCzlAnzSnya1X07eN3V6w:14282529?pretty'
Then, verify whether the document count matches. In this case, the sum of documents in “index-*” indices should be equal to the documents “agg-index-1–10” index.
$ curl -XGET 'http://localhost:9200/_cat/indices?s=index&v=true'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open agg-index-1-10 YhwfRgMbQ8e_AfX9sy2UqQ 3 1 50 0 35kb 17.5kb
green open index-000001 8Dmwh8s-T2Csy8atxzK_8Q 5 1 10 0 46.4kb 23.2kb
green open index-000002 SadtrQ8ZRZWMmjhWrRoaUA 5 1 10 0 29.7kb 14.8kb
green open index-000003 wUw8StUKTEKExeaoOKSUEg 5 1 10 0 55.8kb 25kb
green open index-000004 yPXHeD9nQomucMedA6_Hzg 5 1 10 0 38kb 19kb
green open index-000005 J8aPYx9eRxCPseYvihz-Pg 5 1 10 0 46.4kb 23.2kb
Once document count matches and your applications/pipelines/services are pointing towards the new indices, you can delete the old indices as follows.
$ curl -XDELETE 'http://localhost:9200/indices-*?pretty'
Aggregating specific indices
Sometimes, you may have to aggregate or reduce the shards of a specific set of indices. In such cases, you can use the _reindex API as follows.
$ curl -XPOST 'http://localhost:9200/_reindex?wait_for_completion=false&pretty' -H 'Content-Type: application/json' -d '
> {
> "source": {
> "index": ["index-000001", "index-000002"]
> },
> "dest": {
> "index": "agg-index-1-2"
> }
> }'
{
"task" : "y2N1gjCZTcCUkYXaNTqqzQ:19974516"
}
You can monitor the task as described above. Once you observe “completed”: true in the _tasks API. Ensure that the document count matches and delete unwanted indices.
$ curl -XGET 'http://localhost:9200/_cat/indices?s=index&v=true'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open agg-index-1-2 DofJgv0FRMiLi5XimLMgnw 5 1 20 0 10.8kb 624b
green open index-000001 8Dmwh8s-T2Csy8atxzK_8Q 5 1 10 0 46.4kb 23.2kb
green open index-000002 SadtrQ8ZRZWMmjhWrRoaUA 5 1 10 0 29.7kb 14.8kb
green open index-000003 wUw8StUKTEKExeaoOKSUEg 5 1 10 0 55.8kb 25kb
green open index-000004 yPXHeD9nQomucMedA6_Hzg 5 1 10 0 38kb 19kb
green open index-000005 J8aPYx9eRxCPseYvihz-Pg 5 1 10 0 46.4kb 23.2kb
Decreasing shard size
Sometimes, your shard size might be too large. This can impact cluster recovery as large shards make it difficult. It can also slow down blue/green deployments that are initiated when configuration changes are triggered on your Amazon Elasticsearch Service domain. In this case, you can increase shard count per index when creating new index as below.
$ curl -XPUT 'http://localhost:9200/inc-index-1-2?pretty' -H 'Content-Type: application/json' -d '
> {
> "settings": {
> "index": {
> "number_of_shards": 10,
> "number_of_replicas": 1
> }
> }
> }'
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "inc-index-1-2"
}
Then, similar to above, use the _reindex API to copy all documents from existing to the new index.
$ curl -XPOST 'http://localhost:9200/_reindex?wait_for_completion=false&pretty' -H 'Content-Type: application/json' -d '
{
"source": {
"index": "agg-index-1-2"
},
"dest": {
"index": "inc-index-1-2"
}
}'
{
"task" : "sFCzlAnzSnya1X07eN3V6w:14282529"
}
Once the task is complete, you should see the document count matching before going ahead to delete the old index.
$ curl -XGET 'http://localhost:9200/_cat/indices?s=index&v=true'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open agg-index-1-2 DofJgv0FRMiLi5XimLMgnw 5 1 20 0 48.8kb 24.4kb
green open inc-index-1-2 7d_26FMXSTqPtvfxXVSBcg 10 1 20 0 74.4kb 37.2kb
green open index-000001 8Dmwh8s-T2Csy8atxzK_8Q 5 1 10 0 46.4kb 23.2kb
green open index-000002 SadtrQ8ZRZWMmjhWrRoaUA 5 1 10 0 29.7kb 14.8kb
green open index-000003 wUw8StUKTEKExeaoOKSUEg 5 1 10 0 55.8kb 25kb
green open index-000004 yPXHeD9nQomucMedA6_Hzg 5 1 10 0 38kb 19kb
green open index-000005 J8aPYx9eRxCPseYvihz-Pg 5 1 10 0 46.4kb 23.2kb
