Elasticdump copies data between clusters by reading documents out of one cluster through its search APIs and writing them into the other through its bulk APIs — the destination re-indexes every document under its own rules. That simple design is why the same command can move data between versions, and even between Elasticsearch and OpenSearch, where a snapshot cannot follow. It is also why a cluster copy is a workflow rather than a single command: structure has to land before data, credentials have to be right on both sides, and the result has to be counted before anyone calls it done.
This guide covers the whole job in the order you should do it: planning the copy, moving analyzers and mappings before documents, handling secured clusters, scaling out to many indices at once, and verifying the result. It assumes you already have ElasticDump installed and know the basic single-index command — the focus here is everything around that command that decides whether a real migration succeeds or quietly half-succeeds.
Step 1: Plan before you copy anything
A few questions answered up front prevent the most expensive mid-migration surprises.
What versions are on each side? ElasticDump reads from and writes to Elasticsearch 5 and later and to OpenSearch (supported since version 6.76.0), auto-detecting the cluster flavour. Because documents are re-indexed on arrival, version gaps between source and destination are survivable — but mapping and template rules are not identical across generations. Anything captured from a pre-7 cluster can carry constructs that 7.x and later refuse, such as mapping type names, the old template property on templates (replaced by index_patterns in 6.x), and the type property that 7.x removed from templates. Check the pair before you start, and expect to clean old mapping or template files by hand if you are crossing that boundary.
How much data is it, really? Get the document count and store size of every index you plan to move. Those numbers tell you three things: roughly how long the data pass will take, whether the destination has room, and — critically — what “done” looks like when you verify at the end. Without a source count recorded before the copy, there is nothing to verify against afterwards.
What counts as “the index”? An index is more than its documents. Its analyzer definitions, mapping, settings, aliases and any templates or policies that apply to it are separate objects, and ElasticDump copies each as its own --type pass. A data-only copy of an index whose queries rely on a custom analyzer will produce an index that stores documents but searches them wrong. Decide the full set per index now; the passes in the next step are how each one moves.
All of it, or a subset? If you only need some documents — a time range, one tenant, a sample for a test environment — --searchBody restricts a data pass to documents matching a query. Partial copies are a legitimate plan, not a shortcut, as long as the decision is deliberate and the expected count reflects it.
Step 2: Copy structure before data — analyzer, then mapping
This ordering is the heart of the whole workflow, and it is not optional. Elasticsearch auto-creates an index that does not exist the moment documents are written to it, and when it does, it dynamically guesses field types from those first documents. A guessed type only has to be wrong once — a date guessed as text, a numeric string guessed as a long — and every later document that does not fit the guess is rejected with a mapping error. Structure first is what prevents that entire class of failure.
For each index, the passes run in this order, with the source cluster URL as --input and the destination cluster URL as --output:
elasticdump --input=http://source:9200/my-index --output=http://dest:9200/my-index --type=analyzer elasticdump --input=http://source:9200/my-index --output=http://dest:9200/my-index --type=mapping elasticdump --input=http://source:9200/my-index --output=http://dest:9200/my-index --type=data
The analyzer pass matters because mappings reference analyzers: an index whose mapping names a custom analyzer cannot be created on a destination that does not have that analyzer yet. The mapping pass then creates the index with the correct schema before a single document arrives.
Beyond those three, the full set of types ElasticDump can copy per index is worth knowing, because anything you skip stays behind silently:
--type |
What it carries |
|---|---|
settings |
Index settings |
analyzer |
Custom analyzer definitions |
mapping |
Field mappings (the schema) |
data |
The documents themselves |
alias |
Index aliases pointing at this index |
template / index_template / component_template |
Legacy templates, composable index templates and their components |
policy |
Index lifecycle policies |
index |
A combined pass for the index definition |
Aliases deserve a deliberate decision rather than an afterthought: applications usually query through aliases, so a destination that has the data but not the alias is a migration that looks finished and is not. Templates and policies matter when new indices will be created on the destination under the same regime as the source. Copy them as their own passes, in the same structure-before-data spirit.
One more planning detail: the destination index name comes from the --output URL itself (or --output-index for non-standard paths), so renaming an index during the copy is simply a matter of writing to a differently named destination — create that index’s structure under the new name first, by the same analyzer → mapping sequence.
Step 3: Copy the data
With the structure in place, the data pass reads the source index and writes to the destination. Under the hood ElasticDump walks the index with the Scroll API, which holds a consistent view of the result set during the export; on Elasticsearch 7.17.0 and newer, --searchAfter combined with --pit offers the point-in-time paging approach instead. Those reading mechanics are the same ones described in the discussion of how to get all data from an Elasticsearch index, and they matter here for a practical reason: a copy is a long-running read followed by a long-running write, and its behaviour over hours is governed by batching and context settings, not by the command’s first minute.
The knobs that matter on a big copy:
--limitsets documents per batch (default 100). Larger batches mean fewer requests; smaller batches mean less memory in flight.--scrollTime(default 10m) is how long the scroll context survives between batches. On slow or loaded clusters, raising it protects a long copy from dying mid-way when a gap between batches gets too long.--retryAttemptsdefaults to 0, which means one transient network error ends a multi-hour copy. Raise it for any copy that matters;--retryDelay(5000 ms by default) spaces the attempts.- Writes use the
putmethod by default, so a document whose ID already exists on the destination is updated rather than duplicated — a re-run of a data pass is generally safe to repeat.--bulkActioncan change that behaviour toupdate,deleteorcreatesemantics when a job specifically needs it.
If a long copy is interrupted anyway, all is not lost: --scrollId resumes reading from the last scroll ID while the context is still alive, and --offset restarts from the last known written line — with the caveat that because records are not processed in a guaranteed order since version 6.1.0, skipped lines cannot be guaranteed to be exactly the ones already written. Resumption is a recovery option, not a plan; the plan is the retries and scroll settings above.
Step 4: Secured clusters on both sides
Real clusters usually require authentication, and a cluster-to-cluster copy has two ends to satisfy independently. Credentials can be embedded in each URL, but the environment variables keep passwords off the command line and out of shell history:
export ELASTICDUMP_INPUT_USERNAME=elastic export ELASTICDUMP_INPUT_PASSWORD='source-password' export ELASTICDUMP_OUTPUT_USERNAME=elastic export ELASTICDUMP_OUTPUT_PASSWORD='destination-password'
For TLS, each side takes its own CA file: --input-ca for the source cluster and --output-ca for the destination, with --ca as the shortcut when both clusters genuinely share one CA. Client certificates, where a cluster demands them, come from --cert and --key (and --tlsAuth). You will see NODE_TLS_REJECT_UNAUTHORIZED=0 suggested as a way past self-signed certificate errors; it does disable verification, and it is documented as a bypass — useful to prove TLS is the blocker, a poor thing to leave in a migration script. Pass the CA files instead. On AWS-managed OpenSearch, IAM signing replaces basic credentials: --awsChain lets ElasticDump pick credentials up from the environment in the standard AWS order, with --awsAccessKeyId / --awsSecretAccessKey and --sessionToken available for explicit credentials.
Step 5: Many indices at once with MultiElasticDump
Copying a whole cluster index-by-index works, but past a handful of indices the per-index command list becomes the maintenance problem. That is what multielasticdump exists for. It runs in two directions:
--direction=dumptakes a cluster base URL as input and a directory as output, writing data, mapping and analyzer files for every index that matches.--direction=loadtakes that dump directory as input and a cluster URL as output, loading everything back.
--match filters index names with a regular expression (the default matches everything), --parallel controls how many subprocesses run at once (the default fork count follows the host’s CPU count), and --prefix / --suffix rename indices on load — convenient for staging copies that must not collide with the originals. --matchType can select indices by alias or treat them as data streams.
One trap deserves emphasis because it bites exactly the people doing full copies: multielasticdump ignores the analyzer and alias types by default. A default multielasticdump round trip therefore produces indices without their custom analyzers and without their aliases — the two omissions most likely to break search behaviour and application queries respectively. Adjust --includeType / --ignoreType (which accept data,mapping,analyzer,alias,settings,template) so the types you planned for in step 1 are actually included, rather than discovering the gap during verification.
Whether you split the job into files first or go cluster-to-cluster directly is a judgement call. A directory of dump files doubles as an inspectable artifact you can keep, grep and re-load — line-delimited JSON, one document per line — at the cost of disk space and an extra pass. Direct cluster-to-cluster copies skip the intermediate storage. Both end at the same verification step.
Step 6: Verify the copy — counts, not exit codes
A finished command is not a finished migration. ElasticDump’s --ignore-es-write-errors option defaults to true, which means document rejections during the data pass do not stop the run and do not fail it. The copy tool deliberately leaves verification to you, so build it in as a step:
- Compare document counts per index between source and destination (the
_countAPI on both sides). They should match exactly, unless you copied a subset on purpose — in which case the destination count should match the subset query’s count. - Spot-check that aliases resolve on the destination and that a few representative searches return the same documents on both clusters — counts alone will not catch a missing analyzer changing how text is tokenised.
- If counts disagree, treat that as rejected writes until proven otherwise: read the load output for the rejection reasons, fix the cause (usually a mapping or a type clash), and re-run the data pass — remember that default write behaviour updates documents by ID, so a corrective re-run does not duplicate what already landed.
Skipping this step is how migrations get declared complete on Friday and rolled back on Monday.
Common failure points, briefly
Most failed copies fail at one of a handful of places: data loaded before the mapping, so the destination guessed the schema and then rejected the documents that disagreed; old mapping or template constructs from a pre-7 source refused by a modern destination; TLS or credential settings right on one cluster and wrong on the other; a scroll context expiring during a long data pass that had no retries configured; or a multielasticdump run that silently left analyzers and aliases behind. Each has a named fix rather than a mystery — the symptom-by-symptom walkthrough in Elasticdump not working: common errors and fixes covers them in the order you are likely to meet them.
When a snapshot or reindex is the better tool
ElasticDump is not the only route between clusters, and sometimes it is not the right one. Snapshot and restore moves data at segment level: it is faster on very large indices and incremental across repeated snapshots, but it requires a registered repository and — decisively — compatible Elasticsearch versions on both clusters, because a snapshot preserves the index in its original version’s form. Reindex-from-remote is Elasticsearch’s own built-in option for pulling documents from a remote cluster into a local index. ElasticDump’s advantages are the ones this workflow has used throughout: it re-indexes through the destination’s APIs, so it crosses version gaps and the Elasticsearch–OpenSearch divide, it can produce a plain file as its output, it can copy a queried subset with --searchBody, and it can transform documents mid-move with --transform. The head-to-head reasoning is laid out in Elasticdump vs snapshot and restore; the short version is that backups and same-version restores belong to snapshots, while portable, shaped, cross-version copies are ElasticDump’s home ground. That same portability is what makes the tool a common choice when the destination is not even the same product as the source.
FAQ
Can I copy between different Elasticsearch versions, or to OpenSearch?
Yes — that is one of the main reasons to use ElasticDump for the job. Because the destination re-indexes every document through its own APIs, the copy is not bound by snapshot version compatibility. ElasticDump supports OpenSearch targets since version 6.76.0 and detects the cluster flavour automatically (with --force-os-version, default 7.10.2, as an override). Do expect to clean pre-7 mapping and template constructs before a modern cluster accepts them.
Is copying the data with one --type=data command enough?
No. That copies documents only. Analyzers, mappings, settings, aliases, templates and policies are separate types with their own passes — and if the mapping does not exist before the data arrives, the destination will invent one by guessing, which is the top cause of failed loads.
How do I rename an index during the copy?
The destination name comes from the --output URL (or --output-index), so write to the new name — after creating its structure under that name with the analyzer and mapping passes. With multielasticdump, --prefix and --suffix handle renaming across many indices at once.
The copy finished with no errors. Am I done?
Not until the counts match. Because write errors are ignored by default, a clean exit does not prove every document landed. Compare source and destination counts per index, then check that aliases and a sample of searches behave the same on both sides.
Can I copy only part of an index?
Yes. --searchBody restricts a data pass to documents matching a query, which is the standard way to build a test copy or migrate one tenant at a time. Record the subset’s expected count before you start so verification still means something.
Conclusion
Elasticdump copies between clusters succeed or fail on the workflow around the command: plan the version pair and the full object set, land analyzers and mappings before a single document moves, give each secured cluster its own credentials and CA, watch the multielasticdump defaults that leave analyzers and aliases behind, and finish by counting documents on both sides. Followed in that order, the copy is boring in the best possible way — and “boring”, verified by matching counts, is exactly what a migration should be.