Reindexing and Migrations

In this guide, you'll learn how the Search Module creates physical indexes, fills them, and rebuilds them when a definition changes.

Index Migrations#

A search index definition declares what an index should look like. The physical index in the search engine is a separate thing, so the Search Module has to bring the two in line. It does that with an index migration.

Index migrations run as part of the db:migrate command:

yarn
pnpm
npx

You can also run them on their own with the following command:

yarn
pnpm
npx

The command plans the actions first, then prints what it did:

Terminal
❯info:    Migrating search indexes...❯┌────────────────────────────────────┐❯│                                    │❯│   Created following search indexes │❯│     - product (product)            │❯│                                    │❯└────────────────────────────────────┘❯info:    Search indexes migrated. They are filled when the application starts

Every action is idempotent, so running it twice has no adverse effect. When the physical indexes already match every definition, the command prints Search indexes already up-to-date and changes nothing.

The command reports each action under one of the following:

Dropping Removed Indexes#

When you remove an index definition from your application, the index in the search engine stays behind. So, the command plans a drop for it, which deletes every document it holds and can't be undone. The only way back is to declare the definition again and let it seed.

Since that's destructive, the command lists those indexes and asks which of them to drop before it touches any of them:

Terminal
❯┌──────────────────────────────────────────────────────────┐❯│                                                          │❯│   Select the search indexes to DROP. No                  │❯│   definition declares them any more, and                 │❯│   dropping one deletes every document it holds.          │❯│                                                          │❯└──────────────────────────────────────────────────────────┘❯? Select search indexes to drop❯❯ ◯ article (article_v2, article_v1)

Pass one of the following options to db:migrate or db:migrate:search to skip the prompt:

• --execute-all-search: drop every index that no definition declares any more.
• --execute-safe-search: leave those indexes in place.

How Index Changes Are Handled#

An index name, such as product, isn't one physical index. The Search Module builds a version of the index per definition change, and one of those versions is the index's active version. A query resolves the name to the active version, so which physical index answers a search can change without your code changing.

That's what makes a schema change zero-downtime. When a definition changes, the module builds a new version alongside the active one, fills it, then makes the new version active. Reads keep hitting the old version throughout and never see a half-built one. A version that fails to fill stays behind without ever going live, so the active version keeps answering queries.

The module keeps the active version and the one it's building. It removes every version below the active one at the start of the next migration or rebuild, so a swap that went live doesn't leave its old index in the engine.

A version is also what lets an index move between providers. Setting a different provider on a definition makes the module build the new version on that provider, and the old provider's data only gets dropped once the new version is active.

Seeding at Application Start#

When your application starts in worker or shared mode, the Search Module checks every index and fills the ones that need data. An application running in server mode skips this entirely, so at least one worker or shared process has to run for an index to be filled.

The module runs an index definition's seed function in the following cases:

1. A migration created the index and nothing has filled it yet.
2. The index exists and holds no documents, such as after an engine restart wiped it.
3. A migration built a new version of the index, which the module fills and then makes active.
4. The previous seed didn't complete, so the module resumes it.
5. You rebuild the index on demand with the Search Module's reindex method. Refer to Seeding on Demand.
6. The module runs the catch-up pass that follows a full run, which picks up what changed while that run was writing. Refer to The Catch-Up Pass.

The first four cases run when your application starts. The module records every seed run, so an interrupted run passes its last_key back to the seed function and resumes where it stopped.

Seeding on Demand#

You can rebuild an index manually from the Medusa Admin dashboard, or programmatically with the reindex method of the Search Module's service.

For example, the following workflow step rebuilds only the published products in the product index:

src/workflows/steps/reindex-products.ts
1import { Modules } from "@medusajs/framework/utils"2import {3  createStep,4  StepResponse,5} from "@medusajs/framework/workflows-sdk"6
7export const reindexProductsStep = createStep(8  "reindex-products",9  async (_, { container }) => {10    const searchModuleService = container.resolve(11      Modules.SEARCH12    )13
14    const result = await searchModuleService.reindex({15      index: "product",16      filters: { status: "published" },17    })18
19    return new StepResponse(result)20  }21)

Parameters#

reindex accepts the following input:

Returns#

reindex returns to an object with the following properties:

The method rebuilds every index before it returns this objectg, so you don't poll the job_id to know when it finished.

Reindex Strategies#

You can specify how the module rebuilds an index with the strategy option. The two strategies are:

The Catch-Up Pass#

A full seed run reads your data while your application keeps writing to it, so a record that changes mid-run might land in the index stale, or not at all.

To close that gap, the module runs seed a second time as soon as the run finishes, passing it catchup.since, which is the time the run started. A swap only makes the new index active once this pass finishes.

The following diagram shows a full seed run and the catch-up pass that follows it:

graphSeed handles the pass for you. To handle it in a seed function you write yourself, refer to Handle the Catch-Up Pass.

Keeping Indexes in Sync#

An index holds a copy of your data, so it can fall behind the database. The events and consume properties of an index definition close that gap as changes happen, and a reindex repairs it when the gap grows too wide.

Why an Index Diverges#

An index drifts from the database for one of the following reasons:

Recover with a Reindex#

A reindex is the repair path for a diverged index, since it rebuilds documents from the definition's seed function rather than from an event. Refer to Seeding on Demand for how to call it.

Two details matter when you reindex to recover:

• Pass filters to rebuild only the affected slice. A filtered run costs less than a full rebuild, and it always writes in place, so the documents outside the slice keep serving reads.
• Omit filters for a full repair. Both strategies then rebuild every document from the seed, and either one drops what the seed no longer produces. Keep the default swap if the index must keep answering queries while it rebuilds.