Data management

Contents

This section describes how to manage and backup your XP data

Introduction

XP stores two different types of data, that are crucial for backing up and restoring the system.

Blobs

XP persists most of its data as files - aka blobs. Blobs are both text files (data) and binary files.

XP’s blobstore implementation uses a so-called append-only approach. This simply means that blobs are created once, and used for reading - but they are never updated at a later point. This eliminates traditional file-system problems like locking, and enables aggressive caching. The blobstores are effective since as a single blob may be re-used for a vast number of data nodes created in XP.

For clustered deployments, the blobs must be stored on a dedicated shared/distributed filesystem.

Indicies

In addition to blobs, XP also creates and stores indicies. Indicies are automatically replicated across all data nodes if you have a cluster.

There are essentially two types of indexes.

  • The search indexes which can be updated or re-created through re-indexing.

  • The metadata indicies contain critical repository, branch and tree structures as well as version history.

The metadata indicies are required in order to fully restore XP data.

Indicies MUST be stored on a local filesystem, preferably using fast SSDs.

Tools

Enonic XP offers both management endpoints, and internal APIs to manage your data.

As an alternative to working with the APIs. Check out the following tools:

Snapshot and restore

You may create snapshots of the storage at any time. Snapshots normally completes within seconds. Once you have a snapshot, you may rollback to the snapshot at a later time.

Snapshots will only restore the metadata and search index of your system, not the blobs. As such, you must have an intact set of blobs from when the snapshot was taken. Deleting blobs normally requires running a vacuum job on the storage.

Snapshotting and restoring can be performed by using Enonic CLI, the API or a management application from Enonic Market.

Export and import

You may export and import tree structures of nodes, or even single leaf nodes using the Export and Import feature. Exporting will produce a human-readable export of the selected repo/branch/node and it’s child items.

An export contains metadata and blobs required to re-create the selected nodes in current state. You may import any valid export, but pay attention as export/import will keep your existing node identifiers.

Exporting and importing can be performed by using Enonic CLI, the API or a management application from Enonic Market.

Export and import is limited to work basic current data. They do not include metadata such as version history, branches or commits.

Dump and load

You may dump and load your entire system. Dumping will produce a machine readable file that can be used for loading at a later time.

The dump may also include the version history for your data.

Dumping and loading can be performed by using Enonic CLI, the API or a management application from Enonic Market.

Dumping is not a recommended backup solution, rather use snapshots in combination with a backup of up your blobstore

Vacuum

Even if you delete nodes in your storage, the blobs and essential metadata are not actually removed from XP. Deleting the blobs would prevent rolling back to a previous snapshots, but cleaning up the data is also a time consuming operation.

To perform this operation in a controlled fashion, XP offers a function called Vacuuming. Vacuuming will remove unused nodes older than a specified age (This can be controlled), and then remove any unused blobs or indicies based on this.

Backup and restore

The fastest and simplest way of backing up XP is by completing the following steps, in the specified order:

  1. Take a snapshot

  2. Backup the files in your snapshots folder

  3. Backup the files in your blobstore folder

Given that blobstores only append new files on changes, the blobstore will at any time contain an historical archive of your data.

When restoring, simply use the backup data, and perform a Snapshot restore.


Contents