Transporting Configuration to a New Cluster
16 January 2020 02:06 PM
You have an application running on a particular cluster (the source cluster),
In an ideal world, the application is managed in an "infrastructure as code" manner: all of the configuration information about that cluster is codified in scripts and payloads stored in version control and able to be "replayed" at will. (One way to assure that this is the case is to configure testing for the application in a CI environment that begins by using the deployment scripts to configure the cluster.)
But in the real world, it's all too common for some amount of "tinkering" to have been performed in the Admin UI or via ad hoc calls to the Rest Management API (RMA). And even if that hasn't happened, it's not generally possible to be certain that's the case, so you still have to worry that it might have happened.
Migrating the application
The central theme in doing this "by hand" is that RMA payloads are re-playable. That is, the payload you GET for the properties of a resource is the same as the payload that you PUT to update the properties of that resource.
If you were going to migrate an application by hand, you'd proceed along these lines.
Determine what needs to be migrated
An application consists (more or less by definition) of one or more application servers. Application servers have databases associated with them (those databases may have additional database associations). Databases have forests.
A sufficiently complex application might have application servers divided into different groups of hosts.
Applications may also have users (for example, each application server has a default user; often, but not always, "
Users, in turn, have roles, and roles may have roles and privileges. Code may have amps that use privileges.
That covers most of the bases, but beware that apps can have additional configuration that should be reviewed: security artifacts (certificates, external securities, protected paths or collections, etc.), mime types, etc.
Get Source Configuration
Using RMA, you can get the properties of all of these resources:
Create Target Configuration
Some of the properties of a MarkLogic resource may be references to other resources. For example, an application server refers to databases and a role can refer to a privilege. Consequently, if you just attempt to POST all of the property payloads, you may not succeed. The references can, in fact, be circular so that no sequence will succeed.
The easiest way to get around this problem is to simply create all of the resources using minimal configurations: Create the forests (make sure you put them on the right hosts and configure them appropriately). Create the databases, application servers, roles, and privileges. Create the amps. If you need to create other resources (security artifacts, mime types, etc.) create those.
Finally, PUT the property payloads you collected from the source cluster onto the target cluster. This will update the properties of each application server, database, etc. to be the same as the source cluster.
MarkLogic Documentation - Scripting Cluster Management
MarkLogic Knowledgebase - Transferring data between MarkLogic Server clusters
MarkLogic Knowledgebase - Best Practices for exporting and importing data in bulk
MarkLogic Knowledgebase - Deployment and Continuous Integration Tools