Diagnosing Rebalancer issues after adding or removing a forest
23 February 2016 01:46 PM
After adding or removing a forest and correspond replica forest in a database, we have seen instances where the Rebalancer does not properly distribute the documents amongst existing and newly added forests.
For this particular instance, XDMP-HASHLOCKINGRETRY debug level error message reported repeatedly in the error logs. The messages would look something like:
Gather statistics about the rebalancer to see the number of documents being scheduled. If you run attached script “
If experiencing this issue, the newly added forests will show zero in the “Total to be moved” column in the generated html page.
Perform a cluster wide restart in order to get past this issue. The restart is required to reload all of the configuration files across the cluster. The rebalancer will also check to see if additional rebalancing work needs to occur. The rebalancer should work as expected now and the XDMP-HASHLOCKINGRETRY messages should no longer appear in the logs. If you run the
You can also validate the rebalancer status from the Database Status page in the Admin UI.
The XDMP-HASHLOCKINGRETRY rebalancer issue has fixed in the latest MarkLogic Server releases. However, the rebalancer-preview.xqy script can be used to help diagnose other perceived issues with the Rebalancer.