Linux Huge Pages and Transparent Huge Pages
25 August 2023 01:43 PM
This article contains a
To the Linux kernel, "pages" describe a unit of memory; by
Huge Pages (and Transparent Huge Pages) allow areas of memory to be reserved for resources which are likely to be accessed frequently, such as group level caches. Enabling (and configuring) Huge Pages can increase performance because - when enabled - caches should always be resident in memory.
On Linux systems, MarkLogic recommends setting Linux Huge Pages to around 3/8 the size of your physical memory. For details on setting up Huge Pages, refer to the following KB
Since the OS and server perform many memory allocations that do not and cannot use huge pages, it may not be possible to configure the full 3/8 for huge pages. It is not advised to configure more than 3/8 of memory to huge pages.
Calculating the number of Huge Pages to configure:
On an x86
Alternatively, MarkLogic provides a recommended range for the number of Huge Pages that should be used. This recommendation can be seen in the ErrorLog.txt file located in /var/opt/MarkLogic/Logs/ just after the server is started. Right after starting the server, look for a message that looks like this:
2019-09-01 17:33:14.894 Info: Linux Huge Pages: detected 0, recommend 11360 to 15413
The lower bound includes all group-level caches, while the upper bound also includes in-memory stand sizes.
Allocating Huge Pages
Since Huge Pages require large areas of contiguous physical memory, it is advised to allocate huge pages at boot time. This can be accomplished by
Transparent Huge Pages
The Transparent Huge Page (THP) implementation in the Linux kernel includes functionality that provides compaction. Compaction operations are system level processes that are resource intensive, potentially causing resource starvation to the MarkLogic process. Using static Huge Pages is the preferred memory configuration for several
If you are using Red Hat Enterprise Linux or CentOS 6, you must turn off Transparent Huge Pages (Transparent Huge Pages are configured automatically by the operating system).
The preferred method to disable Transparent HugePages is to add "transparent_hugepage=never" to the kernel boot line in the "/etc/grub.conf" file.
This solution (disabling Transparent HugePages) is covered in detail in this article on RedHat's website
RHEL 6 | CentOS 6: Kernels newer than kernel-2.6.32-504.23.4. A race condition can manifest itself as system crash in region_* functions when
2. Downgrade the kernel to a version before 2.6.32-504.23.4.
3. Disable Huge Pages completely, which might impact MarkLogic’s performance by ~5-15%.