Solving AWS Cloud Formation Template failures due to deprecated support for Python 3.6 and CLB
29 August 2022 01:24 PM
This article is intended to address the impact of AWS deprecation of Python 3.6 (Lambda runtime dependency) and Classic Load Balancer (CLB) on MarkLogic Cloud Formation Templates (CFT).
AWS announced deprecation of Python 3.6 and Classic Load Balancer (CLB).
For Python 3.6, please refer to 'Runtime end of support dates'.
MarkLogic 10 provided CFTs prior to 10.0-9.2 are impacted by the python 3.6 deprecation as MarkLogic uses custom lambdas. CFTs prior to 10.0-9.2 are also impacted by the CLB deprecation since the MarkLogic single-host deployment uses CLB.
1. Upgrade to latest MarkLogic CFT templates:
Starting with release of 10.0-9.2, MarkLogic CFT uses python 3.9 and has removed CLB for single-host deployments.
The fully-qualified domain name (FQDN) of the node is based on internal IP address from the persistent reusable ENI. In single-host cluster without CLB, the FQDN for the node is referenced in the list of outputs as the endpoint to access Admin UI. For example, http://ip-10.x.x.x.ap-southeast-2.compute.internal:8001.
For a single-host cluster in a private subnet, client residing in public domain will not be able to connect to single host directly. Your AWS Administrator will be required to set up a bastion host (jump box) or a reverse proxy, which acts as an addressable middle-tier to route traffic to the MarkLogic host. Alternatively, your Administrator can assign an Elastic IP to single-host which makes the host publicly accessible.
2. Running with MarkLogic prior to 10.0-9.2
2.1: Modify MarkLogic's most current CFT.
You can use the latest version of the MarkLogic CFT, and then change the MarkLogic AMI version inside that CFT to refer to specific prior version of MarkLogic AMI.
2.2: Customized CFT (derived from MarkLogic CFT but with specific modification).
You can modify your copy of template to upgrade to Python 3.9 and remove the use of CLB.
a) To upgrade the Python changes: Please refer to the custom lambda templates (ml-managedeni.template, ml-nodemanger.template) and search for "python3.6" and replace it with "python3.9".
Format to build the URL: https://marklogic-db-template-releases.s3.<<AWS region>>.amazonaws.com/<<ml-version>>/ml-nodemanager.template
Download v10.0-7.1 custom lambda templates for upgrade using below links:
After the changes are done, the modified templates should be uploaded to the s3 bucket. Also, the 'TemplateURL' should be updated in the main CFTs (mlcluster-vpc.template, mlcluster.template) under 'Resources' -> ManagedEniStack, 'Resources' -> NodeMgrLambdaStack.
b) To remove the CLB changes: Please refer to the latest CFT version (mlcluster-vpc.template, mlcluster.template) and compare/modify the templates accordingly.
AWS deprecation does not impact already deployed stack, since the Lambda functions are created during service creation (and only deleted when the service is terminated). Similarly, updating the cluster capacity does not have impact on existing deployed stack.
MarkLogic Cloud Services (DHS)
The issue is already addressed by the MarkLogic Cloud Services team with an upgrade of underlying dependency to "Python 3.9".
Please Note that this Knowledgebase article refers to MarkLogic 10 Cloud Formation Template changes alone. For MarkLogic 9 Cloud Formation templates, work on recommended Solutions is still in progress.