Troubleshooting MarkLogic Certificate Based Authentication
15 August 2018 10:18 AM


With the introduction of Certificate Based Authentication in MarkLogic 9, users can now log into a MarkLogic without entering user/password credentials.

Configuring a MarkLogic AppServer to support TLS Client Certificate Authentication is a little more complex than simple SSL Server based authentication and it may not always be apparent why connections are not working once configuration is completed.

This Knowledgeabse article demonstrates some simple debugging techniques that should help to track down and identify issues encountered with Certificate Based authentication where things are not working as expected.

What is the difference between Client and Server based authentication?

Before starting down the path of troubleshooting it's worth ensuring that we understand what the differences are between TLS Server based authentication and TLS Client Authentication:

With a standard HTTPS connection to a TLS-enabled Application Server, MarkLogic server will send a copy of its X509 Certificate to the client who will then verify the certificate against a list of known Trusted Root certificates installed within the browser or a Java KeyStore for a Java based application (such as MLCP).

When TLS Client Authentication is enabled in MarkLogic for Certificate Based authentication, as well as sending a certificate to the client, MarkLogic Server will request that the client sends a certificate back to the server.

The certificate returned by the client is then used to determine which Internal or External user is used within MarkLogic.

How does the client know which certificate to send?

A web browser can often have multiple client certificates installed so how does it know which certificate to present to MarkLogic Server?

The certificate(s) that the Client can use are controlled by MarkLogic Server's application server settings.  Using the Admin GUI on port 8001, during configuration for Certificate Based authentication, you can specify that a client certificate is required (Configure > Groups > [Your Group Name] > App Servers > [Your App Server Name] > ssl require client certificate : true) and you can also select one or more Certificate Authorities under the ssl client certificate authorities section.

Only Client certificates issued by one of these authorities will be permitted.

If a browser has multiple Client certificates issued by one of the selected Certificate Authorities, the user will be prompted to select the appropriate client certificate to use.

Note: In this case it is important to select the certificate that have been issued to you for use within MarkLogic.

To verify that you have a valid certificate, you can either use a local system tool such as KeyChain Access (in Mac OS X) to check that the Issuer Name details for your client certificate match those of the Certificate Authorities configured in the MarkLogic Application Server settings as per the example above.

Alternatively, if you have a PEM representation of your user certificate you can use the OpenSSL utility to display the Issuer information, e.g.

>$ openssl x509 -in user1.pem -issuer -noout
issuer= /O=MarkLogic/OU=Support/CN=RootCA

Verifying the TLS Handshake

The first stage of MarkLogic Certificate Based authentication requires a successful TLS Handshake to take place between the Client and MarkLogic Server.

If the TLS Handshake fails at any stage, the session will be rejected.

Recommendation: While it is not a required it is highly recommended that you have a working HTTPS Application Server configuration first (using basic authentication) before enabling certificate based authentication.

This will ensure you have a valid TLS Server configuration before you enable TLS Client Authentication and should reduce the amount of troubleshooting required.

The easiest way to view what is taking place during the TLS Handshake is at the TCP packet level using a tool such as Wireshark which has built-in support for decoding the TLS protocol.

If Wireshark can easily be installed on a Client machine, it can be configured to capture TCP traffic to/from the MarkLogic AppServer port; the example below demonstrates capturing set up with a filter for all traffic on port 8010

If it is not possible to install Wireshark on the Client machine, the same information can be captured on the MarkLogic Server using the tcpdump utility.  From there you can create a pcap file on a given port (in this example: 8010), by running the following command:

tcpdump –i any –s 0 –w certauth.pcap port 8010

Once you have run tcpdump long enough to have captured the failing transaction, you can attach the resulting pcap file to a MarkLogic support ticket for further analysis.

If you are able to view the packet trace in Wireshark, you will first need to locate and select the packet where MarkLogic Server sends the Certificate Request

In the Frame details panel, you can drill down to the list of Distinguished Names and check that there is an entry for the Certificate Authority configured in MarkLogic

Having ensured that MarkLogic Server is sending a request for the correct client certificate you should locate the subsequent Certificate response being sent by the client

In the Frame details panel, the first thing to check is whether a Certificate was actually returned by the client. If no Certificate was found by the client that satisfied the MarkLogic Server request, then a Zero Certificate (Certificates Length: 0) is returned

If a Client sends a Zero Certificate the session will be terminated immediately with a TLS Handshake Failure.

In this case you should check that you have correctly installed a client certificate that was issued by the Certificate Authorities configured in MarkLogic for this Application Server.

If a valid certificate was found by the Client you will see the necessary information within the Frame details panel in Wireshark

If the session is terminated at this point with a TLS Handshake Failure the most likely cause is that MarkLogic Server was unable to verify the client certificate to a valid chain of root certificates.

This will typically occur if the Issuing Certificate Authority configured in MarkLogic is part of a chain of Root certificates, often referred to as an Intermediate CA certificate. In this case you should check that all CA certificates in the chain have been installed to the Trusted Store in the MarkLogic Security database.

If no TLS Handshake Failure occurs, you should see a pair of Encrypted Handshake Message packets which indicate that secure encryption has been enabled by both the client and the server and the TLS Handshake has successfully completed

I still get a 401 Unauthorized error

Having first established that a successful TLS Client Authentication has taken place if you still get a 401 Unauthorized error then the likely cause is that MarkLogic has not been able to successfully map the supplied Client certificate to either an Internal or External User.

The first check that MarkLogic Server will make is to look for an Internal User that matches the Common Name (CN) in the supplied Client certificate.

Having checked the userid specified in the certificate Common Name, e.g.

Check that a corresponding Internal MarkLogic userid exists in your Security database; in the Admin GUI in MarkLogic check for a matching user name (Configure > Security > Users > [your user name])

Alternatively, if no internal user matches, MarkLogic will attempt to use the full Subject Distinguished Name in the Client certificate to map to an external security name within a previously defined MarkLogic user.

 In this scenario first check that you have a valid External Security definition configured to perform certificate based authentication (Configure > Security > External Security)

Assign the External Security definition to the Application Server to map the external security name (Configure > Groups > [Your Group Name] > App Servers > [Your App Server Name] > external securities)

Finally check that Internal MarkLogic User has an External Name that matches the Client Certificate Subject Distinguished Name (DN) (Configure > Security > Users > [your user name])

Note: The ordering of the Subject DN in the External name is critical and should follow the highest to lowest level precedence, e.g.

O=MarkLogic,OU=App Users, CN=user1

And not

CN=user1,OU=App Users,O=MarkLogic

If you are unsure you can use the OpenSSL command below to list the Subject DN in the expected order:

>$ openssl x509 -in user1.pem -subject -noout 
subject= /O=MarkLogic/OU=App Users/CN=user1

Further reading

(6 vote(s))
Not helpful

Comments (0)