SSL Certificate Format Definition and Converting examples using openssl

This is more for my reference, but I thought I would share the information for those that struggled with this as I did.  The first section defines the different types of formats that can be used for certificates, and when / how / and by who they are used.  The second section provides examples of how to convert between the different formats using openssl.

Certificate Format Definitions

PEM Format

The PEM format is the most common format that Certificate Authorities issue certificates in. PEM certificates usually have extentions such as .pem, .crt, .cer, and .key. They are Base64 encoded ASCII files and contain “—–BEGIN CERTIFICATE—–” and “—–END CERTIFICATE—–” statements. Server certificates, intermediate certificates, and private keys can all be put into the PEM format.

Apache and other similar servers use PEM format certificates. Several PEM certificates, and even the private key, can be included in one file, one below the other, but most platforms, such as Apache, expect the certificates and private key to be in separate files.

DER Format

The DER format is simply a binary form of a certificate instead of the ASCII PEM format. It sometimes has a file extension of .der but it often has a file extension of .cer so the only way to tell the difference between a DER .cer file and a PEM .cer file is to open it in a text editor and look for the BEGIN/END statements. All types of certificates and private keys can be encoded in DER format. DER is typically used with Java platforms. The SSL Converter can only convert certificates to DER format. If you need to convert a private key to DER, please use the OpenSSL commands on this page.

PKCS#7/P7B Format

The PKCS#7 or P7B format is usually stored in Base64 ASCII format and has a file extention of .p7b or .p7c. P7B certificates contain “—–BEGIN PKCS7—–” and “—–END PKCS7—–” statements. A P7B file only contains certificates and chain certificates, not the private key. Several platforms support P7B files including Microsoft Windows and Java Tomcat.

PKCS#12/PFX Format

The PKCS#12 or PFX format is a binary format for storing the server certificate, any intermediate certificates, and the private key in one encryptable file. PFX files usually have extensions such as .pfx and .p12. PFX files are typically used on Windows machines to import and export certificates and private keys.

When converting a PFX file to PEM format, OpenSSL will put all the certificates and the private key into a single file. You will need to open the file in a text editor and copy each certificate and private key (including the BEGIN/END statments) to its own individual text file and save them as certificate.cer, CACert.cer, and privateKey.key respectively.

OpenSSL Command syntax examples

OpenSSL Convert PEM

Convert PEM to DER

openssl x509 -outform der -in certificate.pem -out certificate.der

Convert PEM to P7B

openssl crl2pkcs7 -nocrl -certfile certificate.cer -out certificate.p7b -certfile CACert.cer

Convert PEM to PFX

openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt -certfile CACert.crt

OpenSSL Convert DER

Convert DER to PEM

openssl x509 -inform der -in certificate.cer -out certificate.pem

Openssl Convert CRT

Convert CRT to PEM

openssl x509 -in mycert.crt -out mycert.pem -outform PEM

OpenSSL Convert P7B

Convert P7B to PEM

openssl pkcs7 -print_certs -in certificate.p7b -out certificate.cer

Convert P7B to PFX

openssl pkcs7 -print_certs -in certificate.p7b -out certificate.cer

openssl pkcs12 -export -in certificate.cer -inkey privateKey.key -out certificate.pfx -certfile CACert.cer

OpenSSL Convert PFX

Convert PFX to PEM

openssl pkcs12 -in certificate.pfx -out certificate.cer -nodes

This information was taken almost directly from the following link.  Thank you SSLShopper for providing this information.

https://www.sslshopper.com/ssl-converter.html

Create and Apply a certificate for Apache web server

I have been asked several times lately how to generate and deploy certificates for web servers.  I am going to explain here how to set a certificate for an Apache web server.

Most users of Linux / LAMP servers and similar will find themselves needing to generate a certificate request to be submitted to a certificate authority. Because of the complexity of certificates, the various formats, and the fact that some users will want to submit a csr generated by a Unix platform to a Microsoft Certificate authority and then use the resulting certificate on the Unix appliance or server. This example will show how quickly and easily you can generate a certificate request file.

Start by issuing the following command on the console of the LAMP server:

openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr

A couple of things to note
First, the rsa:2048 value. This can really be whatever you would like, but at the time of writing this was the standard key size. Second, for those that would like to use dsa that will require setting a pass-phrase that will consequently have to be input every time the server is started.

Another thing to note is the names of the files. The generic name [server] was used here, but I have found that it helps keep certificates straight if you just name them after the URL you are generating them for. For example if the website you are creating a certificate for is http://www.contoso.com, then the file names would be www.contoso.com.key and www.contoso.com.csr respectively.

Finally, the -nodes command removes the pass-phrase.

This command will generate 2 files a key file or private key and a csr file or certificate request. The private key should be kept very secure. If this file is compromised, the server identity can no longer be verified as accurate and using the private key others will be able to decrypt the data between your server and the clients.

Using the csr file, you can either open the file and copy and paste the contents into the certificate authority or you may be able to simply upload the file to the certificate authority. Once you have generated the certificate and you have it in the right format (normally pem), you can use it and the private key to finish setting up your website to use SSL.

Wireless Authentication with Certificates

I love putting in extra effort if it will make things go more smoothly in the future.  Certificate authentication is one of those things that can really take away a lot  of the headache of wireless network authentication.  In order to really achieve effective usability with certificates, you want to make sure that it will work with all devices.  Simply using certificate authentication for domain joined Windows based PCs is just going to frustrate all involved.  This solution should incorporate all devices in order to be successful.  Recently I had something trip me up for quite a while actually.  When generating a certificate never use another account to request the certificate.

I thought if I used my administrator account it would make things easier and go more smoothly since being an administrator gave me more options when generating certificates.  That was my mistake.  I used an administrator account to log on to the certificate request site (this setup is using Microsoft certificate server), and from there I supplied the Certificate Signing Request (CSR) and I requested a user certificate.  When these were generated, they were using the username of the administrative account I supplied not the username I was creating the certificate for.  When creating a certificate via the web interface, always use the account you are generating the certificate for, or authentication will fail every time as the certificate is associated with the authenticated user.

Linux example:

Since I don’t have a good way to push certificates to a Linux platform, I use a manual process.  Linux requires that you have a certificate, the private key, and the certificate authority.  It is also going to ask for an “identity”.  This identity will be the user that you are using to authenticate this connection.  Ironically, it doesn’t use this identity when validating the certificate.  This became very important when I used an administrative account to generate the certificate for the laptop but Microsoft Network Policy Server identifies the certificate belonging to the administrative account not to the identity supplied when connecting.

So, it is important to remember that when using a Microsoft Certificate Authority to generate your certificates, log on to the website with the account that you will be using to authenticate to NPS server.

When you create a user certificate the following steps work for me.  I began by generating a certificate request and corresponding key using the following command:

openssl req -new -newkey rsa:2048 -keyout server.key -out server.csr

Of course there are many ways that this can be accomplished, this is simply an example.  It is important to note that settting a passcode for this key and csr is important because the wireless connection wizard will not allow you to connect using a key file that is not protected with a password.  It also makes the key file more secure and is a good practice.  Once I had the key file and certificate signing request file, I copied the contents of the certificate request file and using a web browser headed over to my certificate authority web interaction page. (http\s://servername/certsrv) and entered the username and password of the user account that would be using this certificate.

Once you have logged on, this is the page you will be provided.
Once you have logged on, this is the page you will be provided.

I then selected request certificate,

These are the options available. Request Certificate is the link that allows you to obtain a certificate.
These are the options available. Request Certificate is the link that allows you to obtain a certificate.

and advanced certificate request on the next page.

this is the option you want to use when you have a certificate signing request that you need to provide to generate your certificate.
this is the option you want to use when you have a certificate signing request that you need to provide to generate your certificate.

This provides a box to paste the certificate signing request into and allows for a couple other options.

The type of certificate we are looking for is a user certificate.
The type of certificate we are looking for is a user certificate.

One of the options is to set the type of certificate you would like to generate.  “User” should be selected here.  The rest of the defaults should be fine for the certificate creation.

Once you have completed all the information, submit the request to obtain the certificate.
Once you have completed all the information, submit the request to obtain the certificate.

Once you submit the request and the certificate server generates the required certificates, you can save them and get ready to finish setting everything up.

Since Linux only likes certificates in the pem format, and since Microsoft generates them in the cer format, a quick conversion will be necessary to complete the steps.  Using the command below, the certificate will be converted to a new pem file.

openssl x509 -inform der -in certificate.cer -out certificate.pem

Now, when connecting, select your authentication type and make sure you choose TLS for certificate authentication.  You will need to supply the key file, the pem file, and the certificate authority file (which can be obtained from the certificate authority server that you generated the certificate from.  It will also need to be converted to a pem file.  Finally, you will need to provide the identity that will be used to connect.

These are the options that will be presented to you when connecting using certificate based authentication.
These are the options that will be presented to you when connecting using certificate based authentication.

That should be all that you need to successfully authenticate to a wireless network using certificates with a Linux based OS.  Connecting using a Mac platform is very similar, but some of the steps are slightly different.

This is what things will look like with a successful connection (if using Gnome 3 interface)
This is what things will look like with a successful connection (if using Gnome 3 interface)

Intermediate Certificates and Apache

If you are using 3rd party certificates, you more than likely need to install an intermediate certificate as well when installing certificates for your website.  The following are the 3 directives that are required and an explanation of each.  These should be placed in the virtual host section / file of the website.

Your virtual host section will need to contain the following (3) directives:

    • SSLCertificateChainFile – This will need to point to the appropriate Intermediate root CA certificates.
    • SSLCertificateFile – This will need to point to the end entity certificate (the actual certificate used for the website)
    • SSLCertificateKeyFile – This will need to point to the private key file associated with your certificate.
  1. Save the changes to the file and quit the text editor
  2. Restart Apache.

MobileIron SCEP Configuration Settings Defined

Option

Description

Name Enter text that identifies this group of SCEP settings
Description Enter additional text that clarifies the purpose of this group of SCEP settings
Enable Proxy Select Enable Proxy. The following proxy options are available:
Cache locally generated keys—leaves the certificate in the VSP certificate store for reuse.
User Certificate—User-based certificates are used for all devices. If you select
this option, revoking the certificate renders all the associated user’s devices as unauthorized.
Device Certificate—For device-based certificates, an individual certificate is
created per device. If you select this option, revoking the certificate renders
only the specific device associated with the certificate as unauthorized.
If you choose to disable the proxy functionality, the certificate will be
generated by the device. This configuration is not supported.

Setting Type

SCEP—select this option if you are using standard certificate-based authenti- cation using a separate CA.

Local—select this option if you are using the MobileIron VSP as the CA. Symantec Managed PKI—select if you are using Symantec’s SCEP solution.

URL

Provide the URL necessary to access your SCEP server. Typically, http://<your_ndes_server>/certsrv/mscep/mscep.dll

For iOS: Note that iOS does not support https with self-signed certificates. Therefore, should you choose to use https, you must have a trusted certificate installed for the portal certificate in order for provisioning to function properly.

Subject

Enter an X.509 name represented as an array of OIDs and values. Typically, set this to the user’s fully qualified domain name. For ease of configuration you can use the $USER_DN$ variable to populate the Subject with the user’s FQDN.

Subject Common Name Type

Select the CN type specified in the certificate template

Subject Alternative Name Type

Select NT Principal Name

Subject Alternative Name Value

Select $USER_UPN$

Key Size

Select the key size (1024, 2048, or 4096)

Key Usage

Specify acceptable use of the key (signing and/or encryption)

Finger Print

Leave this field blank

Challenge Type

Select None, Microsoft SCEP, or Manual to specify the type of challenge to use

Challenge

For a Manual challenge type, enter the pre-shared secret the SCEP server can use to identify the request or user.

Challenge URL

For a Microsoft SCEP challenge type, enter the URL of the trust point defined for your Microsoft SCEP CA.

User Name

Enter the user name for the Microsoft SCEP CA.

Password

Enter the password for the Microsoft SCEP CA.

 

Configuring Certificates for QNAP NAS

Goal

The ability to set and configure SSL certificates that don’t display warnings to visitors is an important, but sometime frustrating process.  This guide will help anyone trying to create and apply a certificate to a QNAP NAS device.  I welcome and appreciate comments that provide issues that have been encountered with this process.  Being familiar with certificates in general, I may have missed some of the common issues encountered when attempting to apply SSL certificates.

Background / Additional Information

A couple of notes about this process.

While self-signed SSL certificates can be generated locally, it defeats the purpose of changing or applying new certificates as visitors will more than likely not be able to verify the validity of the certificate.  This guide is for generating a csr to provide to a 3rd party certificate authority like StartSSL.

Following the guide to generate certificates for Linux Apache web servers found here will be very important as this guide doesn’t detail the creation of the private key, certificate signing request, or submission of the csr to the certificate authority to generate the final certificate.  It also doesn’t cover converting the returned certificate file from cer format to pem format.  This process can be found here.

It should also be noted that some understanding of certificates and how they work is required to complete this process.

Resolution

To generate a csr file to be submitted to a certificate authority, verify that openssl is installed and if it is not, download and install it.  The following commands when run on the QNAP NAS will install openssl.

ipkg update
ipkg install openssl

Once openssl has been successfully installed, the process for generating a certificate signing request (csr) and submitting it to a certificate authority is very similar to the process on a Linux server running Apache.  For details on this process, I have created a document previously on this topic found here.  The commands for generating the csr, and converting the certificate file from cer to pem are identical.  The only difference is the application of the certificate.  The QNAP provides a user friendly web interface that is used to supply the private key and pem certificate file.  These are text boxes and require the private key and certificate file to be pasted into them and then applied.

A quick note, when issuing the openssl command, you may receive a warning message like the following: “Can’t open config file:  /usr/local/openssl.cnf”.  This indicates that the system either can’t find the file, or it is not able to access the file.  Determine which is the case and if it not in the specified path, perform a search to determine the location and copy it to the correct location, or check the permissions of the file and update as necessary.  A file may be found named openssl.cnf.example.  This file can be used by renaming it (I recommend copying it to the new name so you retain the example file) and modifying as needed.

While it might be assumed that applying your own certificate to a QNAP NAS might be a difficult process, if you have any experience with applying certificates, it should be no problem at all.

Troubleshooting

No troubleshooting steps have been defined at this point.

MobileIron Wireless Configuration for Certificate Authentication Troubleshooting Topics

Problem

Setting up certificate based authentication can be extremely difficult and frustrating.  It is great when it works, but getting to that point can be a slow and painful process if inexperienced with implementation and general knowledge of certificates.  This document is meant to serve as a guide for troubleshooting the “simple” things that seem to prevent successful implementation of certificate authentication for wireless networks, and specifically as it relates to MobileIron configurations of mobile devices.

Background / Cause

Many organizations are moving toward ease of administration, better security, and better overall end-user experience.  MobileIron provides all of these for mobile devices, but getting the right configurations can sometimes be more difficult than originally anticipated even with a solid product like MobileIron.  Certificate authentication has many benefits, however, easy setup and implementation is not one of them.  This has caused a need to review and document troubleshooting steps when things go wrong with certificate authentication.

Resolution

Known Issues / Troubleshooting

Problem: |  SSID is not showing up for a hidden network, and the device is not connecting to the wireless network.

Solution: |  In this scenario, 2 things are present and should be noted.

  • The SSID is hidden
  • The device, even though a profile has been applied with all the configuration information fails to connect to the network.

First, if the SSID of a network is hidden, on the configuration page of a wireless network configuration for that SSID, the check box “Hidden Network” must be selected.  Failure to select this option will prevent the device from even looking for the SSID.

hidden network settings

Second, if manually typing the SSID into the “other” option for network selection on the device seems to work, but fails initially, or complains about a certificate, the trusted certificate section has been incorrectly configured in for the wireless configuration.  The mobile device attempts to verify the identity of the device / server at the other end of the connection, but if the certificate of that device / server is not trusted, and / or the certificate common name is not listed in the “trusted certificates” list in the configuration, the connection will fail.

trusted certificate names

To resolve these issues, verify the “Hidden Network” check box is selected for wireless networks using the hidden SSID option.  Attempting to connect to a hidden network without this selected will fail to automatically connect every time.  This is important for hidden networks.  Save the configuration and force the device to check in.

The certificate should be verified by the device for it to connect without issue.  Once a certificate has been verified it will continue to connect.  However, the initial connection will pop-up a warning asking for end-user interaction to approve the certificate the device is not able to verify.  Most organization want to avoid this extra step for end-users even if it is only a one-time occurrence.  In order for this to happen, the certificate must be from a valid 3rd party trust source (internal certificate authorities are acceptable as long as the mobile devices trust that source).  Next, the common name of the device or server displayed on the certificate needs to be added to the “Trusted Certificate Names” list of the configuration.  The “Allow Trust Exceptions” will not “fix” this problem (at least with iOS devices) so it is important that the common name on the certificate of the device or server is put in the “Trusted Certificates” list.

Problem: |  An existing configuration for the Wireless network already exists on the device.  If the device has already been configured to access the wireless network, the profile fails to be properly installed and if a selective wipe is sent to the device, the wireless configuration will remain on the device.

Solution: |  Do not allow wireless access to mobile devices without acceptance of company policies and management of the devices in MobileIron.  This is where using certificate authentication is beneficial.

References

none

Exchange 2010 and Single Name Certificates

Goal / Scope

The purpose of this article is to provide a method for leveraging a single name certificate with Exchange 2010 without getting errors and warnings and potentially having some services fail altogether.  Please NOTE:  It is Microsoft recommended best practice to use a SAN (Subject Alternative Name) certificate with Exchange.  However, if for some reason this is not an option, maybe due to cost, or wanting to leverage an existing certificate, the process below will allow a single name certificate to be used with Exchange.

Notes:
Again, this is likely not a Microsoft supported configuration. While successfully rolled it out in small environments with success, if a SAN certificate can be purchased and used, that is the best case.

Background

Below are some of the things that need to be in place for this configuration to be successful.

Prerequisites

  • An external DNS provider that supports SRV records. You’ll need to insert an SRV record of _autodiscover._tcp.domain.com in DNS for this to work
  • Outlook 2007 with the update rollup released June 27, 2007 (Discussed in this Microsoft KB article) to provide support for Exchange Autodiscover via SRV lookup
  • An SSL certificate for mail.domain.com

Methodology / Process Steps

In the examples shown below, [domain.com] is the domain name, [mail.domain.com] is the URL being set for all services and EXCHANGE is the NetBIOS name of the Exchange server.

  1. Point external DNS for mail.domain.com to the external IP address of the Exchange server.
  2. Create the SRV record _autodiscover._tcp.domain.com with content of [mail.domain.com] on port 443. Your DNS provider might also have you enter it like this:
    Service: _autodiscover
    Protocol: _tcp
    Port Number: 443
    Host: [mail.domain.com]
  3. Point internal DNS for [mail.domain.com] to the internal IP address of the Exchange server.
  4. Set the Internal URLs.

    Note:  These examples can be copied and pasted into a text editor.  Simply replacing [mail.domain.com] with the correct FQDN of the Exchange server and paste the new commands directly into a PowerShell window on the Exchange 2010 server.

    Get-AutodiscoverVirtualDirectory | Set-AutodiscoverVirtualDirectory –InternalUrl "https://mail.domain.com/Autodiscover/Autodiscover.xml"
    Get-ClientAccessServer | Set-ClientAccessServer –AutodiscoverServiceInternalUri "https://mail.domain.com/Autodiscover/Autodiscover.xml"
    Get-WebservicesVirtualDirectory | Set-WebservicesVirtualDirectory –InternalUrl "https://mail.domain.com/Ews/Exchange.asmx"
    Get-OabVirtualDirectory | Set-OabVirtualDirectory –InternalUrl "https://mail.domain.com/Oab"
    Get-OwaVirtualDirectory | Set-OwaVirtualDirectory –InternalUrl "https://mail.domain.com/Owa"
    Get-EcpVirtualDirectory | Set-EcpVirtualDirectory –InternalUrl "https://mail.domain.com/Ecp"
    Get-ActiveSyncVirtualDirectory -Server $CASserver | Set-ActiveSyncVirtualDirectory -InternalUrl "https://mail.domain.com/Microsoft-Server-ActiveSync"
  5. Set the External URLs.

    Get-AutodiscoverVirtualDirectory | Set-AutodiscoverVirtualDirectory –ExternalUrl "https://mail.domain.com/Autodiscover/Autodiscover.xml"
    Get-webservicesVirtualDirectory | Set-webservicesVirtualDirectory –ExternalUrl "https://mail.domain.com/Ews/Exchange.asmx"
    Get-OabVirtualDirectory | Set-OabVirtualDirectory –ExternalUrl "https://mail.domain.com/Oab"
    Get-OwaVirtualDirectory | Set-OwaVirtualDirectory –ExternalUrl "https://mail.domain.com/Owa"
    Get-EcpVirtualDirectory | Set-EcpVirtualDirectory –ExternalUrl "https://mail.domain.com/Ecp"
    Get-ActiveSyncVirtualDirectory | Set-ActiveSyncVirtualDirectory -ExternalUrl "https://mail.domain.com/Microsoft-Server-ActiveSync"
  6. Verify everything was set correctly
    Get-AutodiscoverVirtualDirectory | ft Identity,InternalURL,ExternalUrl
    
    Get-webservicesVirtualDirectory | ft Identity,InternalURL,ExternalUrl
    
    Get-OabVirtualDirectory | ft Identity,InternalURL,ExternalUrl
    
    Get-OwaVirtualDirectory | ft Identity,InternalURL,ExternalUrl
    
    Get-EcpVirtualDirectory | ft Identity,InternalURL,ExternalUrl
    
    Get-ActiveSyncVirtualDirectory | ft Identity,InternalURL,ExternalUrl
  7. Verify everything is working by using the Exchange Remote Connectivity Analyzer located at https://www.testexchangeconnectivity.com

Known Issues / Troubleshooting

This section is for the issues that have well defined and tested solutions.

Problem: | Certificate errors are still present, services fail to function properly like OAB for example

Solution: | Use the Outlook Connectivity Test (found by pressing Ctrl on the keyboard and right clicking the Outlook icon with the mouse.  One of the options will be “Test E-mail AutoConfiguration …”.  After supplying username and password information, it attempt an autodiscover email configuration and supply results of the test in the window.  “Connection Status” is also a useful utility and can be opened using the same method.  This will display the connection status of the Outlook client.

References

The PowerShell code used in this guide was taken directly from this article:

http://blog.cohesivelogic.com/exchange-2010-single-name-ssl-certificates

Extracting Certificate and Private Key Files from a .pfx File

Purpose

There may be scenarios when a certificate file and private key file are required, but only a single .pfx file is available.  A Microsoft based Windows platform doesn’t provide a way to complete this process.

It is still possible to obtain the required files.  By exporting the certificate from the Windows Certificate Store using the Windows MMC into a single .pfx file, separate certificate and private key files can be created from this .pfx file.  Below is the process beginning once the .pfx file has been obtained.

Procedure

  • The first step is to copy the exported file (e.g. certname.pfx)  to a system where OpenSSL is installed.
    NOTE: *.pfx files are in PKCS#12 format and include both the certificate and the private key.
  • The following command will export the private key:
openssl pkcs12 -in certname.pfx -nocerts -out key.pem -nodes

 

  • The next command will export the certificate:
openssl pkcs12 -in certname.pfx -nokeys -out cert.pem

 

  • Finally, the last command will remove the passphrase from the private key: (if needed)
openssl rsa -in key.pem -out server.key

Known Issues / Troubleshooting

This section is for the issues that have well defined and tested solutions.

Problem: |

Solution: |

Problem: |

Solution: |

References

Open SSL pkcs#12 Commands