A Personal Certificate Authority

As mentioned in my previous post about my whitehat lab, I've automated the creation of TLS client and server certificates using a personal certificate authority (CA). This post describes that effort in more detail.

Goals

Recently, I've set up some HTTPS services in my home lab, and I wanted a way to:

• Avoid having to click-through browser certificate warnings.
• Be able to use mutual TLS to secure access to the services instead of using passwords.
• Generate the certificates with a single command without having to remember a bunch of command line switches.
• Store the CA private keys encrypted at rest without using an ad-hoc password-based encryption mechanism.

To meet these goals, I decided to use the 1Password CLI, and the step CLI tool. I had used step before this project, but this was my first time using the 1Password CLI, called op.

Why Not Use Let's Encrypt?

Since my home lab is not publicly accessible (by design), using Let's Encrypt is not possible as that requires services to be accessible in some way over the public Internet, either via DNS or HTTP.

In theory, I could run a local automated certificate management environment (ACME) certificate authority to handle this automatically. But, that's quite complex for just a handful of HTTPS services in my home lab.

Prerequisites

I installed both the CLIs using HomeBrew:

$ brew install step
$ # Per https://developer.1password.com/docs/cli/get-started#install
$ brew install --cask 1password/tap/1password-cli

Creating the Certificate Authority

There are some initial steps required to create the certificate authority (CA). I chose to skip automating this step as it should only be required when I choose to re-generate the certificate authority certificates, which hopefully should be a rare event.

The private keys are stored in 1Password as documents. I also created a dedicated vault to store the files, calling it CA.

I started from the steps documented in the step CLI docs to generate a root CA and an intermediate CA. I changed the commands to generate an RSA key instead of an EC key because there is still some software out there that struggles with EC keys.

Here are the commands with comments in-line:

$ # Generate the root CA first

$ ## Generate a password to encrypt the key at rest and store it on
$ ## the clipboard. At password prompts, paste from the clipboard.
$ openssl rand -base64 21 | pbcopy

$ ## Generate the cert and key, using the password from the clipboard
$ step certificate create --kty RSA \
    --profile root-ca "mcnulty CA" root_ca.crt root_ca.key
Please enter the password to encrypt the private key:
Your certificate has been saved in root_ca.crt.
Your private key has been saved in root_ca.key.

$ ## Decrypt and store the private key in 1Password.
$ ## Paste the password at the prompt and
$ ##  authorize the use of the 1Password CLI.
$ openssl rsa -in ./root_ca.key | \
    op document create \
        --file-name root_ca.key \
        --title Root \
        --vault CA \
        -
Enter pass phrase for ./root_ca.key:
writing RSA key
{"uuid":"emtkabkwmsa37bu2jfzmvlyrz","createdAt":"2023-05-08T16:19:52.455295-05:00","updatedAt":"2023-05-08T16:19:52.455295-05:00","vaultUuid":"i6nhfibmj34k43ew3qybxsd3de"}

$ # Generate the client and server intermediate CAs, using the root CA
$ step certificate create 'mcnulty Client Intermediate CA' \
    --kty RSA \
    client_intermediate_ca.crt client_intermediate_ca.key \
    --profile intermediate-ca \
    --ca ./root_ca.crt \
    --ca-key ./root_ca.key
Please enter the password to decrypt ./root_ca.key:
Please enter the password to encrypt the private key:
Your certificate has been saved in client_intermediate_ca.crt.
Your private key has been saved in client_intermediate_ca.key.

$ ## Decrypt and store the private key in 1Password.
$ ## Paste the password at the prompt.
$ openssl rsa -in ./client_intermediate_ca.key | \
    op document create \
    --file-name client_intermediate_ca.key \
    --title 'Client Intermediate' \
    --vault CA \
    -

$ ## Repeat for the server intermediate CA
$ step certificate create 'mcnulty Server Intermediate CA' \
    --kty RSA \
    server_intermediate_ca.crt server_intermediate_ca.key \
    --profile intermediate-ca \
    --ca ./root_ca.crt \
    --ca-key ./root_ca.key
Please enter the password to decrypt ./root_ca.key:
Please enter the password to encrypt the private key:
Your certificate has been saved in server_intermediate_ca.crt.
Your private key has been saved in server_intermediate_ca.key.
$ openssl rsa -in ./server_intermediate_ca.key | \
    op document create \
        --file-name server_intermediate_ca.key \
        --title 'Server Intermediate' \
        --vault CA -
Enter pass phrase for ./server_intermediate_ca.key:
writing RSA key
{"uuid":"kc4tat5qi5tf6cc5743wtvkuvm","createdAt":"2023-05-08T16:19:53.654111-05:00","updatedAt":"2023-05-08T16:19:53.654111-05:00","vaultUuid":"i6nhfibmj34k43ew3qybxsd3de"}

$ # Remove the private keys from the filesystem
$ rm -f *.key

These commands store three documents in the 1Password CA vault: Root, Client Intermediate, and Server Intermediate. Two intermediates were generated to segment the client and server use cases.

I have stored the corresponding certificates for the CA in a Git repository alongside the script to generate certificates, described in the next section.

Generating the Certificates

With the private keys for the CAs in a 1Password vault, I use the following script to generate TLS certificates, storing the resulting private key in the 1Password vault.

#!/bin/bash
#
# A script to generate a new client or server cert, signed by an
# intermediate CA private key stored in 1Password.
#
# The script assumes that the certificates are available in a
# sibling directory called ca-certs, and the private keys are stored
# in a 1Password vault called CA.
#
SCRIPT_DIR=$(dirname "$(realpath "${BASH_SOURCE[0]}")")
set -ex

function usage() {
  echo "Usage: $0 <server|client> subject-alt-name"
  exit 1
}

if [ $# -ne 2 ]; then
  usage
fi

mode=$1
san=$2

op_item_name=
itm_crt=
case "$mode" in
  server)
    op_item_name="Server Intermediate"
    itm_crt="$SCRIPT_DIR/ca-certs/server-intermediate-ca.crt"
    ;;
  client)
    op_item_name="Client Intermediate"
    itm_crt="$SCRIPT_DIR/ca-certs/client-intermediate-ca.crt"
    ;;
  *)
    usage
    ;;
esac

function cleanup() {
  rm -f ./intm.key "./${san}.key"
}

op document get --vault CA "$op_item_name" --output=./intm.key
trap cleanup EXIT

step certificate create "$san" "./${san}.crt" "./${san}.key" \
  --kty RSA \
  --profile leaf \
  --no-password \
  --insecure \
  --not-after 8760h \
  --ca "$itm_crt" \
  --ca-key ./intm.key \
  --bundle

op document create --vault CA "./${san}.key" --title "${san}"

The important parts of the script are getting the private key from the 1Password vault (line 45), creating the certificate using the key (line 48), and uploading the generated private key to the vault (line 57).

An example execution is:

gen-cert.sh server service.lan.test

Since the step tool does not support taking the private key via stdin like openssl does, the keys are stored temporarily on the filesystem. The trap shell builtin is used to ensure that those keys are cleaned up when the script exits. More on this topic in Limitations.

Using the Certificates

Server

Most HTTPS services require the private keys to be present on the filesystem. So, when building the images/containers/etc. for the HTTPS services, the automation will invoke the 1Password to retrieve the key and temporarily store it in the filesystem for use by the build.

To access these services without getting TLS certificate warnings or errors, I've added the root CA to my login keychain via KeyChain Access.

Client

To use client TLS certificates, they often need to be packaged as a PKCS#12 (.p12) file. The following script can be used to generate the .p12, using a private key stored in 1Password and the certificate stored in the current working directory.

#!/bin/bash
#
# A script to package a PKCS #12 file using a private key stored in 1Password
# and a certificate stored in the current working directory.
#

if [ $# -ne 1 ]; then
  echo "Usage: $0 <san>"
  exit 1
fi

san=$1

function cleanup() {
  rm -f "./${san}.key"
}
trap cleanup EXIT

if [ ! -f "./${san}.crt" ]; then
  echo "./${san}.crt does not exist."
  exit 1
fi

set -ex

op document get --vault CA "${san}" --output="./${san}.key"
openssl pkcs12 -export -inkey "./${san}.key" -in "./${san}.crt" > "./${san}.p12"

Similar to the previous script, the trap builtin is used to ensure that the private key is removed when the script exits. openssl will prompt for a password to encrypt the .p12. As I typically just import the .p12 into a keychain, I use the same openssl | pbcopy trick as before to avoid having to come up with a password. Here is an example execution:

$ openssl rand -base64 21 | pbcopy

$ # Paste the password at the prompts
$ ./get-p12.sh client
+ op document get --vault CA client --output=./client.key
/_redacted_/client.key
+ openssl pkcs12 -export -inkey ./client.key -in ./client.crt
Enter Export Password:
Verifying - Enter Export Password:
+ cleanup
+ rm -f ./client.key

The client.p12 file is generated after this execution.

The HTTPS service requesting client certificates needs to trust the root CA. Usually this means copying the root CA certificate to the same host as the HTTPS service and configuring the server with the path to the certificate. A future post will show how nginx can be used to both terminate TLS and require mutual TLS.

Limitations

Private Keys Temporarily in Plaintext

Both step and openssl require the private keys to be present in a file. This is less than ideal. It would be better if these tools could read keys from arbitrary file descriptors to allow shell redirection to be used instead. This approach would eliminate the need to store private keys on the filesystem, even if just temporarily.

No Revocation Support

Since there is no corresponding server portion of the CA, the CA described by this post does not support any form of certificate revocation.

If a private key is compromised, this would allow an attacker to perform a man-in-the-middle attack for the HTTPS service that is using the private key. To mitigate this issue, the HTTPS service could be moved to a new DNS name and a new certificate issued for that name. The old name could never be reused.

Since this is just for use in my home lab, I think this limitation is an acceptable risk.

Future Steps

Tighter Integration with step

In theory, this functionality could be embedded into the step CLI as it has an extension point to interact with other similar private key vaulting technologies via its step-kms-plugin extension point. I haven't dug too deep into this idea yet, but I think it would remove the need to have a wrapper script because the step tool would invoke 1Password directly. This would also likely remove the need to store the private keys temporarily on the filesystem.

Integrate Directly with Keychain Access

It would be nice if the gen-p12.sh script could take an option to automatically import the .p12 into a keychain. This would eliminate the need to prompt for a password, as the script could generate and use a temporary password.