Module 02 — Keys, Encodings & File Formats

Updated 20 August 2026

Module 02 · Keys, Encodings & File Formats

In Module 01 you made a key and noticed it came out as strange Base64 text. This module opens that file up. By the end you will know what is inside a key, why the same key can be saved in five different file formats, and how to convert between them without breaking anything.

🧠 concept → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (answers hidden)

Prerequisite: Module 01. You already generated a keypair (Exercise B2.1), saw that the private file was much bigger than the public one, and used openssl pkey -pubout. This module explains why all of that is true.


The picture to hold in your head for this whole module.

Think of a padlock and its key.

  • You give open padlocks away to anyone who asks. That is your public key. It does not matter who has one.
  • You keep the one key that opens them in your pocket, and never give it to anybody. That is your private key.

Someone can lock a box with your padlock and send it to you. Only you can open it. That is Module 01's Exercise B2.1, in physical form.

This module is about the padlock and the key as files on disk — what they contain, how they are written down, and what happens when two programs disagree about how to write them down.

Part A · What is inside a key file

A1 · The two kinds of key you will meet

There are two families of key in everyday use. They do the same job. They just use different maths to do it.

The analogy. Both are padlocks. They are simply different brands.

RSA is the big, old, heavy padlock. It has been around since 1977, it works everywhere, and everybody knows how to use it. To be strong it has to be physically large — that is why RSA keys are 2048 or 4096 bits.

Elliptic Curve (EC) is the modern compact padlock. It is a fraction of the size and just as hard to break, because it is built on cleverer engineering. A 256-bit EC key is about as strong as a 3072-bit RSA key.

Smaller padlock, same strength, lighter to carry. That is the whole trade.

Key typeCommon sizeWhen you will see it
RSA2048-bitThe safe default. Works with absolutely everything, including very old clients
RSA4096-bitUsed for root CAs and long-lived keys. Slower, and rarely worth it for a web server
EC (P-256)256-bitThe modern choice for web servers. Small, fast, supported by every current browser
EC (P-384)384-bitHigher-security setups, some government and banking requirements
Ed25519256-bitExcellent, very small, very fast — but not allowed in public web certificates yet. Common for SSH
A trap worth knowing before you meet it. "Bigger number means stronger" is true inside one family and completely false between families.

A 256-bit EC key is not weaker than a 2048-bit RSA key. It is roughly stronger. The numbers count different things, so comparing them directly is like comparing a shoe size to a hat size.

If someone says "we require 4096-bit keys" as a blanket rule, they have usually copied it from a checklist without knowing this.

🧪 Exercise A1.1 — Make one of each and compare them
bash
mkdir -p ~/tls-lab/m02 && cd ~/tls-lab/m02
umask 077

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa2048.key
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out rsa4096.key
openssl genpkey -algorithm EC  -pkeyopt ec_paramgen_curve:P-256 -out ec256.key
openssl genpkey -algorithm ED25519 -out ed25519.key

ls -l *.key
wc -l *.key
Expected result — click to reveal
plain text
-rw------- 1 zaeem zaeem  119 Aug 20 12:04 ed25519.key
-rw------- 1 zaeem zaeem  241 Aug 20 12:04 ec256.key
-rw------- 1 zaeem zaeem 1704 Aug 20 12:04 rsa2048.key
-rw------- 1 zaeem zaeem 3272 Aug 20 12:04 rsa4096.key

   3 ed25519.key
   5 ec256.key
  28 rsa2048.key
  54 rsa4096.key

What to read out of this.

  • The size difference is enormous. The Ed25519 key is 119 bytes. The RSA-4096 key is 3,272 bytes — about 27 times bigger for no extra security. The EC key is 3 lines of text; the RSA key is 54.
  • This is not just about disk space. The key ends up inside the certificate, and the certificate is sent on every single new connection. A smaller key means fewer bytes on the wire, every time, forever.
  • All four files are 0600, because of the umask 077 from Module 01's Exercise D2.1. Good.

Now imagine this at 500 hosts. A busy site handles millions of new connections a day. Switching from RSA-4096 to EC P-256 removes roughly 3 KB from each one. That is real bandwidth and real time, and it costs you nothing in security.

🧪 Exercise A1.2 — Feel the cost of a big RSA key
bash
time openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out /tmp/t2048.key
time openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out /tmp/t4096.key
time openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out /tmp/tec.key
rm -f /tmp/t2048.key /tmp/t4096.key /tmp/tec.key
Expected result — click to reveal
plain text
# RSA 2048
real    0m0.184s

# RSA 4096
real    0m3.921s

# EC P-256
real    0m0.009s

What to read out of this.

  • Your times will be different every single run. Generating an RSA key means guessing large numbers and testing whether they are prime. Sometimes you get lucky on the third guess, sometimes the hundredth. Run the 4096 command three times and you may see 1 second, then 9 seconds, then 4.
  • EC generation is basically instant because there is no prime hunting involved. It just picks a random number in a range.
  • RSA-4096 took roughly 20–40× longer than RSA-2048 for a security gain almost nobody needs.

🔑 Why this matters in real life. This is a classic cause of "the container takes ages to start". Something generates a fresh RSA-4096 key at boot, on a machine with a thin pool of randomness, and the startup hangs for 30 seconds while everyone hunts for a network problem that does not exist.

🎯 Interview questions — Key types and sizes

Q. RSA or ECDSA — which would you choose for a web server, and why?

ECDSA with a P-256 key, for most public web servers in 2026. It gives the same security as RSA-3072 in a key roughly one twelfth the size, the certificate on the wire is smaller, and the signing operation the server does on every handshake is several times cheaper.

The part that shows real experience is knowing when not to: very old clients (Java 6/7, Windows XP, some payment terminals and embedded devices) do not support ECDSA. If you must serve those, the usual approach is a dual-certificate setup — nginx and Apache can both hold an RSA certificate and an ECDSA certificate at the same time and hand each client whichever it understands.

Worth adding: the key type is chosen when you generate the key, and changing it means a brand-new key and a brand-new certificate. It is not a config toggle.

Q. Is a 4096-bit RSA key twice as secure as a 2048-bit one?

No. Security does not scale with the number. RSA-2048 gives roughly 112 bits of security strength; RSA-4096 gives roughly 152. So you pay about 20–40× the key generation time and a permanently slower signing operation for a modest improvement that no realistic attacker is anywhere near.

RSA-2048 is still considered fine by NIST for general use. RSA-4096 makes sense for a root CA whose key must stay safe for 20 years, and rarely anywhere else.

The framing that lands well: "For a certificate that lives 90 days, key size is almost never the weakest link. Key storage is. I would rather spend the effort making sure the key is 0600, not in git, and rotated automatically."

Q. Why is Ed25519 popular for SSH but not used for web certificates?

Ed25519 is excellent — small, fast, and resistant to several classes of implementation mistake that have caused real bugs in ECDSA. SSH adopted it quickly because SSH controls both ends of the connection.

Public web certificates cannot move that fast. A CA may only issue key types listed in the CA/Browser Forum Baseline Requirements, and browsers must all support the type before it is useful. Ed25519 is not in that allowed set, so no public CA will issue you a certificate for one.

You can still use Ed25519 on a private internal CA, where you control every client — which is a good example of the general rule that internal PKI has far more freedom than public PKI. That comes back in Module 12.


A2 · What is actually inside the file

A private key file is not one number. It is a small bundle of numbers.

The analogy. Think of a cake.

The public key is the finished cake. Anyone can look at it, photograph it, and describe it.

The private key is the full recipe — including the two secret ingredients that nobody else knows.

From the recipe you can always bake the cake again. From the cake, you cannot work backwards to the recipe. That is exactly why openssl pkey -pubout works and there is no command that goes the other way.

For an RSA key, the file holds these pieces:

PieceWhat it isSecret?
modulus (n)The two secret primes multiplied together. A huge number — 617 digits for a 2048-bit keyNo — public
publicExponent (e)Almost always 65537. Used with the modulus to lock thingsNo — public
privateExponent (d)The number that unlocks. This is the actual secretYes
prime1, prime2 (p, q)The two original primes. Whoever has these can rebuild everythingYes
exponent1, exponent2, coefficientPre-calculated shortcuts that make decryption about 4× fasterYes

Now look at the top two rows. The modulus and the publicExponent are not secret — and together they are the public key. So the public key really is sitting inside the private key file, which is why deriving it is instant.

🧪 Exercise A2.1 — Open the file up and look
bash
cd ~/tls-lab/m02
openssl pkey -in rsa2048.key -noout -text | head -30

# Just the public bits, on their own
openssl pkey -in rsa2048.key -noout -text | grep -A2 'publicExponent'

# Now the EC key - a completely different shape
openssl pkey -in ec256.key -noout -text
Expected result — click to reveal
plain text
Private-Key: (2048 bit, 2 primes)
modulus:
    00:bb:54:94:d4:b7:d5:2c:f1:c2:a3:85:f3:bc:e4:
    df:9a:71:0c:2e:88:14:6b:37:c9:d0:52:8f:e1:4a:
    ... (many more lines)
publicExponent: 65537 (0x10001)
privateExponent:
    3f:9a:2c:8e:5b:1d:47:f0:a6:c3:9e:82:b7:4d:10:
    ...
prime1:
    00:e2:8a:4f:1b:9c:07:d5:36:a8:11:c4:7e:93:2b:
    ...
prime2:
    00:d3:e0:91:7b:45:ca:28:f6:01:3b:7e:9d:5a:8c:
    ...
exponent1:
    ...

Private-Key: (256 bit)
priv:
    00:8f:2a:1c:74:b3:e9:05:d1:9c:44:f7:a1:2b:8e:
    60:33:a0:15:7d:3c:e2:b8:44:f9:a6:1d:8e:04:c7:
    b3:52:1f
pub:
    04:9c:8b:31:e7:0a:5d:64:2f:c8:19:b3:7a:e5:40:
    ...
ASN1 OID: prime256v1
NIST CURVE: P-256

What to read out of this.

  • publicExponent: 65537 — and it will be 65537 on essentially every RSA key you ever see. It is not random. It was chosen because it is prime, and because in binary it is 10000000000000001 — only two 1-bits — which makes the maths fast. If you see a different value, someone did something unusual on purpose.
  • prime1 and prime2 are the crown jewels. Multiply them together and you get the modulus. Anyone who obtains these two numbers owns your key completely. This is the actual thing you are protecting with chmod 600.
  • The EC key is tiny by comparison — one priv number and one pub point, then the curve name. No primes, no exponents, no shortcuts. That simplicity is why the file was 241 bytes instead of 1704.
  • NIST CURVE: P-256 — an EC key is meaningless without knowing which curve it lives on, so the curve name is stored in the file. prime256v1 and P-256 and secp256r1 are three names for the same curve, which is a genuinely annoying piece of trivia that catches everyone at least once.
🧪 Exercise A2.2 — Prove the public key is already inside the private key
bash
cd ~/tls-lab/m02

# Pull the public key out, twice, and compare
openssl pkey -in rsa2048.key -pubout -out pub-a.pem
openssl pkey -in rsa2048.key -pubout -out pub-b.pem
cmp pub-a.pem pub-b.pem && echo "IDENTICAL every time"

# The modulus is shared by both halves
echo "private file says:"
openssl pkey -in rsa2048.key -noout -modulus | md5sum
echo "public file says:"
openssl pkey -pubin -in pub-a.pem -noout -modulus | md5sum
Expected result — click to reveal
plain text
IDENTICAL every time

private file says:
9e107d9d372bb6826bd81d3542a419d6  -
public file says:
9e107d9d372bb6826bd81d3542a419d6  -

What to read out of this.

  • Deriving the public key is not a calculation, it is a copy. The modulus and exponent are already sitting in the private file. -pubout just writes them out on their own. That is why it is instant and why the result is identical every time.
  • The two md5sum lines match. The private key and the public key share the same modulus — they are two views of one thing.
  • Remember this trick. Comparing modulus hashes is exactly how you prove a key and a certificate belong together, which is Part D2 of this module and one of the most useful things you will learn all week.

💡 We used md5sum here purely to shorten a very long number for comparison. That is a fine use of MD5 — nobody is attacking it, we are just abbreviating. This is the exact nuance from Module 01's Exercise B4.2.

🎯 Interview questions — Inside a key

Q. What is the role of the public key and private key in an SSL certificate?

The public key is published inside the certificate. Anyone who connects receives it. The private key stays on the server, in a separate file, and is never sent anywhere.

During connection setup the server uses the private key to prove it holds the matching half — in modern TLS by signing part of the handshake. The client checks that signature using the public key from the certificate. An attacker can copy your certificate freely, but without the private key it cannot complete a handshake.

The detail worth adding: the public key is also stored inside the private key file, which is why you can always regenerate a public key or a CSR from a private key, but never the other way round. Lose the certificate and you can reissue; lose the private key and you must start over with a new key.

Q. Why is the RSA public exponent almost always 65537?

Because it is prime, it is large enough to avoid a family of attacks that affect very small exponents like 3, and in binary it has only two bits set — which makes the public-key operation fast.

It is effectively a fixed constant across the entire internet. Seeing anything else means somebody deliberately overrode it, and it is worth asking why.

Q. Someone tells you the private key file was accidentally world-readable for a week. What do you do?

Treat it as compromised. Fixing the permissions does not undo exposure, and there is no way to know whether it was read.

The response, in order: generate a brand-new key, issue a new certificate for it, deploy the new pair, then revoke the old certificate and remove the old key. Notice that revocation comes after the replacement is live, and that — as Module 01's Exercise D3.2 showed — revocation may not actually stop clients, so replacement is the part that genuinely protects you.

Then fix the cause: check whether the key is in git history (permanent — deleting the file does not remove it), check backups and log aggregation, and check whether the same key was reused on other hosts.

What separates a strong answer: saying plainly that you cannot "rotate a password" here. A leaked private key is unfixable except by replacement, and that fact is the whole argument for short certificate lifetimes and automated issuance.


A3 · Locking the private key with a passphrase

So far the private key has been protected only by file permissions. You can also encrypt the file itself with a passphrase.

The analogy. Your house key is in your pocket. That is file permissions.

A passphrase puts that house key inside a small safe. Someone who steals the safe still cannot get in.

But now you have to be standing there, every morning, to open the safe. And that is exactly the problem with passphrases on server keys: the server reboots at 3am and sits there waiting for someone to type it in.

🧪 Exercise A3.1 — Put a passphrase on a key, and take it off again
bash
cd ~/tls-lab/m02

# Encrypt an existing key (it will prompt you twice)
openssl pkey -in rsa2048.key -aes256 -out rsa2048-locked.key

# Look at the difference in the first line
head -1 rsa2048.key
head -1 rsa2048-locked.key

# Try to read it - you will be prompted
openssl pkey -in rsa2048-locked.key -noout -text | head -2

# Take the passphrase back off
openssl pkey -in rsa2048-locked.key -out rsa2048-unlocked.key
Expected result — click to reveal
plain text
-----BEGIN PRIVATE KEY-----
-----BEGIN ENCRYPTED PRIVATE KEY-----

Enter pass phrase for rsa2048-locked.key:
Private-Key: (2048 bit, 2 primes)
modulus:

What to read out of this.

  • The header line changed from BEGIN PRIVATE KEY to BEGIN ENCRYPTED PRIVATE KEY. That one line is how every tool knows to ask you for a passphrase. You can tell at a glance, with head -1, whether a key file is locked.
  • The file is now genuinely encrypted, not just marked. Open it in an editor and the Base64 is completely different from the original.
  • Removing the passphrase is trivial if you know it. openssl pkey -in locked.key -out plain.key and it is gone. A passphrase protects the file at rest — against a stolen backup or a laptop — not against someone who already has the passphrase or who can read the key out of a running process's memory.
🧪 Exercise A3.2 — Watch a passphrase break an automated service

This one is meant to fail. It is the reason most production keys have no passphrase.

bash
cd ~/tls-lab/m02

# Simulate what a service does at startup: read the key without a human present
openssl pkey -in rsa2048-locked.key -noout -text < /dev/null
echo "exit code: $?"

# Now the same thing with the passphrase supplied non-interactively
openssl pkey -in rsa2048-locked.key -passin pass:yourpassphrase -noout -text | head -1
Expected result — click to reveal
plain text
Enter pass phrase for rsa2048-locked.key:
Could not read private key from rsa2048-locked.key
4077A1B2C87F0000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:crypto/store/store_result.c:151:
exit code: 1

Private-Key: (2048 bit, 2 primes)

What to read out of this — this is the practical heart of A3.

With no human to type the passphrase, the read simply fails. That is exactly what happens to nginx, Apache, HAProxy or a Java service when it restarts at 3am with a passphrase-protected key: the service does not come up.

So teams do one of these things, and each has a cost:

  1. No passphrase at all. By far the most common. The key is protected by 0600 and by whoever can reach the machine. Simple, and it restarts cleanly.
  2. Passphrase in a file next to the key (ssl_password_file in nginx). This is security theatre — anyone who can read one file can read the other.
  3. Passphrase supplied by a secret manager at boot — Vault, AWS Secrets Manager, systemd credentials. Real protection, real complexity.
  4. Key never on disk at all — held in an HSM or a cloud KMS, so the server can use it but never read it. This is the strongest answer and the one to mention in interviews.

🔑 The honest summary: a passphrase protects a key at rest. On a running server the key must be usable without a human, so the passphrase usually ends up somewhere the attacker can also reach. That is why the industry moved toward short-lived keys and KMS/HSM storage instead of passphrases.

⚠️ Note the error message: unsupported ... store_result.c. It does not say "wrong passphrase" or "no passphrase given". This is the same terse, unhelpful style you met in Module 01's Exercise B5.2 — crypto tooling tells you that something failed, rarely why.

🎯 Interview questions — Protecting the key

Q. How do you ensure secure storage of private keys?

Layered, and worth giving in order of increasing strength:

  • File level: 0600, owned by the service account, on a directory that is 0700. Never in git — and if it ever was, git history is permanent, so the key is burned.
  • Generation: create the key on the machine that will use it. A key that has been emailed or pasted into Slack has been on someone's laptop and in a message archive.
  • At rest: encrypt backups and disk. A passphrase on the key file helps here but breaks unattended restarts, so most teams use a secret manager instead.
  • Best: the key never exists as a readable file. An HSM or cloud KMS performs signing operations on request and never exports the key. Compromising the server no longer means compromising the key.
  • Blast radius: short certificate lifetimes and automated renewal, so a leaked key expires on its own.

The line that lands: "A private key that has been exposed cannot be un-exposed. So I optimise for making exposure both unlikely and short-lived, rather than relying on any single control."

Q. Should production server keys have a passphrase?

Usually no, and being able to explain why not matters more than the answer.

A passphrase protects the key at rest. But a server must be able to start without a human, so the passphrase has to be readable by the machine — which usually means it sits in a file beside the key, where it protects nothing. You have added an outage risk for no real security.

The cases where a passphrase is right: a CA root key that is used deliberately and rarely, keys on developer laptops, and keys in transit. In those cases a human is present anyway.

The better answer to the underlying question: if the concern is a stolen key file, the real fixes are KMS/HSM-backed keys, full-disk encryption, and short-lived certificates — not a passphrase.


Part B · Encodings — how the numbers become a file

Part A was about what a key contains. Part B is about how it gets written down.

This sounds like dull plumbing. It is not. Almost every "the certificate doesn't work" ticket you will ever receive is really an encoding problem, and once you can see the layers you can solve those tickets in seconds instead of hours.

There are exactly two layers, and everything else is a variation on them: DER (the binary form) and PEM (the text form wrapped around it).

B1 · ASN.1 and DER — the binary form

The analogy — and this one really does make the whole thing click.

Imagine a paper form at a government office.

ASN.1 is the form design. It says: "Box 1 is a surname. Box 2 is a date. Box 3 is a list of addresses." It describes what fields exist and what type each one is. It says nothing about handwriting.

DER is the filling-in rules. "Block capitals. Black ink. Dates as DD/MM/YYYY. No extra spaces." Follow them and two people entering the same information produce two identical pieces of paper — down to the last mark.

Why does that matter so much? Because in Module 01 you learned that a CA signs the hash of the bytes. If there were two different-but-equally-valid ways to write the same certificate, the two versions would hash differently and the signature would only work on one of them. DER exists precisely to make sure there is exactly one correct set of bytes.

TermWhat it actually is
ASN.1A language for describing data structures. Like a schema. Not a file format
BERA flexible set of rules for writing ASN.1 as bytes. Several encodings can be valid for the same data
DERA strict subset of BER: exactly one valid encoding per value. This is what certificates and keys use

Every DER item is written the same way: Tag, Length, Value. What kind of thing it is, how many bytes long, then the bytes. Structures nest inside each other, so a certificate is a box containing boxes.

🧪 Exercise B1.1 — Turn a key into its raw binary form and look at it
bash
cd ~/tls-lab/m02

# Convert PEM (text) to DER (binary)
openssl pkey -in ec256.key -outform DER -out ec256.der

ls -l ec256.key ec256.der

# Look at the raw bytes
xxd ec256.der | head -4

# Now ask OpenSSL to explain the structure
openssl asn1parse -inform DER -in ec256.der
Expected result — click to reveal
plain text
-rw------- 1 zaeem zaeem 138 Aug 20 12:41 ec256.der
-rw------- 1 zaeem zaeem 241 Aug 20 12:04 ec256.key

00000000: 3081 8702 0100 3013 0607 2a86 48ce 3d02  0.....0...*.H.=.
00000010: 0106 082a 8648 ce3d 0301 0704 6d30 6b02  ...*.H.=....m0k.
00000020: 0101 0420 8f2a 1c74 b3e9 05d1 9c44 f7a1  ... .*.t.....D..
00000030: 2b8e 6033 a015 7d3c e2b8 44f9 a61d 8e04  +.`3..}<..D.....

    0:d=0  hl=3 l= 135 cons: SEQUENCE
    3:d=1  hl=2 l=   1 prim: INTEGER           :00
    6:d=1  hl=2 l=  19 cons: SEQUENCE
    8:d=2  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
   17:d=2  hl=2 l=   8 prim: OBJECT            :prime256v1
   27:d=1  hl=2 l= 109 prim: OCTET STRING      [HEX DUMP]:306B0201010420...

What to read out of this.

  • The DER file is 138 bytes; the PEM file was 241. The text version is about 75% bigger, because Base64 adds roughly a third and then line breaks and headers add more. DER is the compact truth; PEM is a readable wrapper around it.
  • The very first byte is 30. That is the ASN.1 tag for SEQUENCE — "a box containing things". Nearly every certificate and key file in the world starts with 30, and that is a genuinely useful fact: xxd file | head -1 tells you instantly whether you are looking at DER.
  • Read the asn1parse output as a nested list. d=0 is the outer box, d=1 is one level in, d=2 is two levels in. The outer SEQUENCE contains a version number, another SEQUENCE describing the algorithm, and an OCTET STRING holding the actual key material.
  • id-ecPublicKey and prime256v1 are OIDs — Object Identifiers. They are globally unique numbers that name algorithms and fields, so that everybody agrees what "EC public key on curve P-256" means. OpenSSL is printing the friendly name; the file contains the number.

💡 openssl asn1parse is your X-ray machine. When a file will not load and you cannot tell why, running asn1parse on it tells you whether the internal structure is intact or genuinely corrupt.

🧪 Exercise B1.2 — Give a tool the wrong form on purpose
bash
cd ~/tls-lab/m02

# ec256.der is binary. Ask OpenSSL to read it as text.
openssl pkey -in ec256.der -noout -text

echo "---"

# And the reverse: read the text file as binary
openssl pkey -inform DER -in ec256.key -noout -text
Expected result — click to reveal
plain text
Could not read private key from ec256.der
40C7B1E2A87F0000:error:1E08010C:DECODER routines::unsupported:crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input type: PEM

---

Could not read private key from ec256.key
40C7B1E2A87F0000:error:1E08010C:DECODER routines::unsupported:crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input type: DER

What to read out of this.

  • Both failed, and the two errors are nearly identical apart from the last three characters: Input type: PEM versus Input type: DER.
  • Those last three characters are the whole diagnosis. OpenSSL is telling you what form it expected. Input type: PEM means "I was told to read text and found something that is not text". So the fix is -inform DER.
  • The file itself is perfectly fine in both cases. Nothing is corrupt. The tool was simply told the wrong thing.

🔑 Build this reflex now: when a key or certificate "will not load", your first question is not "is it broken?" It is "is it in the form the tool expects?" In practice that solves it more often than anything else, and you can check in one second with head -1 file.

Now imagine this at 500 hosts. A deployment pipeline that assumes PEM will fail on the one team that exported DER from a Windows tool. Making your scripts detect the form (head -c1 | xxd → does it start with 30?) rather than assume it is a five-line change that removes a whole category of ticket.

🎯 Interview questions — DER and ASN.1

Q. What is DER and why do certificates use it rather than something simpler?

DER is a set of strict rules for writing ASN.1 data structures as bytes, where every value has exactly one valid encoding.

That uniqueness is the whole point. A CA signs the hash of the certificate's bytes. If the same certificate could legitimately be written two different ways, it would produce two different hashes, and a signature valid on one would fail on the other. DER removes that ambiguity.

Where this bites in real life: it is also why you must never "tidy up" a certificate file — re-wrapping lines, changing whitespace, or round-tripping it through a tool that re-encodes can change the bytes and invalidate the signature. Certificates are moved, never edited.

Q. How can you tell whether a file is DER or PEM without opening it in an editor?

head -1 file — if you see -----BEGIN SOMETHING-----, it is PEM. If you see binary noise, it is DER.

More precisely: xxd file | head -1. A DER certificate or key almost always starts with byte 30, the ASN.1 tag for SEQUENCE.

file cert.der also works on most systems and will say "data" for DER and "ASCII text" or "PEM certificate" for PEM.

Practical note: never trust the extension. A .crt file is PEM about as often as it is DER, and Windows tools happily produce either.


B2 · Base64 and PEM — the text form

DER is binary. Binary does not survive email, chat, copy-paste, YAML files or web forms — things get mangled, bytes get eaten, line endings get rewritten.

So we translate the binary into plain letters and numbers. That translation is Base64, and the finished, labelled file is PEM.

The analogy. You need to send a small metal object through the post.

Base64 is like spelling something out over a bad phone line using "Alpha, Bravo, Charlie". It takes more words than saying it normally — about a third more — but it arrives intact.

PEM is the envelope you put it in, with the contents written on the outside: -----BEGIN PRIVATE KEY-----. The label tells the person at the other end what is inside and how to unpack it.

And remember Module 01's Exercise A1.1: Base64 is not a lock. The envelope is not sealed. Anyone can open it. It exists to survive the journey, not to keep secrets.

A PEM file is exactly three things:

  1. A -----BEGIN <label>----- line
  2. The DER bytes, Base64-encoded, wrapped at 64 characters per line
  3. A -----END <label>----- line

That is all. There is nothing else in there.

🧪 Exercise B2.1 — Decode a PEM file by hand

You are going to strip the envelope yourself and prove that what falls out is exactly the DER file from B1.1.

bash
cd ~/tls-lab/m02

# Strip the BEGIN/END lines, then Base64-decode what is left
grep -v -- '-----' ec256.key | base64 -d > manual.der

ls -l manual.der ec256.der

# Are they the same file?
cmp manual.der ec256.der && echo "IDENTICAL - PEM really is just Base64 around DER"

sha256sum manual.der ec256.der
Expected result — click to reveal
plain text
-rw------- 1 zaeem zaeem 138 Aug 20 12:55 manual.der
-rw------- 1 zaeem zaeem 138 Aug 20 12:41 ec256.der

IDENTICAL - PEM really is just Base64 around DER

4bfd2c8b6f1eec7a25b1c3a0d9f8e47256b0d3f19c8a4e26f7b53d0a1c96e482  manual.der
4bfd2c8b6f1eec7a25b1c3a0d9f8e47256b0d3f19c8a4e26f7b53d0a1c96e482  ec256.der

What to read out of this.

  • You just did with grep and base64 what OpenSSL does internally. PEM genuinely is nothing more than DER wrapped in Base64 with a label. There is no magic and no extra transformation.
  • The two files are byte-identical, confirmed by both cmp and matching SHA-256 hashes.
  • This means you can read a certificate anywhere. Stuck on a machine with no OpenSSL? grep -v -- '-----' cert.pem | base64 -d | xxd still shows you the bytes.

💡 A safety note. This trick is fine on public things like certificates. Be careful doing it with private keys on a shared machine — the decoded key lands on disk, and if your umask is not set it may land world-readable. That is why every exercise in this track starts with umask 077.

🧪 Exercise B2.2 — Break a PEM file the way it breaks in real life

Three of the most common real-world PEM failures, produced deliberately. Read each error.

bash
cd ~/tls-lab/m02
cp ec256.key broken1.key && cp ec256.key broken2.key && cp ec256.key broken3.key

# 1. A dash lost during a copy-paste
sed -i '1s/-----BEGIN/----BEGIN/' broken1.key
openssl pkey -in broken1.key -noout 2>&1 | head -2

echo "---"

# 2. Windows line endings, from pasting into Notepad
sed -i 's/$/\r/' broken2.key
file broken2.key
openssl pkey -in broken2.key -noout 2>&1 | head -2

echo "---"

# 3. A stray space at the start of a Base64 line
sed -i '2s/^/ /' broken3.key
openssl pkey -in broken3.key -noout 2>&1 | head -2
Expected result — click to reveal
plain text
# 1 - missing dash
Could not read private key from broken1.key
40C7B1E2A87F0000:error:1E08010C:DECODER routines::unsupported:crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input type: PEM

---
# 2 - Windows line endings
broken2.key: ASCII text, with CRLF line terminators
(no error - it loaded fine)

---
# 3 - stray leading space
Could not read private key from broken3.key
40C7B1E2A87F0000:error:1E08010C:DECODER routines::unsupported:crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input type: PEM

What to read out of this — and case 2 is the interesting one.

  • Cases 1 and 3 fail with the identical message. A missing dash and a stray space produce exactly the same error, and the error names neither problem. This is why PEM troubleshooting feels like guesswork until you know what to look for.
  • Case 2 did NOT fail. OpenSSL tolerates Windows CRLF line endings. But many other tools do not — Java, some load balancers, several cloud certificate importers, and older versions of nginx will reject the very same file that OpenSSL accepted happily.

🔑 That is the lesson worth carrying: "OpenSSL can read it" does not mean "everything can read it". When a certificate works locally but is rejected on upload to a cloud provider, CRLF is one of the first things to check, and the fix is one command:

bash
sed -i 's/\r$//' cert.pem      # or:  dos2unix cert.pem

A quick way to spot all three problems at once:

bash
cat -A broken3.key | head -3
#  ----BEGIN PRIVATE KEY-----$        <- dashes visible, $ marks the true line end
#  ^I or ^M would show tabs / carriage returns

cat -A makes invisible characters visible. It is the single most useful command for a PEM file that "looks fine".

Now imagine this at 500 hosts. Someone pastes a certificate into a ticket, a colleague copies it out of the browser, and the trailing newline is lost. It deploys to 12 servers before anyone notices that three of them silently kept the old certificate because the new file would not parse. Always validate a PEM after writing it: openssl x509 -in new.crt -noout -subject before reloading anything.

🎯 Interview questions — PEM and Base64

Q. What is the difference between PEM and DER?

DER is the raw binary encoding. PEM is that same binary, Base64-encoded and wrapped in -----BEGIN X----- / -----END X----- lines so it can be safely copied, pasted, emailed and stored in text files.

They hold identical information — you can convert back and forth with no loss. PEM is roughly a third larger because of Base64, plus the headers and line breaks.

Practical framing: PEM is what you see almost everywhere on Linux; DER turns up in Windows and Java tooling and in some hardware appliances. Since neither the extension nor the tool's expectation is reliable, the useful habit is to check with head -1 before assuming.

Q. A certificate works on your Linux box but a cloud console rejects it on upload. Where do you look?

A short checklist, roughly in order of how often each is the cause:

  1. Line endings. CRLF from a Windows editor. OpenSSL tolerates it; many importers do not. dos2unix or sed -i 's/\r$//'.
  2. Missing or extra content. A lost trailing newline, a stray space, or a lost dash from copy-paste. cat -A shows all of it.
  3. Wrong object in the file. Uploading the CSR instead of the certificate, or the private key instead of the public certificate. head -1 tells you immediately.
  4. Chain expectations. Many importers want the leaf certificate and the intermediates in separate fields, or in a specific order. Sending a combined bundle where a single certificate is expected fails.
  5. Wrong encoding. The console wants PEM and you exported DER.

What this really demonstrates in an interview is that you treat "it works here" as evidence about your tool, not about the file.


B3 · The label on the envelope matters

You have already seen two different labels: BEGIN PRIVATE KEY and BEGIN ENCRYPTED PRIVATE KEY. There are more, and they are not decoration — the label tells you how the bytes inside are arranged.

The analogy. Two parcels arrive. Both contain a kettle.

Parcel A says: "Kettle. Voltage 240V. Instructions inside." You can unpack it without knowing anything in advance.

Parcel B says just: "Contents: appliance." You have to already know what kind of appliance it is, or you cannot do anything with it.

BEGIN PRIVATE KEY is parcel A. It is self-describing — the bytes inside start by naming the algorithm, so any tool can work it out. This is PKCS#8.

BEGIN RSA PRIVATE KEY is parcel B. The type is written only on the outside label, and the bytes inside just assume you already know it is RSA. This is PKCS#1, sometimes called "traditional" format.

PEM labelStandardWhat it means
-----BEGIN PRIVATE KEY-----PKCS#8The modern default. Works for RSA, EC, Ed25519 — the algorithm is named inside
-----BEGIN ENCRYPTED PRIVATE KEY-----PKCS#8Same, but passphrase-protected (Exercise A3.1)
-----BEGIN RSA PRIVATE KEY-----PKCS#1Older "traditional" RSA-only format. Still very common in the wild
-----BEGIN EC PRIVATE KEY-----SEC1Older EC-only format, the EC equivalent of the row above
-----BEGIN PUBLIC KEY-----SPKIA public key of any type. This is what -pubout gives you
-----BEGIN CERTIFICATE-----X.509A certificate. Not a key — the commonest mix-up of all
-----BEGIN CERTIFICATE REQUEST-----PKCS#10A CSR. Module 04's subject
Where this actually hurts you.

Java historically accepts only PKCS#8 (BEGIN PRIVATE KEY). Hand it a BEGIN RSA PRIVATE KEY file and it refuses, often with an error that says nothing useful about format.

Meanwhile plenty of older tooling, and a lot of documentation written before 2015, produces PKCS#1.

So "the key doesn't work in the Java app but works fine in nginx" is a format problem, not a key problem — and it is a one-command fix.

🧪 Exercise B3.1 — Convert between the two formats and watch the label change
bash
cd ~/tls-lab/m02

# What have we got right now?
head -1 rsa2048.key

# Convert PKCS#8 -> PKCS#1 (traditional)
openssl rsa -in rsa2048.key -traditional -out rsa-pkcs1.key
head -1 rsa-pkcs1.key

# Convert back: PKCS#1 -> PKCS#8
openssl pkcs8 -topk8 -nocrypt -in rsa-pkcs1.key -out rsa-pkcs8.key
head -1 rsa-pkcs8.key

# Is it still the same key underneath?
openssl pkey -in rsa2048.key   -pubout | sha256sum
openssl pkey -in rsa-pkcs1.key -pubout | sha256sum
openssl pkey -in rsa-pkcs8.key -pubout | sha256sum
Expected result — click to reveal
plain text
-----BEGIN PRIVATE KEY-----
-----BEGIN RSA PRIVATE KEY-----
-----BEGIN PRIVATE KEY-----

8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -
8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -
8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -

What to read out of this.

  • All three hashes are identical. The key never changed. Only the packaging did. Converting between PKCS#1 and PKCS#8 is genuinely lossless and completely safe — it is repackaging, not re-generating.
  • -nocrypt matters. Without it, openssl pkcs8 -topk8 will prompt you for a passphrase and produce an ENCRYPTED PRIVATE KEY. If a conversion suddenly starts asking for a password you did not expect, that flag is what you forgot.
  • -traditional is the flag that goes the other way. On OpenSSL 3.x, openssl rsa outputs PKCS#8 by default; -traditional gives you the older PKCS#1 form. On OpenSSL 1.x the default was the opposite, which is why old blog posts and your machine sometimes disagree.

💡 The two commands to remember. They cover almost every private key format problem you will ever have:

bash
openssl pkcs8 -topk8 -nocrypt -in any.key -out pkcs8.key    # -> BEGIN PRIVATE KEY
openssl rsa -in any.key -traditional -out pkcs1.key         # -> BEGIN RSA PRIVATE KEY
🧪 Exercise B3.2 — Prove the contents differ even though the key is the same
bash
cd ~/tls-lab/m02

# Same key, two packagings - are the files the same?
cmp rsa2048.key rsa-pkcs1.key && echo same || echo "DIFFERENT FILES"

# Look at the structure of each
openssl asn1parse -in rsa2048.key   | head -5
echo "---"
openssl asn1parse -in rsa-pkcs1.key | head -5
Expected result — click to reveal
plain text
DIFFERENT FILES

    0:d=0  hl=4 l=1214 cons: SEQUENCE
    4:d=1  hl=2 l=   1 prim: INTEGER           :00
    7:d=1  hl=2 l=  13 cons: SEQUENCE
    9:d=2  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   20:d=2  hl=2 l=   0 prim: NULL
---
    0:d=0  hl=4 l=1187 cons: SEQUENCE
    4:d=1  hl=2 l=   1 prim: INTEGER           :00
    7:d=1  hl=4 l= 257 prim: INTEGER           :BB5494D4B7D52CF1C2A385F3BCE4DF...
   20:d=2  hl=2 l=   0 prim: NULL

What to read out of this — this is the parcel analogy, made visible.

  • The PKCS#8 version has OBJECT :rsaEncryption near the top. That is the algorithm, written inside the file. Parcel A: self-describing. Any tool can read the first few bytes and know what it is holding.
  • The PKCS#1 version goes straight to INTEGER :BB5494D4... — that is the modulus, with no algorithm named anywhere. Parcel B: the tool must already know it is RSA, because the file never says so.
  • The PKCS#8 file is 27 bytes bigger. That is the cost of the label. Cheap, and worth it.

🔑 So the rule you can now derive rather than memorise: always prefer PKCS#8 (BEGIN PRIVATE KEY) when you have a choice, because it is self-describing and every modern tool accepts it. Convert to PKCS#1 only when something old specifically demands it.

🎯 Interview questions — Key formats

Q. What is the difference between BEGIN PRIVATE KEY and BEGIN RSA PRIVATE KEY?

BEGIN PRIVATE KEY is PKCS#8 — a generic wrapper that names the algorithm inside the file itself, so it works for RSA, EC, Ed25519 and anything else.

BEGIN RSA PRIVATE KEY is PKCS#1, the older RSA-only "traditional" format, where the algorithm is implied by the label rather than recorded in the data.

They can hold exactly the same key. Converting between them is lossless: openssl pkcs8 -topk8 -nocrypt one way, openssl rsa -traditional the other.

The reason to care: Java and several other stacks accept only PKCS#8, so a key that works perfectly in nginx can be rejected by a Java service. Recognising that as a packaging problem rather than a key problem turns a long debugging session into a one-line fix.

Q. You are handed a .pem file and told "it's the certificate". How do you check?

head -1 first — the label tells you what it really is: CERTIFICATE, PRIVATE KEY, RSA PRIVATE KEY, CERTIFICATE REQUEST or PUBLIC KEY.

Then confirm it parses as what it claims: openssl x509 -in file.pem -noout -subject -dates for a certificate, openssl pkey -in file.pem -noout -text for a key, openssl req -in file.pem -noout -subject for a CSR.

Also worth checking: grep -c BEGIN file.pem. More than one means it is a bundle — several objects concatenated in one file — and some tools want only a single object.

The habit being tested: never trust the filename, the extension, or what the person handing it to you believes it is. Thirty seconds of checking saves an outage.


Part C · Extensions and container files

C1 · The file extension is a hint, not a fact

The analogy. Someone hands you a file called holiday-photo.doc.

Does that make it a Word document? No. It makes it a file that somebody named holiday-photo.doc. The name is a note written on the outside by a human. The contents are whatever they are.

Certificate file extensions work exactly like this. There is no rule, no standard and no enforcement. People choose whatever their tool defaulted to.

ExtensionWhat people usually meanWhat it might actually be
.pemText format, Base64 with BEGIN/END linesA certificate, a key, a CSR, a chain, or all of them at once
.crt .cerA certificatePEM or DER. .cer from Windows is very often DER
.keyA private keyPEM or DER, PKCS#8 or PKCS#1, encrypted or not
.derBinary formUsually right, but says nothing about what is inside
.csr .reqA certificate signing requestPEM or DER (Module 04)
.p12 .pfxKey + certificate + chain, in one password-protected fileReliably PKCS#12 — one of the few honest extensions
.jksA Java keystoreOld proprietary Java format. Modern Java uses PKCS#12 instead
.p7b .p7cA certificate chain, no private keyPKCS#7. Common from Windows CAs

So instead of reading the name, you ask the file. This is the decision tree to follow, and it takes about three seconds:

Diagram source
flowchart TD
    A["📄 A file somebody<br>handed you"] --> B{"file says<br>text or data?"}
    B -->|"ASCII text"| C{"head -1 shows<br>which BEGIN label?"}
    B -->|"data"| D{"xxd - what is<br>the first byte?"}
    C -->|"CERTIFICATE"| E["📜 A certificate"]
    C -->|"PRIVATE KEY"| F["🔑 A key<br>PKCS#8 format"]
    C -->|"RSA PRIVATE KEY"| G["🔑 A key<br>PKCS#1 format"]
    C -->|"ENCRYPTED PRIVATE KEY"| H["🔒 A key with<br>a passphrase"]
    C -->|"CERTIFICATE REQUEST"| I["📝 A CSR<br>Module 04"]
    C -->|"no BEGIN line at all"| J["❌ Damaged PEM<br>check cat -A for CRLF<br>or a lost dash"]
    D -->|"30"| K["📦 DER<br>retry with -inform DER"]
    D -->|"anything else"| L["💼 Probably PKCS#12<br>try openssl pkcs12 -info"]
    style E fill:#d5e8d4,stroke:#82b366,stroke-width:2px
    style F fill:#ffe6cc,stroke:#d79b00,stroke-width:2px
    style G fill:#ffe6cc,stroke:#d79b00,stroke-width:2px
    style H fill:#ffe6cc,stroke:#d79b00,stroke-width:2px
    style J fill:#ffcccc,stroke:#cc0000,stroke-width:2px
🧪 Exercise C1.1 — Identify a mystery file without trusting its name
bash
cd ~/tls-lab/m02

# Make three deliberately misleading files
cp rsa2048.key    mystery1.crt      # a key, named as a certificate
cp ec256.der      mystery2.pem      # binary, named as text
cp ../example-com.pem mystery3.key  # a certificate, named as a key

# Now identify each one WITHOUT looking at the name
for f in mystery1.crt mystery2.pem mystery3.key; do
  echo "=== $f ==="
  file "$f"
  head -c 30 "$f" | xxd | head -1
done
Expected result — click to reveal
plain text
=== mystery1.crt ===
mystery1.crt: ASCII text
00000000: 2d2d 2d2d 2d42 4547 494e 2050 5249 5641  -----BEGIN PRIVA
00000010: 5445 204b 4559 2d2d 2d2d 2d0a 4d49 4945  TE KEY-----.MIIE

=== mystery2.pem ===
mystery2.pem: data
00000000: 3081 8702 0100 3013 0607 2a86 48ce 3d02  0.....0...*.H.=.

=== mystery3.key ===
mystery3.key: PEM certificate
00000000: 2d2d 2d2d 2d42 4547 494e 2043 4552 5449  -----BEGIN CERTI
00000010: 5449 4649 4341 5445 2d2d 2d2d 2d0a 4d49  FICATE-----.MI

What to read out of this.

  • mystery1.crt is a private key. Named .crt, contains BEGIN PRIVATE KEY. If you had uploaded that to a load balancer expecting a certificate, you would have leaked your private key into a system that logs uploads.
  • mystery2.pem is binary DER. file says just data, and the bytes start with 30 — the SEQUENCE tag from Exercise B1.1. Named .pem, and it is not PEM at all.
  • mystery3.key is a certificate, and file even identifies it correctly as "PEM certificate".

🔑 The 3-second identification routine, worth making automatic:

bash
file  suspicious-file        # text or binary?
head -1 suspicious-file      # if text: what does the label say?
xxd suspicious-file | head -1  # if binary: does it start with 30? then DER

⚠️ The real danger here is not confusion, it is leakage. Mixing up a .key and a .crt in one direction is a broken deploy. In the other direction it is a private key disclosure. Always run head -1 before you upload, paste or attach anything.

Clean up the misleading files so you do not confuse yourself later:

bash
rm -f mystery1.crt mystery2.pem mystery3.key

🎯 Interview questions — File formats and extensions

Q. What is the difference between .pem, .crt, .cer, .key and .der?

Only .der and .p12/.pfx carry reliable meaning. The rest are conventions, not standards, and are frequently wrong.

The two things that are real:

  • Encoding: PEM (Base64 text with BEGIN/END lines) versus DER (raw binary). Determine it with head -1 or file.
  • Content: certificate, private key, CSR, or a bundle of several. The PEM label states it directly; for DER you have to try parsing it.

What this question is really testing is whether you verify or assume. The answer that lands: "I never trust the extension. file plus head -1 takes three seconds and has stopped me pasting a private key into a ticket more than once."


C2 · PKCS#12 — everything in one locked file

The analogy. You are moving to a new office and need to bring your ID badge and your office key with you.

Carrying them loose means one of them gets lost. So you put both in a small locked briefcase with a combination on it.

That briefcase is a PKCS#12 file — usually named .p12, or .pfx if it came from Microsoft. It holds a private key, its certificate, and usually the chain above it, all in one file, all protected by one password.

It exists for one reason: moving things between systems. Windows, Java, macOS Keychain and most load-balancer consoles all speak PKCS#12, so it is the common language for "here is my key and certificate".

A .p12 usually contains a private key. Treat it exactly like a private key.

It has a password, which makes people relaxed about emailing it. Do not be. The password is often weak, often sent in the same thread, and the file is offline-crackable. A leaked .p12 with a guessable password is a leaked private key.

🧪 Exercise C2.1 — Build a PKCS#12 file and look inside it

You do not have a certificate for your own key yet — that is Module 04. So you will build the other common shape of .p12: a truststore, holding certificates and no private key.

bash
cd ~/tls-lab/m02

# Grab a real certificate chain to work with
openssl s_client -connect example.com:443 -showcerts </dev/null 2>/dev/null \
  | sed -n '/BEGIN CERT/,/END CERT/p' > chain.pem

grep -c 'BEGIN CERTIFICATE' chain.pem

# Pack them into a PKCS#12 truststore
openssl pkcs12 -export -nokeys -in chain.pem -out truststore.p12 -passout pass:changeit

ls -l truststore.p12
file truststore.p12

# What is inside?
openssl pkcs12 -info -in truststore.p12 -passin pass:changeit -nokeys -noout
Expected result — click to reveal
plain text
2

-rw------- 1 zaeem zaeem 2418 Aug 20 13:22 truststore.p12
truststore.p12: data

MAC: sha256, Iteration 2048
MAC length: 32, salt length: 8
PKCS7 Encrypted data: PBES2, PBKDF2, AES-256-CBC, Iteration 2048, PRF hmacWithSHA256
Certificate bag
Bag Attributes: <No Attributes>
subject=CN=*.example.com
issuer=C=US, O=DigiCert Inc, CN=DigiCert Global G3 TLS ECC SHA384 2020 CA1
Certificate bag
Bag Attributes: <No Attributes>
subject=C=US, O=DigiCert Inc, CN=DigiCert Global G3 TLS ECC SHA384 2020 CA1
issuer=C=US, O=DigiCert Inc, OU=DigiCert Global Root G3

What to read out of this.

  • file says just data. PKCS#12 is binary, like DER. You cannot open it in an editor and see anything useful.
  • "Certificate bag" appears twice. PKCS#12 organises its contents into bags — a bag for certificates, a bag for keys. Both certificates from the chain went in, and each one is listed with its subject and issuer.
  • MAC: sha256 and PBKDF2, AES-256-CBC, Iteration 2048. The file really is encrypted, with the password stretched through PBKDF2 — the same idea you met in Module 01's Exercise B1.1. Note the iteration count of 2048, which is low by modern standards. That is the default, and it is why a weak .p12 password is genuinely crackable offline.
  • The MAC is what detects a wrong password, which is the next exercise.

⚠️ Your issuer names, serials and dates will not match mine. example.com's certificate is renewed every few months and the CA rotates its intermediates, so the exact strings above are a snapshot, not a constant. What you should check is the shape: two certificate bags, and the first one's issuer= line equal to the second one's subject= line. Every expected output in this track that involves a live third-party site works the same way — match the structure, not the text.

💡 The other shape of the command, for when you do have a matching key and certificate — you will run this for real in Module 04:

bash
openssl pkcs12 -export \
  -inkey server.key \          # the private key
  -in server.crt \             # its certificate
  -certfile chain.pem \        # the intermediates above it
  -name "my-server" \          # a friendly label inside the file
  -out bundle.p12
🧪 Exercise C2.2 — Get the password wrong, and then unpack it properly
bash
cd ~/tls-lab/m02

# Wrong password - meant to fail
openssl pkcs12 -info -in truststore.p12 -passin pass:definitely-wrong -noout

echo "--- now the right one ---"

# Unpack the certificates back out to PEM
openssl pkcs12 -in truststore.p12 -passin pass:changeit -nokeys -out unpacked.pem
grep -c 'BEGIN CERTIFICATE' unpacked.pem
Expected result — click to reveal
plain text
Mac verify error: invalid password?
40A7C1B2E87F0000:error:11800071:PKCS12 routines:PKCS12_item_decrypt_d2i_ex:mac verify failure:crypto/pkcs12/p12_decr.c:180:

--- now the right one ---
2

What to read out of this.

  • Mac verify error: invalid password? — and notice this is a much better error than the one you got in Module 01's Exercise B1.2, where a wrong password produced bad decrypt from a padding routine.
  • The reason is the MAC. PKCS#12 stores a keyed checksum over the contents. If the password is wrong, that checksum does not match, and the file can say so confidently and immediately. It does not have to decrypt garbage and hope the padding looks wrong.
  • This is authenticated encryption in action — the same principle behind the AEAD cipher suites in TLS 1.3 that you saw in Exercise A2.2 of Module 01. Wrong key fails loudly, not silently.
  • Unpacking gave back 2 certificates, exactly what went in. The container is lossless.

⚠️ A very common real-world stumble: when a .p12 does contain a private key, openssl pkcs12 -in bundle.p12 -out out.pem will write the key into out.pem encrypted with a new passphrase it prompts you for. If you want it unencrypted you must add -noenc (called -nodes before OpenSSL 3.0). Forgetting that flag is why so many people end up with a key their web server cannot read — which is Exercise A3.2 all over again.

🎯 Interview questions — PKCS#12

Q. What is a .p12 / .pfx file and when would you use one?

A PKCS#12 file: a single password-protected binary container holding a private key, its certificate, and usually the intermediate chain. .pfx is Microsoft's name for the same thing.

You use it whenever key material has to move between systems — importing into Windows or IIS, into a Java keystore, into macOS Keychain, or uploading to a load balancer or cloud console. Most of those tools do not accept loose PEM files.

The caution worth voicing: because it has a password, people treat a .p12 as safe to email. It is not. The default PBKDF2 iteration count is low, the password is usually weak, and the file can be attacked offline at leisure. It should be handled with the same care as a bare private key, and ideally moved through a secret manager rather than a chat window.

Q. Why does PKCS#12 give a clear "invalid password" error when openssl enc does not?

Because PKCS#12 includes a MAC — a keyed integrity check over the contents. A wrong password produces a MAC mismatch, which is detectable directly and unambiguously.

Plain openssl enc with a CBC cipher has no such check. It decrypts with whatever key it was given, produces garbage, and only notices because the garbage does not end in valid padding — hence the unhelpful bad decrypt message.

The general principle to state: encryption without authentication cannot tell you whether decryption succeeded. That is exactly why every TLS 1.3 cipher suite is AEAD, and why modern designs never use encryption alone.


C3 · Java keystores — the same idea with different words

If you work anywhere near Java applications you will meet keystores, and the vocabulary is different enough to be confusing even though the ideas are identical.

The analogy. Everyone else in the office uses the standard locked briefcase. Java turned up with its own briefcase design, called JKS, and used it for twenty years.

Since Java 9 it uses the standard one — PKCS#12 — by default. But there are still plenty of old JKS briefcases lying around in production, and plenty of documentation that assumes them.

Java wordWhat it means in plain terms
KeystoreA file holding your own private keys and certificates. "Here is who I am"
TruststoreA file holding other people's certificates that you have decided to trust. "Here is who I believe"
AliasA name label for one entry inside the store, so you can refer to it
JKSJava's old proprietary container format
PKCS12The standard container — the default since Java 9, and what you should use
cacertsThe truststore that ships with the JDK, holding the public root CAs. Java's own version of /etc/ssl/certs
Keystore versus truststore is the distinction that actually matters, and it is where most Java TLS confusion comes from.

A keystore answers "who am I?" — it holds your private key.

A truststore answers "who do I believe?" — it holds certificates you trust, and no private keys.

This maps directly onto Module 01: your keystore holds the key from Exercise B2.1; your truststore is the trust anchor store from Exercise C3.1. When a Java app says PKIX path building failed, it is a truststore problem — the app cannot find a trusted issuer. When it says it cannot find a key or alias, it is a keystore problem.

🧪 Exercise C3.1 — Read a PKCS#12 file with Java's own tool

Skip this if you have no JDK installed; the concept still stands.

bash
cd ~/tls-lab/m02

# Is keytool available?
which keytool || echo "no JDK installed - read the expected output and move on"

# Java reads the PKCS#12 file you built in C2.1 directly
keytool -list -keystore truststore.p12 -storetype PKCS12 -storepass changeit

# And Java's own bundled trust store
keytool -list -cacerts -storepass changeit | head -6
Expected result — click to reveal
plain text
Keystore type: PKCS12
Keystore provider: SUN

Your keystore contains 2 entries

1, 20 Aug 2026, trustedCertEntry,
Certificate fingerprint (SHA-256): A4:1F:9C:82:7B:03:E5:6D:41:...
2, 20 Aug 2026, trustedCertEntry,
Certificate fingerprint (SHA-256): 7E:B2:04:D9:15:8A:6C:33:F0:...

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 148 entries

amazonrootca1 [jdk], 12 Nov 2025, trustedCertEntry,

What to read out of this.

  • Java read the OpenSSL-created file without any conversion. That is the whole point of PKCS#12 being the default now — it is a genuinely shared format, and the days of converting everything to JKS are over.
  • trustedCertEntry means "a certificate I trust", with no private key. A key-bearing entry would show as PrivateKeyEntry. That one word tells you whether you are looking at a keystore or a truststore.
  • cacerts holds 148 entries. That is Java's own list of root CAs, completely separate from the 146 you counted in /etc/ssl/certs in Module 01's Exercise C3.1. Two different lists on the same machine.

🔑 This is the single most useful Java-TLS fact you can carry into an interview. Adding an internal root CA to the operating system with update-ca-certificates does nothing for a Java application. Java reads cacerts, not /etc/ssl/certs. The symptom is a PKIX path building failed error on a host where curl to the very same URL works perfectly.

bash
keytool -importcert -cacerts -storepass changeit -alias my-internal-root -file myroot.crt

Now imagine this at 500 hosts. Every JVM has its own cacerts, and a JDK upgrade replaces the file — silently discarding any root you added by hand. Teams that add roots manually rediscover this every upgrade cycle. The durable fix is to bake the trust store into the container image or manage it with configuration management, never by hand on a live host.

🎯 Interview questions — Java keystores

Q. What is the difference between a keystore and a truststore?

A keystore holds your own identity — private keys with their certificates. A truststore holds certificates belonging to others that you have chosen to trust, and contains no private keys.

They are usually the same file format (PKCS#12 today, JKS historically), which is why people conflate them. The difference is purpose, and it maps to the two sides of TLS: a server presents from its keystore, and validates the other side against its truststore.

The diagnostic value: PKIX path building failed is always a truststore problem — the JVM cannot chain to a trusted root. "Alias does not exist" or "no key entry" is a keystore problem.

Q. A Java app cannot reach an internal HTTPS service, but curl on the same host works fine. Why?

Because the JVM has its own trust store — $JAVA_HOME/lib/security/cacerts — completely separate from the operating system's. The internal root CA was added to the OS store with update-ca-certificates or update-ca-trust, which the JVM never reads.

Fix: keytool -importcert -cacerts -alias internal-root -file root.crt, or point the app at a custom truststore with -Djavax.net.ssl.trustStore.

The detail that shows operational experience: a JDK upgrade replaces cacerts and silently drops anything added by hand. So the durable fix is to bake it into the image or manage it with config management. And the same class of problem exists for Node.js, Python's certifi, and Firefox — every one keeps its own list.

Q. Should you still use JKS?

No. PKCS#12 has been the default since Java 9, it is an open standard that OpenSSL and every other tool can read, and JKS is Oracle-proprietary with weaker protection of its contents.

Converting is one command: keytool -importkeystore -srckeystore old.jks -destkeystore new.p12 -deststoretype PKCS12.

Worth mentioning: you will still meet JKS in older systems, and keytool still reads it — it just warns you that the format is proprietary and suggests migrating.


Part D · Working with these files day to day

D1 · The conversion table

Every conversion you are likely to need, in one place. The pattern is nearly always the same: -inform says what is going in, -outform says what comes out.

ConversionCommand
Certificate PEM → DERopenssl x509 -in c.pem -outform DER -out c.der
Certificate DER → PEMopenssl x509 -inform DER -in c.der -out c.pem
Key PEM → DERopenssl pkey -in k.pem -outform DER -out k.der
Key DER → PEMopenssl pkey -inform DER -in k.der -out k.pem
Key PKCS#1 → PKCS#8openssl pkcs8 -topk8 -nocrypt -in k.pem -out k8.pem
Key PKCS#8 → PKCS#1openssl rsa -in k8.pem -traditional -out k1.pem
Add a passphrase to a keyopenssl pkey -in k.pem -aes256 -out k-enc.pem
Remove a passphraseopenssl pkey -in k-enc.pem -out k.pem
PEM key + cert → PKCS#12openssl pkcs12 -export -inkey k.pem -in c.pem -certfile chain.pem -out b.p12
PKCS#12 → PEM (everything)openssl pkcs12 -in b.p12 -noenc -out all.pem
PKCS#12 → key onlyopenssl pkcs12 -in b.p12 -nocerts -noenc -out k.pem
PKCS#12 → certificate onlyopenssl pkcs12 -in b.p12 -clcerts -nokeys -out c.pem
PKCS#7 (.p7b) → PEMopenssl pkcs7 -inform DER -in c.p7b -print_certs -out chain.pem
JKS → PKCS#12keytool -importkeystore -srckeystore a.jks -destkeystore a.p12 -deststoretype PKCS12
Two flags that cause most conversion accidents:

-noenc (called -nodes before OpenSSL 3.0) — without it, extracting a key from a .p12 writes it back out encrypted, and your web server then cannot start. Exercise A3.2, in a new costume.

-nocrypt on openssl pkcs8 -topk8 — same problem, different command. Without it you get an ENCRYPTED PRIVATE KEY you did not ask for.

If a conversion unexpectedly prompts you for a password, one of these two is what you forgot.

The same information as a picture — every arrow is reversible, and none of them changes the key:

Diagram source
flowchart LR
    P["✉️ PEM<br>text with BEGIN/END"]
    D["📦 DER<br>raw binary"]
    C["💼 PKCS#12<br>.p12 / .pfx"]
    J["🗄️ JKS<br>old Java format"]
    P -->|"-outform DER"| D
    D -->|"-inform DER"| P
    P -->|"pkcs12 -export"| C
    C -->|"pkcs12 -in ... -noenc"| P
    J -->|"keytool -importkeystore"| C
    P --> L{"Wrong key label<br>for your tool?"}
    L -->|"need BEGIN PRIVATE KEY"| L1["pkcs8 -topk8 -nocrypt"]
    L -->|"need BEGIN RSA PRIVATE KEY"| L2["rsa -traditional"]
    style P fill:#d5e8d4,stroke:#82b366,stroke-width:2px
    style D fill:#e1d5e7,stroke:#9673a6,stroke-width:2px
    style C fill:#ffe6cc,stroke:#d79b00,stroke-width:2px
🧪 Exercise D1.1 — Round-trip a key through every format and prove nothing was lost
bash
cd ~/tls-lab/m02

# PEM -> DER -> PKCS#1 -> PKCS#8 -> back to where we started
openssl pkey  -in rsa2048.key -outform DER -out r1.der
openssl pkey  -inform DER -in r1.der -out r2.pem
openssl rsa   -in r2.pem -traditional -out r3-pkcs1.pem
openssl pkcs8 -topk8 -nocrypt -in r3-pkcs1.pem -out r4-pkcs8.pem

# Compare the public key derived from each stage
for f in rsa2048.key r2.pem r3-pkcs1.pem r4-pkcs8.pem; do
  printf '%-16s ' "$f"
  openssl pkey -in "$f" -pubout 2>/dev/null | sha256sum | cut -c1-16
done
Expected result — click to reveal
plain text
rsa2048.key      8c4a1f92e7b530d6
r2.pem           8c4a1f92e7b530d6
r3-pkcs1.pem     8c4a1f92e7b530d6
r4-pkcs8.pem     8c4a1f92e7b530d6

What to read out of this.

  • Four different files, four different encodings, one identical key. Format conversion never changes the key. It only changes how the key is written down.
  • This is why conversion is safe and why you should never feel nervous doing it. You are not regenerating anything, and you cannot accidentally weaken a key by converting it.
  • The check itself is the technique to remember. Deriving the public key and hashing it gives you a short, comparable fingerprint of any key file, whatever format it is in. That is exactly the tool you need for the next section.

D2 · Does this key actually match this certificate?

This is the most immediately useful skill in the module. When a web server refuses to start after a certificate change, this check finds the cause in about ten seconds.

The analogy. You have a padlock and a key in front of you. Both look perfectly fine. But are they a pair?

You could try the key in the lock. Or — much faster — you could read the serial number stamped on each one and see whether they match.

That serial number is the public key. It is stamped inside the certificate, and it can be derived from the private key. Compare the two and you have your answer without touching a web server.

The technique in one line: derive the public key from both sides, hash each, compare.

🧪 Exercise D2.1 — Prove a key matches itself, then catch a mismatch
bash
cd ~/tls-lab/m02

# --- The MATCH case: the same key, from two different files ---
openssl pkey -in rsa2048.key   -pubout | sha256sum
openssl pkey -in r4-pkcs8.pem  -pubout | sha256sum

echo "=== now a real certificate against our key ==="

# --- The MISMATCH case: a real certificate, and a key that is not its partner ---
openssl x509 -in ../example-com.pem -noout -pubkey | sha256sum
openssl pkey -in rsa2048.key -pubout | sha256sum
Expected result — click to reveal
plain text
8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -
8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -

=== now a real certificate against our key ===
b81e4f37d0a95c26e73f18b4a0d92c5e6f31870b4ac2d95e18f3b06d7c4a2e91  -
8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -

What to read out of this.

  • First pair: identical. Same key, two file formats, same fingerprint. This is what a matching key and certificate look like.
  • Second pair: completely different. Of course — example.com's certificate contains DigiCert-issued key material belonging to someone else entirely, and our rsa2048.key is a key we made this morning. They are not a pair, and the check says so instantly.
  • Notice the commands are almost the same. openssl x509 -pubkey for a certificate, openssl pkey -pubout for a key. Both print a BEGIN PUBLIC KEY block, so both can be hashed and compared directly.

🔑 Learn this as a one-liner. You will use it more than almost anything else in this track:

bash
diff <(openssl x509 -in server.crt -noout -pubkey) \
     <(openssl pkey -in server.key -pubout) \
  && echo "MATCH" || echo "MISMATCH"

💡 You will also see the older RSA-only version of this check in a lot of documentation. It works, but only for RSA keys — the -pubkey version above works for EC and everything else too:

bash
openssl x509 -noout -modulus -in server.crt | openssl md5
openssl rsa  -noout -modulus -in server.key | openssl md5

You will run the full matching version for real in Module 04, once you have created a certificate for a key you own.

🧪 Exercise D2.2 — Recognise what a mismatch looks like from the server's side

You do not need to run a web server to learn these messages. Read them and file them away — each one means "the key and the certificate are not a pair".

The errors a mismatch actually produces — click to reveal

nginx:

plain text
nginx: [emerg] SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl/server.key") failed
(SSL: error:05800074:x509 certificate routines::key values mismatch)

Apache httpd:

plain text
AH02565: Certificate and private key server.example.com:443:0 from
/etc/ssl/server.crt and /etc/ssl/server.key do not match

HAProxy:

plain text
[ALERT] unable to load SSL private key from PEM file '/etc/haproxy/server.pem'.

Java:

plain text
java.security.UnrecoverableKeyException: Cannot recover key

What to read out of this.

  • nginx and Apache tell you plainly. key values mismatch and do not match are unusually clear messages by TLS standards — take the gift.
  • HAProxy and Java do not. Their messages point at the file, not at the mismatch, and will send you hunting for a corrupt file or a permissions problem that does not exist.
  • Whatever the wording, run the diff one-liner from D2.1 first. It takes ten seconds and rules the cause in or out before you touch anything else.

How does a mismatch happen in practice? Almost always one of these:

  1. A new certificate was issued from a new CSR with a new key, and only the certificate got deployed.
  2. The certificate was renewed on one server but the key came from another.
  3. Two renewals ran close together and the files got crossed.
  4. Someone regenerated the key "to be safe" after the CSR was already submitted.

Now imagine this at 500 hosts. The reason this is worth automating is that a mismatch is invisible until a restart — and restarts often happen hours or days later, during an unrelated deploy, at which point nobody connects the two events. A pre-deploy check that runs the diff one-liner and refuses to proceed on mismatch costs five lines and prevents a whole class of 3am incident.

🎯 Interview questions — Matching keys and certificates

Q. How do you check that a private key matches a certificate?

Derive the public key from both and compare:

bash
diff <(openssl x509 -in cert.crt -noout -pubkey) <(openssl pkey -in key.pem -pubout)

If they are identical, they are a pair. The older RSA-specific version compares modulus hashes with openssl x509 -noout -modulus | openssl md5 against openssl rsa -noout -modulus | openssl md5, but that only works for RSA — the -pubkey form works for EC too.

Why it works: the certificate contains the public key, and the public key can always be derived from the private key. So the public key is the shared identifier between the two files.

Where to use it: as a pre-deploy gate. A mismatch does not surface until the service restarts, which may be days after the bad deploy, so catching it at deploy time is worth the five lines.

Q. nginx says key values mismatch after a certificate renewal. Walk me through it.

First, confirm it with the diff check above rather than assuming — thirty seconds, no guessing.

Then find which half is wrong. The usual cause is that the certificate was issued from a different key than the one on the server: someone generated a fresh CSR (and therefore a fresh key) somewhere else, and only the resulting certificate was copied across.

Recovery: find the key that was used to create the CSR. If it still exists, deploy that key alongside the certificate. If it is genuinely lost, the certificate is useless — you must generate a new key, submit a new CSR, and reissue.

The prevention worth naming: keep the key and certificate together as a unit through the whole issuance flow, and let automation own it end to end. Every mismatch I have seen came from a manual step where the two halves travelled separately.


D3 · Files that hold more than one thing

The analogy. A PEM file is a folder, not a single sheet of paper. You can staple several documents into it and it is still one folder.

Your web server is often handed a folder containing your certificate and the certificates above it. It reads them in order, top to bottom.

Because PEM is just labelled text blocks, you can concatenate them. A file with three BEGIN CERTIFICATE blocks is completely valid, and this is how fullchain.pem files work.

Order matters, and it is the opposite of what feels natural. Your own certificate goes first, then the intermediate that signed it, then the one above that. Never put the root in.

Why it is that order, what breaks when it is wrong, and how to tell — all of that is Module 05. For now the only thing you need is: a PEM file can hold several objects, and the order is not arbitrary.

🧪 Exercise D3.1 — Split a bundle apart and count what is in it
bash
cd ~/tls-lab/m02

# chain.pem from C2.1 holds more than one certificate
grep -c 'BEGIN CERTIFICATE' chain.pem

# Print a one-line summary of every certificate in the file
openssl crl2pkcs7 -nocrl -certfile chain.pem \
  | openssl pkcs7 -print_certs -noout

# Split it into numbered single-certificate files
csplit -z -f cert- -b '%02d.pem' chain.pem '/BEGIN CERTIFICATE/' '{*}'
ls -l cert-*.pem

# Confirm each piece is a valid certificate on its own
for f in cert-*.pem; do
  printf '%-14s ' "$f"
  openssl x509 -in "$f" -noout -subject
done
Expected result — click to reveal
plain text
2

subject=CN=*.example.com
issuer=C=US, O=DigiCert Inc, CN=DigiCert Global G3 TLS ECC SHA384 2020 CA1

subject=C=US, O=DigiCert Inc, CN=DigiCert Global G3 TLS ECC SHA384 2020 CA1
issuer=C=US, O=DigiCert Inc, OU=DigiCert Global Root G3

-rw------- 1 zaeem zaeem 1424 Aug 20 14:10 cert-00.pem
-rw------- 1 zaeem zaeem 1180 Aug 20 14:10 cert-01.pem

cert-00.pem    subject=CN=*.example.com
cert-01.pem    subject=CN=DigiCert Global G3 TLS ECC SHA384 2020 CA1

What to read out of this.

  • grep -c 'BEGIN CERTIFICATE' is the fastest way to find out what you are holding. One means a single certificate. Two or three means a chain. It is the first thing to run when a tool complains about "too many certificates" or "expected a single certificate".
  • openssl x509 -in chain.pem would only ever show you the FIRST one. This trips people up constantly — you check the dates on a bundle, see a valid certificate, and never notice that the intermediate inside it expired last week. To see them all, use the crl2pkcs7 | pkcs7 -print_certs trick above.
  • Read the subject/issuer pairs as a ladder. Certificate 1's issuer is certificate 2's subject. That is the same ladder you saw in Module 01's Exercise D3.1, now in a file rather than on the wire.
  • csplit splits the bundle cleanly because each certificate begins with a predictable line. Handy when a tool insists on separate files for the leaf and the chain.

🔑 The habit to build: whenever someone hands you a "certificate", run grep -c BEGIN on it before anything else. Half of all certificate-deployment problems come from a bundle being given where a single certificate was expected, or the reverse.

🎯 Interview questions — Bundles

Q. What is the difference between cert.pem, chain.pem and fullchain.pem?

These are the names Let's Encrypt uses, and they have become a general convention:

  • cert.pem — your certificate alone, one object.
  • chain.pem — the intermediate certificate(s) above yours, without yours.
  • fullchain.pem — your certificate followed by the intermediates. This is what most web servers want.
  • privkey.pem — the private key, which is not a certificate at all.

nginx wants fullchain.pem in ssl_certificate. Apache historically wanted them split across SSLCertificateFile and SSLCertificateChainFile, though since 2.4.8 it accepts a combined file too.

The mistake to name: using cert.pem where fullchain.pem was needed. It works in your browser, because browsers cache intermediates they have seen before — and fails for other clients, producing unable to get local issuer certificate. That asymmetry is exactly why "it works in Chrome" is not a test.

Q. How do you inspect every certificate in a bundle rather than just the first?

openssl x509 -in bundle.pem -noout -text only ever reads the first object, which is a genuinely common source of missed problems — an expired intermediate hiding behind a valid leaf.

To see them all:

bash
openssl crl2pkcs7 -nocrl -certfile bundle.pem | openssl pkcs7 -print_certs -noout

Or split first with csplit and loop over the pieces.

Why it matters operationally: intermediates expire too, and monitoring that only checks the leaf will not warn you. Any expiry check worth having walks the whole bundle.


Part E · Putting it together

E1 · How this all fits — the complete picture

Diagram source
flowchart TD
    K["🔢 THE KEY ITSELF<br>just numbers<br>modulus, exponents, primes"]
    K --> S1["Structure it<br>PKCS#8 self-describing<br>or PKCS#1 RSA-only"]
    S1 --> D["📦 DER<br>the binary form<br>exactly one valid encoding"]
    D --> P["✉️ PEM<br>Base64 + BEGIN/END label<br>survives copy-paste and email"]
    D --> C1["💼 PKCS#12 .p12/.pfx<br>key + cert + chain<br>one password"]
    P --> C2["📚 Bundle .pem<br>several certificates<br>concatenated in order"]
    C1 --> J["🗄️ Java keystore<br>PKCS12 today<br>JKS historically"]
    style K fill:#ffe6cc,stroke:#d79b00,stroke-width:2px
    style D fill:#e1d5e7,stroke:#9673a6,stroke-width:2px
    style P fill:#d5e8d4,stroke:#82b366,stroke-width:2px

Read it from the top: the key is just numbers. Everything below that is packaging — first a structure that says what kind of key it is, then a strict binary encoding, then a text wrapper so it can travel, then optional containers that bundle several things together.

Nothing in this diagram changes the key. Every arrow is reversible and lossless. That is why the whole module is really about recognising which layer you are looking at, and why head -1 answers so many questions.


E2 · Production practice

HabitWhy
Run head -1 before you upload, paste or attach any certificate fileThe extension lies. Mixing up a .key and a .crt in the wrong direction is a private key disclosure
Prefer EC P-256 keys for web servers unless you must support very old clientsSame security as RSA-3072, roughly one twelfth the size, cheaper signing on every handshake
Store private keys as PKCS#8 (BEGIN PRIVATE KEY)Self-describing and accepted everywhere. PKCS#1 is rejected by Java and several other stacks
No passphrase on unattended server keys — use KMS, HSM or a secret managerA passphrase the machine can read is not protecting anything, and one it cannot read stops the service booting
Check key↔certificate match before deploying, not after restartingA mismatch is invisible until restart, which may be days later during an unrelated change
grep -c 'BEGIN CERTIFICATE' on anything called "the certificate"Tells you instantly whether it is a single certificate or a bundle, which is half of all deployment problems
Normalise line endings on every certificate file you receiveOpenSSL tolerates CRLF; Java, cloud consoles and some load balancers do not
Treat a .p12 exactly like a bare private keyIts password is usually weak, the default PBKDF2 iteration count is low, and it can be cracked offline
Never hand-edit a PEM file — replace it wholeRe-wrapping lines or "tidying" whitespace changes the bytes, and the bytes are what was signed
Manage Java's cacerts in the image or with config management, never by handA JDK upgrade replaces the file and silently discards anything added manually

E3 · Capstone exercise

Do this without looking anything up. A colleague has dropped a folder of badly-named files on you and gone on holiday. Work out what everything is, normalise it, and find the trap.

This exercises every section of the module: key types, DER versus PEM, PKCS#8 versus PKCS#1, line endings, containers, and key↔certificate matching.

Setup — run this to create the folder:

bash
mkdir -p ~/tls-lab/m02/capstone && cd ~/tls-lab/m02/capstone

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -outform DER -out a.dat
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out b.txt
cp ../../example-com.pem c.crt
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out /tmp/tmp.key
openssl rsa -in /tmp/tmp.key -traditional -out d.pem 2>/dev/null
sed -i 's/$/\r/' d.pem
rm -f /tmp/tmp.key
cp ../truststore.p12 e.key

ls -l

Brief. Produce a short report answering all six of these, using only the command line:

  1. For each of a.dat, b.txt, c.crt, d.pem, e.key — is it text or binary, and what does it contain?
  2. Which of them are private keys, and what key type and size is each?
  3. Convert every private key to unencrypted PKCS#8 PEM, named key1.pem, key2.pem, key3.pem.
  4. Does any key in the folder match the certificate in c.crt? Prove it either way.
  5. One file has Windows line endings. Find it and fix it.
  6. List the contents of e.key without writing anything to disk. The password is changeit.
Model answer — attempt it first, then click

1 & 2 — identify everything:

bash
cd ~/tls-lab/m02/capstone

for f in a.dat b.txt c.crt d.pem e.key; do
  echo "=== $f ==="
  file "$f"
  head -1 "$f" | cat -A | cut -c1-60
done
plain text
=== a.dat ===
a.dat: data
0..$.....                                    <- binary, no BEGIN line
=== b.txt ===
b.txt: ASCII text
-----BEGIN PRIVATE KEY-----$
=== c.crt ===
c.crt: PEM certificate
-----BEGIN CERTIFICATE-----$
=== d.pem ===
d.pem: ASCII text, with CRLF line terminators
-----BEGIN RSA PRIVATE KEY-----^M$          <- ^M is the carriage return
=== e.key ===
e.key: data

a.dat is binary and starts with 30 — DER. e.key is binary but does not start with 30, and its extension says .key; try it as PKCS#12. Now confirm the key details:

bash
openssl pkey -inform DER -in a.dat -noout -text | head -1
openssl pkey -in b.txt -noout -text | tail -2
openssl pkey -in d.pem -noout -text | head -1
plain text
Private-Key: (2048 bit, 2 primes)          <- a.dat: RSA 2048, DER
ASN1 OID: prime256v1
NIST CURVE: P-256                          <- b.txt: EC P-256, PKCS#8 PEM
Private-Key: (2048 bit, 2 primes)          <- d.pem: RSA 2048, PKCS#1, CRLF

Answer to 1 and 2: three private keys (a.dat RSA-2048 in DER, b.txt EC P-256 in PKCS#8 PEM, d.pem RSA-2048 in PKCS#1 with CRLF), one certificate (c.crt), one PKCS#12 container (e.key).

3 — normalise all three keys to unencrypted PKCS#8 PEM:

bash
openssl pkcs8 -topk8 -nocrypt -inform DER -in a.dat  -out key1.pem
openssl pkcs8 -topk8 -nocrypt -in b.txt              -out key2.pem
openssl pkcs8 -topk8 -nocrypt -in d.pem              -out key3.pem

head -1 key1.pem key2.pem key3.pem

All three now read -----BEGIN PRIVATE KEY-----. Note that openssl pkcs8 handled the CRLF file without complaint — but the original would still have been rejected by a Java service.

4 — does any key match the certificate?

bash
CERT_PUB=$(openssl x509 -in c.crt -noout -pubkey | sha256sum)
echo "cert : $CERT_PUB"
for k in key1.pem key2.pem key3.pem; do
  printf '%-10s ' "$k"
  openssl pkey -in "$k" -pubout | sha256sum
done
plain text
cert : b81e4f37d0a95c26e73f18b4a0d92c5e6f31870b4ac2d95e18f3b06d7c4a2e91  -
key1.pem   3f9a2c8e5b1d47f0a6c39e82b74d105fc8e3a19b6d24f870e5c1a93b47d6082e  -
key2.pem   d3e0917b45ca28f6013b7e9d5a8c46e2f70b91d3c85a24e6f7b53d0a1c96e482  -
key3.pem   8c4a1f92e7b530d6a8c19f4e73b02d5a6f81c93b47e0d258a1f6b3c9e074d582  -

No match, and that is the correct answer. c.crt is example.com's real certificate — its private key lives on that site's infrastructure, not in this folder. None of the three fingerprints comes close.

A second, faster argument worth putting in the report: compare the key types before you even hash anything.

bash
openssl x509 -in c.crt -noout -text | grep -A1 'Public Key Algorithm'
for k in key1.pem key2.pem key3.pem; do printf '%-10s ' "$k"; openssl pkey -in "$k" -noout -text | head -1; done

If the certificate holds an EC key and a candidate is RSA, they cannot possibly be a pair and there is no need to compare fingerprints at all. Ruling candidates out cheaply before doing the expensive check is a good habit generally.

This is the trap. The instinct is to keep converting until something matches. Nothing will. Recognising that the answer is legitimately "none of them" — and being able to prove it rather than just failing to find a match — is the point of requirement 4.

5 — find and fix the CRLF file:

bash
file *.pem *.crt *.txt 2>/dev/null | grep CRLF
sed -i 's/\r$//' d.pem
file d.pem
plain text
d.pem: ASCII text, with CRLF line terminators
d.pem: ASCII text

6 — list the container without extracting:

bash
openssl pkcs12 -info -in e.key -passin pass:changeit -nokeys -noout

-noout is what keeps it off disk. Two Certificate bag entries appear, no private key — so e.key is a truststore, despite being named .key. Worth flagging in the report: a file named .key that contains no key at all is exactly the kind of thing that causes an outage when somebody assumes.

The three things this capstone is really teaching:

  1. Identify before you act. Every one of the six answers came from file and head -1 before any conversion was attempted.
  2. Converting is safe; assuming is not. Format conversion never damages a key, so there is no risk in normalising early. The risk is entirely in guessing what a file is.
  3. "No match" is a valid, provable answer. Being able to demonstrate that a key and a certificate are not a pair is just as operationally useful as showing that they are.

E4 · Official documentation — what to bookmark and how to read it

The single most useful link for this module: OpenSSL Cookbook by Ivan Ristić — free to read online, third edition published June 2026. It is the practical companion to the reference manuals, organised by what you are trying to do rather than by command name.

Make it a reflex: when you need a command you have not used before, check the Cookbook first and the man page second. The man page tells you what every flag does; the Cookbook tells you which three flags you actually need.

Core reference pages

LinkWhat it is for
OpenSSL Cookbook (free online)Task-oriented recipes for key and certificate management. Start here
OpenSSL command indexEvery subcommand, each with its own page
openssl pkey · openssl genpkeyThe modern, key-type-agnostic commands. Prefer these over openssl rsa / openssl ec
openssl pkcs8 · openssl pkcs12Key format conversion, and building or unpacking .p12 containers
openssl asn1parseThe X-ray machine. Use it when a file will not load and you need to see the structure
RFC 7468 — Textual encodings (PEM)The definitive list of valid PEM labels and the exact rules for the BEGIN/END lines
RFC 5958 — PKCS#8 · RFC 8017 App. A.1 — PKCS#1 · RFC 5915 — EC keysThe three private key structures behind the three PEM labels
RFC 7292 — PKCS#12The .p12 / .pfx container format
ITU-T X.690 — BER, CER, DERThe encoding rules themselves. Dense, but the definitive source on why DER is unambiguous
Oracle keytool referenceEvery keytool flag, and confirmation that PKCS12 is the default keystore type since JDK 9
NIST SP 800-57 Part 1 — key managementThe authority to cite for key sizes and equivalent security strengths

How to read an OpenSSL conversion command

Almost every conversion follows the same four-part shape. Once you see the pattern you stop memorising commands:

plain text
openssl <subcommand>  -inform DER  -in old.file  -outform PEM  -out new.file
        |             |            |             |             |
        |             |            |             |             +-- where it goes
        |             |            |             +-- what form comes out
        |             |            +-- where it comes from
        |             +-- what form is going in
        +-- what KIND of object:  pkey / x509 / req / pkcs12 / pkcs8

The subcommand is the part people get wrong. It must match the kind of object, not the file extension: pkey for keys, x509 for certificates, req for CSRs, pkcs12 for .p12 containers.

The offline alternative

bash
openssl help                      # every subcommand
openssl pkcs12 -help              # every flag for one subcommand
man openssl-pkey                  # full man page, if the docs package is installed
keytool -help                     # Java's equivalent
keytool -list -help               # flags for one keytool subcommand
openssl list -key-managers        # what key types this build supports  (3.x)
🧪 Exercise E4.1 — Find the right flag without a browser

You have a .p12 and you need only the private key out of it, unencrypted. Work out the flags from the CLI alone.

bash
openssl pkcs12 -help 2>&1 | grep -iE 'nocerts|noenc|nodes|clcerts|nokeys'
Expected result — click to reveal
plain text
-nokeys             Don't output private keys
-nocerts            Don't output certificates
-clcerts            Only output client certificates
-cacerts            Only output CA certificates
-noenc              Don't encrypt private keys
-nodes              Don't encrypt private keys; deprecated

What to read out of this.

  • The flags are all negative, which is genuinely confusing until you see them listed together. You do not say "give me the key" — you say "don't give me the certificates". So the answer is -nocerts -noenc.
  • -nodes is listed as deprecated, with -noenc as the replacement. Almost every blog post and Stack Overflow answer still says -nodes. It still works, but knowing the current name is a small signal that you read the docs rather than the first search result.
  • -clcerts versus -cacerts is how you separate the leaf certificate from the intermediates when unpacking a bundle.

So the full command is:

bash
openssl pkcs12 -in bundle.p12 -nocerts -noenc -out server.key

⚠️ And immediately after running it: chmod 600 server.key. OpenSSL does not set restrictive permissions on files it extracts from a .p12, so the key can land world-readable if your umask is loose. This is a real and commonly missed step.


E5 · Self-assessment

Answer each out loud before opening it. If your answer is materially thinner than the one behind the toggle, that topic is worth a second pass.

1. What is the difference between PEM and DER?

DER is the raw binary encoding. PEM is that same binary, Base64-encoded and wrapped in -----BEGIN X----- / -----END X----- lines so it survives copy-paste and email.

Identical information, no loss either way. PEM is about a third larger. Tell them apart with head -1, or xxd | head -1 — DER almost always starts with byte 30.

2. BEGIN PRIVATE KEY versus BEGIN RSA PRIVATE KEY — what is the difference and why does it matter?

PKCS#8 versus PKCS#1. PKCS#8 is self-describing (the algorithm is named inside the file) and works for any key type; PKCS#1 is RSA-only with the type implied by the label.

It matters because Java and several other stacks accept only PKCS#8. Conversion is lossless: openssl pkcs8 -topk8 -nocrypt one way, openssl rsa -traditional the other.

3. Is a 256-bit EC key weaker than a 2048-bit RSA key?

No — it is roughly equivalent to RSA-3072, so slightly stronger. The bit counts measure different things and cannot be compared across families.

EC is also much smaller on the wire and cheaper to sign with, which is why it is the modern default for web servers.

4. How do you check whether a private key matches a certificate?

Derive the public key from both and compare:

bash
diff <(openssl x509 -in c.crt -noout -pubkey) <(openssl pkey -in k.pem -pubout)

It works because the certificate contains the public key, and the public key can always be derived from the private key. Run it as a pre-deploy gate — a mismatch is otherwise invisible until a restart.

5. What is in a .p12 file, and how carefully should you handle one?

A private key, its certificate, and usually the chain, in one password-protected binary container. .pfx is the same thing under Microsoft's name.

Handle it exactly like a bare private key. The default PBKDF2 iteration count is low and the password is usually weak, so it is crackable offline. Never email one.

6. Why can PKCS#12 tell you "invalid password" when openssl enc cannot?

PKCS#12 includes a MAC — a keyed integrity check. A wrong password fails the MAC immediately and unambiguously.

Plain CBC encryption has no such check; it decrypts to garbage and only notices via a padding error. That is the difference between authenticated and unauthenticated encryption, and it is why every TLS 1.3 suite is AEAD.

7. Should a production web server key have a passphrase?

Usually no. The server must start unattended, so the passphrase ends up somewhere the machine can read — typically a file next to the key, protecting nothing while adding an outage risk.

The real answers to key protection are KMS or HSM-backed keys, disk encryption, 0600 permissions and short certificate lifetimes. A passphrase is right for CA root keys and developer laptops, where a human is present anyway.

8. A Java app cannot reach an internal HTTPS service but curl works. What is your first hypothesis?

The JVM's own trust store, cacerts, does not contain the internal root — the root was added to the OS store, which Java never reads.

Fix with keytool -importcert -cacerts, or a custom truststore. And bake it into the image, because a JDK upgrade replaces cacerts and silently drops manual additions.

9. Someone hands you server.crt and says it is the certificate. What do you check first?

head -1 for the label, and grep -c 'BEGIN CERTIFICATE' for the count. The first tells you it is actually a certificate and not a key or a CSR; the second tells you whether it is a single certificate or a bundle.

Then openssl x509 -noout -subject -dates to confirm it parses and is current. Never trust the extension or the person's description.

10. Why must you never hand-edit a PEM file?

Because a signature covers the exact bytes. Re-wrapping lines, changing whitespace, or letting an editor rewrite line endings changes those bytes, and the signature no longer verifies.

Certificates are copied whole and replaced whole. If a file needs changing, regenerate or re-download it.

11. What does openssl asn1parse give you that openssl x509 -text does not?

-text gives a friendly interpretation, and only works if the file parses as the expected object type. asn1parse shows the raw nested structure — tags, lengths, offsets — regardless of what the object is supposed to be.

So it is the tool for a file that will not load: it tells you whether the internal structure is intact and you have the wrong command, or genuinely corrupt.


E6 · Command reference — everything from this module

The commands introduced in Module 02, grouped by what you are trying to achieve. The ones marked ⭐ are genuinely daily-use.

Identify a mystery file

bash
file suspicious.pem                            # ⭐ text or binary?
head -1 suspicious.pem                         # ⭐ if text, what does the label say?
xxd suspicious.der | head -1                   # ⭐ if binary, starts with 30? then DER
cat -A suspicious.pem | head -3                # ⭐ reveal CRLF, tabs, stray spaces
grep -c 'BEGIN CERTIFICATE' bundle.pem         # ⭐ single certificate or a bundle?
openssl asn1parse -in weird.file               # X-ray the structure when nothing loads

Generate keys

bash
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa.key    # ⭐
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ec.key   # ⭐ modern default
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-384 -out ec384.key
openssl genpkey -algorithm ED25519 -out ed.key                               # SSH, private CAs

Inspect keys

bash
openssl pkey -in k.pem -noout -text            # ⭐ everything inside
openssl pkey -in k.pem -noout -text | head -1  # ⭐ key type and size, fast
openssl pkey -in k.pem -pubout                 # ⭐ derive the public half
openssl pkey -in k.pem -noout -modulus         # RSA modulus only
head -1 k.pem                                  # ⭐ PKCS#8, PKCS#1 or encrypted?

Convert encodings

bash
openssl pkey -in k.pem -outform DER -out k.der           # PEM -> DER
openssl pkey -inform DER -in k.der -out k.pem            # DER -> PEM
openssl x509 -in c.pem -outform DER -out c.der           # certificate PEM -> DER
openssl x509 -inform DER -in c.der -out c.pem            # ⭐ certificate DER -> PEM
openssl pkcs8 -topk8 -nocrypt -in k.pem -out k8.pem      # ⭐ -> PKCS#8 (BEGIN PRIVATE KEY)
openssl rsa -in k8.pem -traditional -out k1.pem          # -> PKCS#1 (BEGIN RSA PRIVATE KEY)

Passphrases

bash
openssl pkey -in k.pem -aes256 -out k-locked.pem         # add a passphrase
openssl pkey -in k-locked.pem -out k.pem                 # ⭐ remove a passphrase
openssl pkey -in k-locked.pem -passin pass:secret -noout -text   # supply it non-interactively
openssl pkey -in k-locked.pem -passin file:/path/pw -noout       # from a file

PKCS#12 containers

bash
openssl pkcs12 -export -inkey k.pem -in c.crt -certfile chain.pem -out b.p12   # ⭐ build
openssl pkcs12 -info -in b.p12 -noout                    # ⭐ what is inside, nothing written
openssl pkcs12 -in b.p12 -noenc -out all.pem             # ⭐ unpack everything (UNENCRYPTED key)
openssl pkcs12 -in b.p12 -nocerts -noenc -out k.pem      # ⭐ just the key
openssl pkcs12 -in b.p12 -clcerts -nokeys -out c.pem     # ⭐ just the leaf certificate
openssl pkcs12 -in b.p12 -cacerts -nokeys -out chain.pem # just the intermediates
chmod 600 k.pem                                          # ⭐ ALWAYS, after extracting a key

Bundles

bash
grep -c 'BEGIN CERTIFICATE' fullchain.pem                              # ⭐ how many?
openssl crl2pkcs7 -nocrl -certfile bundle.pem | openssl pkcs7 -print_certs -noout   # ⭐ list ALL
csplit -z -f cert- -b '%02d.pem' bundle.pem '/BEGIN CERTIFICATE/' '{*}'  # split apart
cat server.crt intermediate.crt > fullchain.pem                        # ⭐ build one
openssl pkcs7 -inform DER -in c.p7b -print_certs -out chain.pem         # .p7b -> PEM

Java

bash
keytool -list -keystore store.p12 -storetype PKCS12                     # ⭐ what is in it
keytool -list -cacerts -storepass changeit                              # ⭐ Java's OWN trust store
keytool -importcert -cacerts -alias internal-root -file root.crt        # ⭐ trust an internal CA
keytool -importkeystore -srckeystore a.jks -destkeystore a.p12 -deststoretype PKCS12

Fixing common damage

bash
sed -i 's/\r$//' cert.pem            # ⭐ strip Windows line endings
dos2unix cert.pem                    # ⭐ same thing, if installed
tail -c 1 cert.pem | xxd             # is there a trailing newline? (should end 0a)
The three-command reflex for any certificate file somebody hands you, before you do anything else with it:
bash
file mystery.pem                            # text or binary
head -1 mystery.pem                         # what does it claim to be
grep -c 'BEGIN' mystery.pem                 # one object or several

Three commands, three seconds, nothing changed — and between them they prevent both the commonest deployment failure and the commonest accidental private-key disclosure.


Next — Module 03 · Inside an X.509 Certificate.

You have now opened up a key file and seen exactly what is in it. In Module 01's Exercise C2.1 you pulled a real certificate off the internet and read four lines of it — subject, issuer, dates, serial. Module 03 opens the whole thing: every field, every extension, what SAN really is and why the Common Name stopped mattering, and how to read openssl x509 -text output line by line without skipping the parts you do not recognise.

Official reading ahead of it: RFC 5280 §4 — Certificate and Certificate Extensions Profile and openssl x509.

📚 Sources for the interview questions

Question selection was cross-referenced against publicly published 2026 SSL/TLS, OpenSSL and Java keystore interview question sets, then rewritten and deepened:

Technical claims were verified against primary sources rather than the question sets: RFC 7468, RFC 5958, RFC 5915, RFC 7292, RFC 8017, ITU-T X.690, NIST SP 800-57 Part 1, the Oracle keytool reference and the OpenSSL 3.x manual pages.

Answers were rewritten and deepened rather than reproduced — published versions are usually correct but shallow, and the added operational detail is what actually differentiates a candidate in the room.

Spotted a mistake or want something added? Send me a note.