Entropy Gradient Reversal Or: How we survived CVE-2008-0166 Kees - - PowerPoint PPT Presentation

Entropy Gradient Reversal

Or: How we survived CVE-2008-0166 Kees Cook Matt Zimmerman Opsen 2014-04-22

May 2008

“why do you have my key?”

Fix it quick!

• penssl (0.9.8g-9) unstable; urgency=high

[ Christoph Martin ] * Include updated debconf translations (closes: #473477, #461597, #461880, #462011, #465517, #475439) [ Kurt Roeckx ] * ssleay_rand_add() really needs to call MD_Update() for buf.

• - Kurt Roeckx <kurt@roeckx.be>

Wed, 07 May 2008 20:32:12 +0200

“Time to release an advisory”

“hold on just a tick…”

Relevant to your interests

From: Florian Weimer <fw@deneb.enyo.de> To: vendor-sec@lst.de Subject: [vendor-sec] Broken openssl in Debian Date: Thu, 08 May 2008 01:52:17 +0200 Debian ships a bogus modification of OpenSSL which disables proper seeding of the PRNG (the patch to crypto/rand/md_rand.c). The issue is already semi-public, thanks to an uncoordinated upload by the maintainer. This should only affect Debian-derived distros.

Bank holiday Monday

Meanwhile, downstream...

Cast of characters

• Kees Cook, Ubuntu security team (on vacation in Canada)
• Jamie Strandboge, Ubuntu security team
• Colin Watson, Debian/Ubuntu OpenSSH maintainer
• Matt Zimmerman, Ubuntu CTO
• Florian Weimer, Red Hat Product Security Team
• James Troup, Debian/Canonical

Test case

/* gcc -o omfg omfg.c -lcrypto */ #include <openssl/rand.h> #include <stdio.h> int main() { unsigned char data[128]; unsigned i; RAND_bytes(data, 128); for (i = 0; i < 128; ++i) printf("%08X", (unsigned)data[i]); puts(""); } $ for i in $(seq 1 1000000); do ./omfg; done | sort -u | wc -l 32003 <- this number should be 1000000

Sources of “entropy”

• process ID
• CPU endianness

Find some alternate endian machines and LD_PRELOAD “getpid”:

http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/trusty/openssh- blacklist/trusty/view/head:/generate-blacklist.sh

It gets worse

• All SSH keys generated by this code
• All SSL certificates generated by this code
• All OpenVPN certificates generated by this

code

Response plan

“This is outstandingly bad. At the very least, we need to: 1- fix it (patch is trivial) 2- educate people that they need to regenerate their keys and certs 3- generate the list of "weak keys" and refuse to use them for SSH 4- hope no one has packet captures from weak SSL connections 5- hope the world doesn't burn 6- find out if places like cacert.org (and others) are using Debian-derived openssl 7- pray we are not brutally lampooned”

Choosing the lesser evil

May 13 2008

Updating your system:

• 1. Install the security updates

Once the update is applied, weak user keys will be automatically rejected where possible (though they cannot be detected in all cases). If you are using such keys for user authentication, they will immediately stop working and will need to be replaced (see step 3). OpenSSH host keys can be automatically regenerated when the OpenSSH security update is applied. The update will prompt for confirmation before taking this step.

• 2. Update OpenSSH known_hosts files

The regeneration of host keys will cause a warning to be displayed when connecting to the system using SSH until the host key is updated in the known_hosts file. The warning will look like this: [...]

• 3. Check all OpenSSH user keys

The safest course of action is to regenerate all OpenSSH user keys, except where it can be established to a high degree of

certainty that the key was generated on an unaffected system. Check whether your key is affected by running the ssh-vulnkey tool, included in the security update. By default, ssh-vulnkey will check the standard location for user keys (~/.ssh/id_rsa, ~/.ssh/id_dsa and ~/.ssh/identity), your authorized_keys file (~/.ssh/authorized_keys and ~/.ssh/authorized_keys2), and the system's host keys (/etc/ssh/ssh_host_dsa_key and /etc/ssh/ssh_host_rsa_key). To check all your own keys, assuming they are in the standard locations (~/.ssh/id_rsa, ~/.ssh/id_dsa, or ~/.ssh/identity): $ ssh-vulnkey To check all keys on your system: $ sudo ssh-vulnkey -a To check a key in a non-standard location: $ ssh-vulnkey /path/to/key If ssh-vulnkey says "Unknown (no blacklist information)", then it has no information about whether that key is affected. If in doubt, destroy the key and generate a new one.

• 4. Regenerate any affected user keys

OpenSSH keys used for user authentication must be manually regenerated, including those which may have since been transferred to a different system after being generated. New keys can be generated using ssh-keygen, e.g.: $ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/user/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/user/.ssh/id_rsa. Your public key has been saved in /home/user/.ssh/id_rsa.pub. The key fingerprint is: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 user@host

• 5. Update authorized_keys files (if necessary)

Once the user keys have been regenerated, the relevant public keys must be propagated to any authorized_keys files on remote systems. Be sure to delete the affected key.

Details

USN-612-3 addressed a weakness in OpenSSL certificate and keys generation in OpenVPN by adding checks for vulnerable certificates and keys to OpenVPN. A regression was introduced in OpenVPN when using TLS, multi-client/server mode, and specifying a user or group which caused OpenVPN to not start when using valid SSL certificates. It was also found that openssl-vulnkey from openssl-blacklist would fail when stderr was not available. This caused OpenVPN to fail to start when used with applications such as NetworkManager. This update fixes these problems. We apologize for the inconvenience.

May 14 2008

How did we get here?

20 months earlier…

How did we get here?

How did we get here?

Debian package maintainer,

• penssl

How did we get here?

“If it helps with debugging, I'm in favor of removing them”

• Ulf Möller, OpenSSL contributor

“use -DPURIFY”

• Geoff Thorpe, Member, OpenSSL development team

“So yes I think not using the uninitialized memory [...] helps valgrind.”

• Marco Roeland, some dude on the internet

How did we get here?

• penssl (0.9.8b-1) unstable; urgency=low

* New upstream release

• New functions added (EVP_CIPHER_CTX_new, EVP_CIPHER_CTX_free), bump shlibs.
• CA.pl/CA.sh now calls openssl ca with -extensions v3_ca, setting CA:TRUE

instead of FALSE.

• CA.pl/CA.sh creates crlnumber now. (Closes: #347612)

* Run debconf-updatepo, which really already was in the 0.9.8a-8 version as it was uploaded. * Add Galician debconf translation. Patch from Jacobo Tarrio <jtarrio@trasno.net> (Closes: #361266) * libssl0.9.8.postinst makes uses of bashisms (local variables) so use #!/bin/bash * libssl0.9.8.postinst: Call set -e after sourcing the debconf script. * libssl0.9.8.postinst: Change list of service that may need to be restarted:

• Replace ssh by openssh-server
• Split postgresql in postgresql-7.4 postgresql-8.0 postgresql-8.1
• Add: dovecot-common bind9 ntp-refclock ntp-simple openntpd clamcour

fetchmail ftpd-ssl proftpd proftpd-ldap proftpd-mysql proftpd-pgsql * libssl0.9.8.postinst: The check to see if something was installed wasn't working. * libssl0.9.8.postinst: Add workaround to find the name of the init script for proftpd and dovecot. * libssl0.9.8.postinst: Use invoke-rc.d when it's available. * Change Standards-Version to 3.7.0:

• Make use of invoke-rc.d

* Add comment to README.Debian that rc5, mdc2 and idea have been disabled (since 0.9.6b-3) (Closes: #362754) * Don't add uninitialised data to the random number generator. This stop valgrind from giving error messages in unrelated code. (Closes: #363516) * Put the FAQ in the openssl docs. * Add russian debconf translations from Yuriy Talakan <yt@amur.elektra.ru> (Closes #367216)

• - Kurt Roeckx <kurt@roeckx.be> Thu, 4 May 2006 20:40:03 +0200

What happened?

• A subtle bug was discovered
• A well-meaning developer attempted to fix it
• They sought a trusted opinion about their patch

○ ...and seemed to receive it

• An even more subtle bug was introduced

○ There was just enough entropy to obscure the issue

• A long time passed!
• A lot of people had a bad day^Wweek^Wlife

It could have been worse

• The issue could have become public before

fixes were available

• A Debian advisory was almost released

several days earlier, without key blacklists and other critical components

• It could have happened in upstream

OpenSSL

Post hoc

Criticism

“My issue is that it was committed to a public repository five days before an advisory was

• issued. Only a single attacker has to notice that and realise its import in order to start exploiting

vulnerable systems – and I will be surprised if that has not happened.”

• Ben Laurie, Member, OpenSSL core team

Criticism

Criticism

“Fifthly, I said that openssl-dev was not the way to ensure you had the attention of the OpenSSL team. Many have pointed out that the website says it is the place to discuss the development of OpenSSL, and this is true, it is what it says. But it is wrong. The reality is that the list is used to discuss application development questions and is not reliably read by the development team.”

• Ben Laurie, Member, OpenSSL core team

Criticism

“It seems very unlikely that the patch in question would have ever made it into an OpenSSL release [if submitted upstream]”

• Jake Edge, Linux Weekly News

CVE-2014-0166

... hashOut.data = hashes + SSL_MD5_DIGEST_LEN; hashOut.length = SSL_SHA1_DIGEST_LEN; if ((err = SSLFreeBuffer(&hashCtx)) != 0) goto fail; if ((err = ReadyHash(&SSLHashSHA1, &hashCtx)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &clientRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; goto fail; /* MISTAKE! THIS LINE SHOULD NOT BE HERE */ if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; err = sslRawVerify(...); ...

CVE-2014-0160

Lessons learned

• Crypto implementations are subtle and quick

to anger

• Humans are fallible
• Humans collaborating across contexts are

even more fallible

• Use UNIQUE constraints, and listen to

them!

• Remember to laugh in the face of disaster

References

• Debian bug report #363516 (2006-04-19)
• Debian, OpenSSL and a Lack of Cooperation

(Linux Weekly News, 2008-05-14)

• Debian and OpenSSL: the aftermath (Ben

Laurie, 2008-05-14)

• H.D. Moore's debian-openssl page (archive.
• rg)

Thank you