Out of memory (Wikipedia Lab Guide)

# Out of Memory (OOM) Conditions: A Deep Dive for System Engineers

## 1) Introduction and Scope

This study guide provides a technically detailed examination of Out of Memory (OOM) conditions in modern computer systems. We will explore the underlying mechanisms, architectural implications, practical manifestations, and defensive strategies related to memory exhaustion. The scope encompasses both physical and virtual memory limitations, common operating system behaviors, and the implications for application stability and system resilience. This guide is intended for system administrators, kernel developers, and security professionals seeking a profound understanding of memory management failures.

## 2) Deep Technical Foundations

### 2.1) Memory Addressing and Architectures

Early computing architectures were severely constrained by the physical limitations of RAM and the addressing capabilities of processors.

*   **Limited Address Spaces:** 8-bit and 16-bit processors had very small addressable memory spaces (e.g., 64KB for Intel 8080, 1MB for Intel 8086). This meant that even small programs could exhaust available physical RAM. For instance, an 8086 CPU with a 20-bit address bus could only address $2^{20}$ bytes (1 MiB) of physical memory. Any attempt to access beyond this would result in undefined behavior or hardware-level faults.

*   **Physical Address Extension (PAE):** Introduced in x86 architectures (specifically the Intel Pentium Pro and later), PAE allowed 32-bit processors to access more than 4GB of physical RAM. It achieved this by introducing a third level of page tables (page directory pointer tables), expanding the physical address bus width to 36 bits, thus enabling access to $2^{36}$ bytes (64 GiB) of physical memory. However, individual processes were still typically limited to a 4GB virtual address space (though some OS extensions could increase this).

*   **64-bit Architectures:** Modern 64-bit systems (e.g., x86-64, ARMv8-A) offer vastly larger virtual address spaces. The theoretical maximum for x86-64 is 256 TiB (using 48-bit virtual addresses) or even 16 EiB (using 64-bit virtual addresses), though current implementations typically use 48-bit virtual addresses. This significantly reduces the likelihood of a single process exhausting its virtual address space. The actual physical memory limit is determined by the motherboard and CPU's memory controller capabilities.

### 2.2) Virtual Memory and Paging

Virtual memory is a fundamental abstraction that decouples the logical address space of a process from the physical RAM. It allows processes to use more memory than physically available and provides memory protection.

*   **Page Tables:** The Memory Management Unit (MMU), a hardware component integrated into the CPU, uses page tables to translate virtual addresses generated by the CPU into physical addresses. A page table is a hierarchical data structure. For example, in a 4-level paging system (common in x86-64), a 48-bit virtual address is typically broken down into 5 parts:
    *   Bits 47-39: Page Map Level 4 (PML4) index
    *   Bits 38-30: Page Directory Pointer Table (PDPT) index
    *   Bits 29-21: Page Directory (PD) index
    *   Bits 20-12: Page Table (PT) index
    *   Bits 11-0: Page Offset (determines the byte within the page)

    Each entry in a page table (Page Table Entry or PTE) contains the physical frame number and status bits. Common PTE bits include:
    *   **P (Present):** Indicates if the page is currently in physical RAM.
    *   **R/W (Read/Write):** Controls write permissions.
    *   **U/S (User/Supervisor):** Controls user-mode access.
    *   **A (Accessed):** Set by the hardware when the page is accessed.
    *   **D (Dirty):** Set by the hardware when the page is written to.
    *   **NX (No Execute) / XD (Execute Disable):** Prevents code execution from this page.

    ```
    Virtual Address (48 bits)
    +-----------------+-----------------+-----------------+-----------------+-----------------+
    | PML4 Index (9)  | PDPT Index (9)  | PD Index (9)    | PT Index (9)    | Page Offset (12)|
    +-----------------+-----------------+-----------------+-----------------+-----------------+
                                     |
                                     v
    PML4 Table (e.g., 512 entries)
        |
        v
    PDPT Table (e.g., 512 entries)
        |
        v
    Page Directory (e.g., 512 entries)
        |
        v
    Page Table (e.g., 512 entries)
        |
        v
    Physical Frame (e.g., 4 KiB)
    +-----------------+
    | Page Offset (12)|
    +-----------------+
    ```

*   **Paging:** When a process accesses a virtual address whose corresponding physical page is not currently in RAM (P bit in PTE is 0), a **page fault** occurs. This is a CPU exception. The operating system's page fault handler then:
    1.  **Validate Access:** Checks if the virtual address is valid for the process and if the attempted operation (read/write/execute) is permitted based on PTE flags. If not, it's a segmentation fault/access violation.
    2.  **Locate Page:** If valid, it determines if the page is stored on disk (swap space or file-backed). This information is typically encoded in the PTE or a separate data structure.
    3.  **Page Reclamation:** If no free physical frame is available, the OS selects a victim page using page replacement algorithms (e.g., approximated LRU, clock algorithm).
    4.  **Write Back Dirty Pages:** If the victim page is "dirty" (modified, D bit set) and not shared, it's written back to its backing store (swap file or original file).
    5.  **Load Page:** The requested page is read from its backing store into the newly freed physical frame.
    6.  **Update PTE:** The PTE for the faulting virtual address is updated with the physical frame number and flags (P=1, etc.).
    7.  **Resume Execution:** The interrupted instruction is restarted.

    ```pseudocode
    // Simplified Page Fault Handler (Kernel Mode)
    function handle_page_fault(faulting_virtual_address, error_code):
        // error_code bits indicate: Present, Write, User/Supervisor
        is_present = (error_code & 1)
        is_write_fault = (error_code & 2)
        is_user_mode = (error_code & 4)

        process = current_process()
        pte = lookup_page_table_entry(process.page_directory_base, faulting_virtual_address)

        if pte is invalid or not found:
            // Address is not mapped or invalid for this process
            raise_segmentation_fault(faulting_virtual_address)
            return

        if is_present:
            // Page is in RAM, but fault is due to permission violation
            if is_write_fault and not pte.writable:
                // Handle copy-on-write or permission error
                if pte.is_shared_and_readonly: // COW scenario
                    frame = allocate_physical_frame()
                    if frame is null:
                        trigger_oom_killer("Page fault: no frames for COW")
                        return
                    copy_page(pte.physical_frame_number, frame)
                    update_pte(pte, frame, writable=true)
                    resume_execution()
                else: // Permission violation
                    raise_protection_fault(faulting_virtual_address)
                    return
            else: // Access is valid (e.g., read to read-only page is okay if P=1)
                resume_execution() // Might be a protection fault that is allowed
            return

        // Page is not present in RAM
        if pte.on_disk:
            // Page is swapped out or file-backed
            frame = allocate_physical_frame()
            if frame is null:
                // System is under memory pressure, try to reclaim
                victim_va = select_victim_page_for_eviction()
                victim_pte = lookup_page_table_entry(process.page_directory_base, victim_va)

                if victim_pte.dirty and not victim_pte.is_shared:
                    write_page_to_disk(victim_va, victim_pte.disk_location)
                free_physical_frame(victim_pte.physical_frame_number)
                update_pte(victim_pte, invalid=true) // Mark as not present
                frame = victim_pte.physical_frame_number // Reuse the freed frame

                if frame is null: // Still no frame after reclamation
                    trigger_oom_killer("Page fault: no frames after reclamation")
                    return

            load_page_from_disk(faulting_virtual_address, frame)
            update_pte(pte, frame, present=true, writable=(pte.writable_flag), on_disk=false)
            resume_execution()
        else:
            // Page is not valid (e.g., accessing unmapped memory)
            raise_segmentation_fault(faulting_virtual_address)
            return
    ```

*   **Swap Space:** A dedicated partition or file on disk used by the OS to store pages that have been evicted from physical RAM. The size of the swap space is a critical system configuration parameter.
*   **Memory-Mapped Files (mmap/MapViewOfFile):** Allows a file on disk to be mapped directly into a process's virtual address space. Accessing pages within this range triggers page faults, and the OS reads/writes directly to/from the file. This can consume virtual address space and potentially disk space if modified pages are written back. Anonymous memory mappings (not backed by a file) are also common and use swap space.

### 2.3) Commit Charge (Windows)

Windows uses a "commit charge" mechanism to track the total amount of virtual memory that the system has *committed* to backing with physical resources (RAM or page file). The commit limit is the sum of available physical memory and the size of the page file(s). When a process requests memory (e.g., via `VirtualAlloc`), the OS checks if the commit limit can accommodate the requested committed virtual memory. If the commit charge exceeds the commit limit, subsequent memory allocation requests that require committing memory will fail, often with `ERROR_NOT_ENOUGH_QUOTA`. This prevents the system from overcommitting beyond its capacity to provide backing store.

## 3) Internal Mechanics / Architecture Details

### 3.1) Memory Allocation Mechanisms

*   **Kernel Allocators:** The OS kernel itself requires memory for data structures like process control blocks (PCBs), file descriptors, page tables, kernel threads, network buffers, and device driver data. It uses specialized, highly optimized allocators:
    *   **Buddy Allocator:** Manages contiguous blocks of physical memory. It divides blocks into halves recursively until a suitable size is found. When blocks are freed, it attempts to merge adjacent "buddy" blocks to create larger free blocks.
    *   **Slab Allocator (Linux):** Built on top of the buddy allocator, it caches frequently used kernel objects (e.g., `inode` structures, `task_struct`s). It pre-allocates "slabs" of memory and carves out fixed-size objects from them, reducing fragmentation and allocation overhead.

*   **User-Space Allocators:** Applications typically use libraries like `malloc` (C), `new` (C++), or managed memory runtimes (Java, .NET). These libraries often maintain heaps and use system calls to request larger chunks of memory from the kernel when their internal heaps are exhausted:
    *   **Unix-like Systems:**
        *   `sbrk()`: Increments the program's data segment size. Older, less efficient.
        *   `mmap()`: Maps anonymous memory or files into the process's address space. Preferred for large or file-backed allocations. `malloc` implementations often use `mmap` for large requests and a private heap for smaller, frequent allocations.
    *   **Windows:**
        *   `VirtualAlloc()`: The primary API for reserving or committing virtual memory. `HeapAlloc` is often used on top of `VirtualAlloc` for managing heaps.

### 3.2) Copy-on-Write (COW)

COW is a memory optimization technique used by `fork()` system calls on Unix-like systems.

*   When a process forks, the child process initially shares the parent's memory pages. The OS marks these shared pages as read-only in both processes' page tables.
*   If either the parent or child attempts to write to a shared page, a page fault occurs.
*   The OS's page fault handler intercepts this. It then:
    1.  Allocates a new physical page.
    2.  Copies the contents of the original shared page to the new page.
    3.  Updates the writing process's page table to point to the new, private copy, marking it as writable.
    4.  The original shared page remains read-only for the other process.
    5.  Execution resumes.

**OOM Implication of COW:** If a system has many processes forked without significant writes, memory usage might appear low due to sharing. However, if many of these processes subsequently attempt to write to their shared pages, the system might suddenly need to allocate many new copies of these pages. This can lead to a rapid increase in physical memory consumption and potentially trigger an OOM condition if the system's memory capacity is exceeded. This is particularly relevant in scenarios involving large numbers of forked processes, such as web servers handling many requests or parallel processing frameworks.

### 3.3) The Linux OOM Killer

When the Linux kernel detects a critical OOM situation (i.e., it cannot satisfy a memory allocation request, even after attempting to reclaim memory via page replacement and swapping), it invokes the OOM Killer. This is a panic-avoidance mechanism.

*   **Heuristics and Scoring:** The OOM Killer uses a scoring system to select a process to terminate. The goal is to kill the "least useful" process that will free up the most memory. Factors influencing the score include:
    *   `vm_size`: Total virtual memory size of the process.
    *   `rss`: Resident Set Size (physical memory used).
    *   `oom_score_adj`: A user-configurable value (from -1000 to +1000) that adds a penalty to the score. Setting it to -1000 makes the process immune to the OOM Killer. Default is 0.
    *   Process priority (`nice` value).
    *   CPU time consumed.
    *   Whether the process is running as root.
    *   Whether the process is in a `sysadmin` group.
    *   Whether the process is `init` or a kernel thread (these are generally immune).

    The formula is roughly: `score = (oom_score_adj * 1000 / MAX_ADJ) + (mem_usage_in_pages * K)`. Processes with higher scores are more likely to be killed.

*   **Selection and Termination:** The process with the highest OOM score is chosen for termination. The kernel then sends a `SIGKILL` signal to this process, which cannot be caught or ignored, ensuring immediate termination.
*   **Logging:** The OOM Killer logs its actions to `dmesg` or `/var/log/messages` (depending on system configuration), providing crucial debugging information about which process was killed, its memory usage, and its OOM score.

**Example OOM Killer Log Snippet:**

[12345.678901] Out of memory: Kill process 9876 (apache2) score 1000 or sacrifice child
[12345.678910] Killed process 9876 (apache2) total-vm:1234567kB, anon-rss:87654kB, file-rss:1234kB, shmem-rss:567kB
[12345.679000] oom_reaper: reaped memory. nr_reclaimed=12345

This log indicates that the kernel identified an OOM condition, calculated scores, and decided to kill process `9876` (identified as `apache2`) because it had a high score (1000 in this simplified example). It also shows the memory breakdown of the killed process.

### 3.4) Control Groups (cgroups) and OOM

cgroups provide a mechanism for resource isolation and management. The OOM Killer can be cgroup-aware, allowing for more granular control over memory limits.

*   **cgroup OOM Reaper:** When a cgroup hits its configured memory limit (e.g., `memory.max` in cgroup v1, `memory.high` in cgroup v2), the kernel can trigger an OOM event *specifically within that cgroup*. The OOM Killer will then select a process *from within that specific cgroup* to kill, preventing the OOM condition from affecting the entire system. This is fundamental for container orchestration platforms like Kubernetes.
*   **cgroup v2 `memory.high`:** Setting a `memory.high` threshold can trigger proactive memory reclamation or OOM events *before* the hard `memory.max` limit is reached. This allows for more graceful handling of memory pressure within a cgroup.

### 3.5) Pressure Stall Information (PSI)

PSI is a Linux kernel feature that provides visibility into resource pressure. It measures the amount of time a task (or group of tasks) has spent waiting for a resource (CPU, memory, I/O).

*   **`psi` Filesystem:** Mounted at `/proc/pressure/`, it exposes metrics for CPU, memory, and I/O pressure.
    *   `/proc/pressure/memory` contains `avg10`, `avg60`, `avg300` (average time waiting for memory in the last 10, 60, and 300 seconds) and `total` (total time waiting since boot).
    *   `full` indicates pressure on all memory resources (RAM + swap).
    *   `some` indicates pressure on RAM, but swap might still be available.
*   **Memory Pressure:** High memory pressure indicates that the system is spending significant time reclaiming memory (paging/swapping) or waiting for memory allocations to succeed. This is a strong precursor to OOM conditions.
*   **`systemd-oomd`:** This systemd service daemon monitors PSI metrics. If it detects sustained memory pressure (e.g., `memory.pressure.some` or `memory.pressure.full` exceeding thresholds), it can proactively take action *before* the kernel's OOM Killer is invoked. Actions can include killing specific processes based on their memory usage and priority, or even initiating a system reboot if configured. This can lead to a smoother recovery and prevent the abrupt termination of critical services.

## 4) Practical Technical Examples

### 4.1) Heap Exhaustion in C

A common scenario is a program that continuously allocates memory on the heap without freeing it, leading to a memory leak.

```c
#include <stdio.h>
#include <stdlib.h>
#include <string.h> // For memset
#include <unistd.h> // For usleep

#define MB (1024 * 1024)

int main() {
    void *ptr;
    size_t allocation_size = 1 * MB; // Allocate 1MB chunks
    unsigned long long allocation_count = 0;

    printf("Starting memory allocation loop. Press Ctrl+C to stop.\n");

    while (1) {
        // malloc typically uses mmap for large allocations on modern systems
        // or manages its own heap using sbrk/VirtualAlloc for smaller chunks.
        ptr = malloc(allocation_size);

        if (ptr == NULL) {
            perror("malloc failed");
            fprintf(stderr, "Out of memory condition detected.\n");
            fprintf(stderr, "Total allocations before failure: %llu\n", allocation_count);
            break;
        }

        // Optional: Initialize the allocated memory to avoid zero-fill costs
        // and to make it more realistic that the memory is being used.
        // memset(ptr, 0xAA, allocation_size);

        allocation_count++;
        if (allocation_count % 100 == 0) { // Print status every 100MB
            printf("Allocated %llu MB at %p\n", allocation_count, ptr);
        }

        // Optional: Add a small sleep to make it observable and prevent
        // immediate system lockup if running on limited resources.
        // usleep(1000); // 1ms sleep
    }

    printf("Exiting.\n");
    // In a real program, you would free allocated memory here if possible.
    // However, in an OOM scenario, this point might not be reached.
    return 0;
}

Execution and Observation:
Compile and run this program. On a system with limited RAM and swap, it will eventually cause malloc to return NULL. The dmesg output on Linux will likely show the OOM Killer selecting this process.

To observe ulimit impact (see 4.3):
Before running, execute ulimit -v 1048576 (limits virtual memory to 1GB in KB). The program will fail much sooner.

4.2) Virtual Memory Exhaustion with mmap

Mapping large files or anonymous memory regions can exhaust virtual address space or commit charge.

import mmap
import os
import sys

# Size of the memory map (e.g., 2GB)
MAP_SIZE = 2 * 1024 * 1024 * 1024 # 2 GiB

print(f"Attempting to create memory maps of size {MAP_SIZE} bytes...")

maps = []
try:
    # Keep creating maps until an error occurs
    while True:
        # Create an anonymous memory map (backed by swap/RAM)
        # MAP_PRIVATE ensures it's not shared and writes will cause copies.
        # Use mmap.MAP_ANONYMOUS on systems that support it (Linux, macOS).
        # On Windows, this requires specific flags or a temporary file.
        if sys.platform.startswith('linux') or sys.platform.startswith('darwin'):
            mem_map = mmap.mmap(-1, MAP_SIZE, flags=mmap.MAP_PRIVATE | mmap.MAP_ANONYMOUS)
        elif sys.platform.startswith('win'):
            # On Windows, MAP_ANONYMOUS is not directly supported.
            # We can simulate it by mapping a zero-filled file, but for
            # demonstrating virtual address space exhaustion, just trying
            # to allocate large virtual memory is often sufficient.
            # For a true Windows demo, one might need to use ctypes and
            # VirtualAlloc directly. Here, we'll rely on Python's higher-level
            # abstractions which might use VirtualAlloc internally.
            # A simple mmap of a large size might fail due to commit charge.
            # Let's try a simpler approach that might hit commit limit.
            try:
                # This might fail directly with MemoryError or WindowsError
                mem_map = mmap.mmap(-1, MAP_SIZE) # Uses system default backing
            except (MemoryError, OSError) as e:
                print(f"Windows mmap failed: {e}. Likely hit commit limit or other quota.")
                break
        else:
            print(f"Unsupported platform: {sys.platform}")
            break

        # Attempt to write to the map to force page allocation (if supported and needed)
        # This ensures the virtual pages are actually backed by physical resources.
        try:
            mem_map[0] = b'A'
            print(f"Successfully mapped {MAP_SIZE} bytes. Map object: {mem_map}")
            maps.append(mem_map) # Keep the map alive
        except (ValueError, IndexError, OSError) as e:
            print(f"Error writing to map: {e}. This map might not be usable.")
            mem_map.close()
            break

        if len(maps) > 10: # Limit the number of maps to prevent system freeze
            print("Created too many maps, stopping.")
            break

except MemoryError as e:
    print(f"\nCaught MemoryError: {e}")
    print(f"System likely ran out of virtual address space or physical memory/swap.")
except OSError as e:
    print(f"\nCaught OSError: {e}")
    print(f"This could be due to system limits (e.g., commit charge on Windows, file descriptor limits).")
except Exception as e:
    print(f"\nCaught unexpected exception: {e}")

print(f"\nFinished. Successfully created {len(maps)} memory maps.")
print("The system might be in an unstable state. Close other applications or reboot.")

# Keep the script running so maps remain allocated until manually interrupted
# or until the OS cleans them up on exit.
try:
    input("Press Enter to attempt to close maps and exit...")
finally:
    for m in maps:
        try:
            m.close()
        except Exception as e:
            print(f"Error closing map: {e}")
    print("Maps closed.")

Execution and Observation:
This Python script uses mmap to request large chunks of virtual memory. Depending on available RAM, swap, and virtual address space limits, this can lead to:

• Virtual Address Space Exhaustion: On 32-bit systems or if many large mappings are created, the process might run out of its 2^32 or 2^48 byte virtual address space.
• Commit Charge Exhaustion (Windows): The system might refuse the mmap request if the commit limit is reached.
• Physical Memory/Swap Exhaustion: If the mapped memory is actually used (e.g., by writing to it), it will consume physical RAM or swap space, potentially triggering the OOM Killer.

4.3) Process Limits (ulimit)

System administrators can set per-process resource limits using ulimit (on Unix-like systems) or similar mechanisms in systemd service units.

Setting a Limit (Bash):

# Set the maximum virtual memory size for the current shell and its children to 1GB
# The argument is in KB. 1GB = 1024 * 1024 KB.
ulimit -v 1048576

# Check current limits
ulimit -a

Demonstration:
Run the C program from Example 4.1 within a shell where ulimit -v has been set to a low value (e.g., 1GB). The program will likely fail with malloc returning NULL much sooner, and the error message might indicate a "Resource temporarily unavailable" or similar, potentially before the kernel's OOM Killer is invoked, as the malloc library or mmap system call respects the ulimit.

Systemd Service Unit Example:
For services managed by systemd, memory limits can be set in the .service file:

[Service]
# Limit virtual memory size to 2GB
LimitAS=2G
# Limit resident set size (physical memory) to 1GB
LimitRSS=1G
# Other limits like LimitDATA, LimitMEMLOCK can also be set.

4.4) Packet Field Analysis (Conceptual)

While network packet processing itself doesn't directly consume RAM in the same way as malloc, inefficient or malicious packet handling can indirectly lead to memory exhaustion. Consider a server application that receives network data.

Example: Vulnerable HTTP Server Buffer Handling

Imagine a simplified HTTP request parser that allocates a buffer based on a Content-Length header:

// Simplified C pseudo-code
char *handle_http_request(network_socket sock) {
    char header_buffer[1024];
    ssize_t bytes_read = recv(sock, header_buffer, sizeof(header_buffer) - 1, 0);
    header_buffer[bytes_read] = '\0';

    char *content_length_str = find_header(header_buffer, "Content-Length");
    if (content_length_str) {
        long content_length = strtol(content_length_str, NULL, 10);

        // *** VULNERABILITY ***
        // If content_length is excessively large (e.g., 4GB or more),
        // and the system's malloc/mmap can satisfy it, it will allocate
        // a huge buffer.
        if (content_length > 0 && content_length < MAX_EXPECTED_SIZE) { // MAX_EXPECTED_SIZE is too large!
            char *body_buffer = malloc(content_length + 1); // +1 for null terminator
            if (body_buffer == NULL) {
                fprintf(stderr, "Failed to allocate body buffer.\n");
                // Handle allocation failure gracefully
                return NULL;
            }
            // Read the body
            ssize_t body_bytes_read = read_full_data(sock, body_buffer, content_length);
            body_buffer[content_length] = '\0';

            // Process body_buffer...
            return body_buffer; // Memory leak if not freed later!
        } else {
            // Handle invalid or excessively large content length
            send_error_response(sock, 413); // Payload Too Large
            return NULL;
        }
    }
    // ... handle other headers ...
    return NULL;
}

Attack Scenario (Denial of Service via Memory Exhaustion):
An attacker could send a request with a Content-Length header specifying an enormous value (e.g., Content-Length: 4000000000). If the server's parsing logic doesn't adequately validate this value against system limits or reasonable application constraints, it might attempt to allocate gigabytes of memory. If multiple such requests are processed concurrently, the system could quickly exhaust its physical RAM and swap space, leading to an OOM condition and the OOM Killer terminating server processes.

Packet Snippet (Conceptual):

POST /upload HTTP/1.1
Host: vulnerable.server.com
Content-Length: 4000000000  <-- Maliciously large value
Content-Type: application/octet-stream

[... 4GB of binary data ...]

5) Common Pitfalls and Debugging Clues

5.1) Misinterpreting OOM Messages

• "Cannot allocate memory" / ENOMEM: This is a generic system error indicating a failed memory allocation request. It could be due to:
  • Physical RAM exhaustion.
  • Swap space exhaustion.
  • Virtual address space exhaustion (less common on 64-bit, but possible with many large mappings).
  • Hitting per-process limits (ulimit, cgroups).
  • Hitting system-wide commit limits (especially on Windows).
• dmesg / syslog OOM Killer Entries: These are definitive signs that the kernel invoked the OOM Killer. The log message will explicitly state "Out of memory: Kill process X (name) score Y". Analyze the logged process and its score.
• Application-Specific Errors:
  • Java: java.lang.OutOfMemoryError: Java heap space (Heap exhaustion), java.lang.OutOfMemoryError: Requested array size exceeds VM limit (Array allocation failure).
  • C++: std::bad_alloc exception thrown by new or new[].
  • .NET: System.OutOfMemoryException.
    These indicate memory exhaustion within the managed runtime or C++ exception handling, often triggered by underlying virtual memory allocation failures.

5.2) Debugging Strategies

1. Monitor System Resources:

   • Linux: top, htop (real-time process view, memory usage, load), vmstat (virtual memory stats, paging, swap activity), free (RAM and swap usage), sar (historical system activity), smem (more detailed memory reporting, including PSS - Proportional Set Size).
   • Windows: Task Manager (Performance tab, Memory section), Resource Monitor (Memory tab).
   • macOS: Activity Monitor.

2. Analyze Kernel Logs:

   • Linux: dmesg, /var/log/syslog, /var/log/messages. Look for OOM Killer messages, page fault errors, and memory reclamation warnings.
   • Windows: Event Viewer (System log).

3. Trace Memory Allocations:

   • Linux:
     • valgrind --tool=massif: A powerful heap profiler. It tracks memory allocations over time, showing peak usage and allocation hotspots.
     • strace -p <pid>: Traces system calls for a process. Look for mmap, brk, sbrk, munmap, mremap calls and their return values.
     • ltrace -p <pid>: Traces library calls. Useful for malloc, calloc, realloc, free.
     • perf mem: Advanced performance analysis for memory events.
   • Application Profilers: Java profilers (JProfiler, VisualVM, YourKit), .NET profilers (dotTrace, Visual Studio Profiler), Python profilers (memory_profiler, objgraph).
   • AddressSanitizer (ASan): Compile-time instrumentation (e.g., with GCC/Clang -fsanitize=address) to detect memory errors like leaks, use-after-free, buffer overflows.

4. Examine Process Limits:

   • Linux: ulimit -a for the shell, or inspect /proc/<pid>/limits for running processes.
   • Systemd: Check the service unit file for LimitAS, LimitRSS, etc.

5. Check cgroup Limits:

   • Linux: Inspect files in /sys/fs/cgroup/memory/<cgroup_path>/ (cgroup v1) or /sys/fs/cgroup/<cgroup_path>/memory.max (cgroup v2).

6. Review Application Code:

   • Memory Leaks: Unfreed malloc/new calls, objects not garbage collected, growing data structures without limits.
   • Unbounded Growth: Data structures (lists, maps, caches) that grow indefinitely without an eviction policy.
   • Large Static/Global Data: Excessive memory allocated at program startup.
   • Inefficient Buffering: Reading more data than necessary, or holding onto large buffers longer than required.
   • Fork Bombs: Recursive fork() calls that rapidly create processes, potentially leading to resource exhaustion.

5.3) Thrashing

Excessive paging (constant swapping of pages between RAM and disk) is known as thrashing. This occurs when the total memory required by active processes (the "working set") significantly exceeds available physical RAM.

Symptoms:

• Extremely high disk I/O activity (disk activity light constantly on, high IOPS).
• System becomes extremely unresponsive, slow to the point of being unusable.
• High CPU usage, but little actual productive work is done (CPU is often waiting for I/O).

Debugging:

• vmstat Output: High si (swap-in) and so (swap-out) values, along with high wa (I/O wait) CPU time, are

Source

• Wikipedia page: https://en.wikipedia.org/wiki/Out_of_memory
• Wikipedia API endpoint: https://en.wikipedia.org/w/api.php
• AI enriched at: 2026-03-30T23:24:48.765Z