S-0203 - Supplemental 203 - RF Protocol Hunting Workflow

Author: Patrick Luan de Mattos
Category Path: sdr-hacking
Audience Level: Advanced
Generated at: 2026-04-02T23:41:08.686Z

Supplemental Chapter: RF Protocol Hunting Workflow

1) Position of this Supplemental Chapter in the Advanced SDR Roadmap

This supplemental chapter builds upon foundational knowledge in Software Defined Radio (SDR) and Radio Frequency (RF) security. It is positioned within an advanced roadmap, assuming the reader is already familiar with:

• SDR Fundamentals: Basic signal processing concepts (sampling, FFT, demodulation), common modulation schemes (ASK, FSK, PSK), and the operation of SDR hardware.
• RF Security Principles: Understanding of RF vulnerabilities, spectrum awareness, and the importance of protocol analysis for security assessments.
• GNU Radio Basics: Familiarity with the GNU Radio Companion (GRC) and basic block usage.
• Python for SDR: Proficiency in Python for scripting SDR tasks, data manipulation, and basic signal analysis.

This chapter focuses on the systematic process of reverse-engineering unknown RF protocols, moving from raw signal observation to the development of a probable framing model. It bridges the gap between passive observation and active deciphering, a critical skill for both defensive analysis and ethical offensive security research.

2) Deep Conceptual Explanation

RF protocol hunting is the systematic process of reverse-engineering an unknown or proprietary wireless communication protocol. The ultimate goal is to understand the structure, encoding, and meaning of the transmitted data, enabling tasks such as:

• Security Auditing: Identifying vulnerabilities in wireless systems.
• Interoperability: Enabling communication with devices that lack documentation.
• Forensics: Analyzing captured RF traffic for incident response.
• Interference Mitigation: Understanding the nature of interfering signals.

This workflow emphasizes a disciplined, analytical approach, transforming raw spectral observations into a structured understanding of the protocol's framing. It's a detective-like process, where each clue gathered informs the next step.

The core of this workflow lies in identifying and exploiting patterns. Unlike wired protocols where headers and delimiters are often standardized, RF protocols can be highly varied. We must infer these structures from the signal's behavior, timing, and the resulting data.

The process can be broadly categorized into the following phases:

• Observation & Characterization: Initial capture and analysis of the raw RF signal.
• Pattern Identification: Searching for recurring structures and timing within the signal.
• Hypothesis Generation: Proposing potential framing mechanisms based on observed patterns.
• Model Refinement: Testing hypotheses and iteratively improving the framing model.

3) Architecture and Signal Reasoning

The typical architecture for RF protocol hunting involves a combination of SDR hardware, signal analysis software, and scripting capabilities.

Hardware:

• SDR Receiver: A wideband SDR (e.g., HackRF, LimeSDR, RTL-SDR) capable of capturing the relevant frequency band.
• Antenna: Appropriately chosen for the frequency of interest.

Software Stack:

• Signal Analysis Tools:
  • GNU Radio Companion (GRC): For building signal processing flowgraphs, visualizing signals, and performing initial demodulation.
  • Inspectrum/Universal Radio Hacker (URH): Dedicated tools for analyzing unknown protocols, including waterfall visualization, signal modulation analysis, and bit pattern extraction.
  • SDR# (SDRSharp) / GQRX: For initial spectrum monitoring and signal identification.
• Scripting Language:
  • Python: For automating analysis, decoding, and building custom tools.

Signal Reasoning Steps:

1. Spectrum Monitoring and Signal Identification:

   • Use an SDR receiver and visualization software (e.g., GQRX, SDR#) to identify the target signal within the spectrum.
   • Observe its bandwidth, center frequency, and general activity.
   • Note any periodic emissions or bursts.

2. Signal Capture:

   • Use the SDR to capture a significant duration of the target signal. Aim for multiple transmissions to observe variations and repetitions.
   • Record the sample rate, center frequency, gain settings, and timestamps.

3. Waterfall Analysis (Initial Pass):

   • Load the captured IQ data into a tool like Inspectrum or URH.
   • Examine the waterfall display. Look for:
     • Consistent signal shapes: Do transmissions have a similar visual signature?
     • Timing and periodicity: Are there regular gaps between transmissions? Are there repeating patterns within a transmission?
     • Bursts vs. Continuous Waves: Is it a short, distinct burst or a more continuous stream?
     • Modulation characteristics: Can you infer the modulation type (e.g., ASK, FSK, PSK) from the visual representation or by using built-in analysis tools?

4. Demodulation and Decoding (Initial Attempt):

   • Based on the initial waterfall analysis, attempt to demodulate the signal.
   • If it appears to be ASK (Amplitude Shift Keying), simple amplitude thresholding might suffice.
   • If it appears to be FSK (Frequency Shift Keying), frequency discriminators are needed.
   • If it appears to be PSK (Phase Shift Keying), phase demodulation is required.
   • Tools like URH excel at this, allowing you to apply different demodulation schemes and visualize the resulting bits.

5. Bitstream Analysis:

   • Once a bitstream is obtained, analyze it for patterns.
   • Bit Duration: Measure the duration of '1's and '0's. Are they constant? Do they vary? This hints at encoding schemes like Manchester or NRZ.
   • Synchronization/Preamble: Look for repeating patterns at the beginning of transmissions. These are often used for clock recovery and packet synchronization.
   • Data Structure: Identify repeating byte sequences, potential start/end delimiters, or fixed-length fields.
   • Parity/Checksums: Look for patterns that might indicate error detection mechanisms.

6. Hypothesis Formulation (Framing Model):

   • Based on the identified patterns, propose a hypothetical framing model. This model will describe:
     • Preamble: The synchronization sequence.
     • Start Delimiter: A unique pattern marking the beginning of the payload.
     • Payload Length: How is the length of the data indicated (e.g., fixed length, length field)?
     • Data Fields: Are there identifiable fields within the payload (e.g., command, address, data)?
     • End Delimiter: A pattern marking the end of the payload.
     • Interframe Gap (IFG): The silence between transmissions.

7. Model Refinement and Validation:

   • Use the hypothesized framing model to parse new captured data.
   • If the parsing is consistent and produces meaningful data (even if unintelligible at first), the model is likely on the right track.
   • If parsing fails or produces garbage, revisit the bitstream analysis and refine the hypothesis.
   • Consider edge cases: What happens if a bit is missed or flipped? How does the protocol handle this?

4) Python Examples When Applicable

Python is invaluable for automating tasks and performing detailed analysis on extracted bitstreams.

Example: Basic Bitstream Analysis Script

This script assumes you have a raw bitstream (a list or array of 0s and 1s) and performs basic analysis like calculating bit duration and identifying potential sync patterns.

import numpy as np
from collections import Counter

def analyze_bitstream(bitstream, sample_rate, min_duration_threshold=0.00001):
    """
    Analyzes a raw bitstream for basic characteristics.

    Args:
        bitstream (list or np.array): A sequence of 0s and 1s.
        sample_rate (float): The sample rate at which the bits were captured.
        min_duration_threshold (float): Minimum duration to consider a bit.
    """
    if not bitstream:
        print("Bitstream is empty.")
        return

    # Convert to numpy array for easier manipulation
    bitstream = np.array(bitstream)

    # Calculate bit durations
    # Find transitions (where bit changes)
    transitions = np.where(np.diff(bitstream))[0]
    if len(transitions) < 2:
        print("Not enough transitions to analyze durations.")
        return

    # Calculate lengths of consecutive identical bits
    durations = np.diff(np.insert(transitions, [0, len(transitions)], [-1, len(bitstream) - 1]))

    # Convert durations to seconds
    durations_sec = durations / sample_rate

    # Filter out very short durations that might be noise or sampling artifacts
    valid_durations = durations_sec[durations_sec >= min_duration_threshold]

    if not valid_durations.size:
        print("No valid bit durations found after filtering.")
        return

    # Analyze dominant bit durations (potential bit periods)
    duration_counts = Counter(np.round(valid_durations, 6)) # Round for grouping
    most_common_durations = duration_counts.most_common(5)

    print(f"--- Bitstream Analysis ---")
    print(f"Total bits: {len(bitstream)}")
    print(f"Number of transitions: {len(transitions)}")
    print(f"Dominant bit durations (seconds):")
    for duration, count in most_common_durations:
        print(f"  {duration:.6f}s ({count} occurrences)")

    # Basic sync pattern search (e.g., looking for a long sequence of 0s or 1s)
    max_consecutive_zeros = 0
    max_consecutive_ones = 0
    current_zeros = 0
    current_ones = 0

    for bit in bitstream:
        if bit == 0:
            current_zeros += 1
            current_ones = 0
        else:
            current_ones += 1
            current_zeros = 0
        max_consecutive_zeros = max(max_consecutive_zeros, current_zeros)
        max_consecutive_ones = max(max_consecutive_ones, current_ones)

    print(f"Longest sequence of 0s: {max_consecutive_zeros} bits")
    print(f"Longest sequence of 1s: {max_consecutive_ones} bits")

    # A very simple approach to find repeating sequences (e.g., potential preamble)
    # This is computationally intensive for long streams and might need optimization
    print("\nSearching for repeating 16-bit sequences (potential preamble candidates)...")
    window_size = 16
    sequences = {}
    for i in range(len(bitstream) - window_size + 1):
        seq = tuple(bitstream[i : i + window_size])
        sequences[seq] = sequences.get(seq, 0) + 1

    sorted_sequences = sorted(sequences.items(), key=lambda item: item[1], reverse=True)
    print("Most frequent 16-bit sequences:")
    for seq, count in sorted_sequences[:5]:
        print(f"  {''.join(map(str, seq))}: {count} occurrences")

# --- Example Usage ---
# In a real scenario, 'bitstream_data' would come from a demodulator.
# Let's simulate a simple bitstream with a preamble and some data.
# Preamble: 1010101010101010 (16 bits)
# Data:     01100001 01100010 (ASCII 'a', 'b')
simulated_preamble = [1, 0] * 8
simulated_data = [0, 1, 1, 0, 0, 0, 0, 1,  0, 1, 1, 0, 0, 0, 1, 0] # 'ab'
simulated_bitstream = simulated_preamble + simulated_data
simulated_sample_rate = 1e6 # Example sample rate, adjust as needed

analyze_bitstream(simulated_bitstream, simulated_sample_rate)

# Example with slightly varying bit durations (simulating real-world imperfections)
bit_0_duration = 0.0001 # seconds
bit_1_duration = 0.00011 # seconds
num_bits = 100
noisy_bitstream = []
for _ in range(num_bits):
    bit = np.random.randint(0, 2)
    if bit == 0:
        num_samples = int(bit_0_duration * simulated_sample_rate * (1 + np.random.uniform(-0.05, 0.05)))
    else:
        num_samples = int(bit_1_duration * simulated_sample_rate * (1 + np.random.uniform(-0.05, 0.05)))
    noisy_bitstream.extend([bit] * num_samples)

print("\n--- Analyzing Noisy Bitstream ---")
analyze_bitstream(noisy_bitstream, simulated_sample_rate)

Explanation:

• analyze_bitstream function: Takes a bitstream, sample rate, and a minimum duration threshold.
• Bit Durations: It calculates the time spent on each bit by finding transitions and measuring the length of consecutive identical bits. This is crucial for understanding the encoding (e.g., fixed-length bits vs. variable-length).
• Dominant Durations: It identifies the most common bit durations, which often correspond to the fundamental bit periods for '0' and '1'.
• Consecutive Sequences: It finds the longest sequences of '0's and '1's, which can hint at idle states or specific framing elements.
• Repeating Sequences: A basic search for repeating short sequences can help identify potential preambles or sync words. More advanced algorithms like Lempel-Ziv can be applied here for longer streams.

5) GNU Radio Examples When Applicable

GNU Radio is excellent for building the signal processing chain from raw IQ samples to a demodulated bitstream.

Example: Basic FSK Demodulation and Bit Extraction in GRC

This GRC flowgraph demonstrates how to demodulate a simple Frequency Shift Keying (FSK) signal and extract a bitstream. This is a common starting point for protocol hunting.

Flowgraph Description:

1. Source: UHD USRP Source (or OSMOSDR Source, File Source for recorded data) captures IQ samples.
2. Frequency Xlating FIR Filter: Shifts the signal to baseband and filters out unwanted noise.
3. Quadrature Demod: Demodulates the FSK signal. The output is proportional to the instantaneous frequency deviation.
4. Low Pass Filter: Smooths the output of the Quadrature Demod.
5. Binary Slicer: Converts the analog signal into a digital stream of 0s and 1s by comparing it against a threshold.
6. Clock Recovery MM: Recovers the clock from the noisy bitstream to resample at the correct moments, ensuring accurate bit detection.
7. Correlate Access Code Tag: (Optional but highly recommended) This block searches for a known synchronization preamble (access code). Once found, it aligns the output bitstream.
8. Packet Encoder/Decoder: (For more structured protocols) If you suspect a specific packet structure, these blocks can help. For raw protocol hunting, we often bypass these initially and analyze the raw bits.
9. Sink: File Sink or UDP Sink to export the resulting bitstream for further Python analysis, or Waterland Sink for visualization.

Visual Example (Conceptual GRC Flowgraph):

+---------------------+     +---------------------+     +-----------------+     +------------------+
| UHD USRP Source     | --> | Freq Xlating FIR    | --> | Quadrature Demod| --> | Low Pass Filter  |
| (IQ Samples)        |     | Filter              |     | (FSK Output)    |     |                  |
+---------------------+     +---------------------+     +-----------------+     +------------------+
                                                                                              |
                                                                                              v
+-------------------+     +----------------------+     +--------------------------+     +----------------+
| Binary Slicer     | --> | Clock Recovery MM    | --> | Correlate Access Code Tag| --> | File Sink      |
| (0s and 1s)       |     | (Bitstream Sync)     |     | (Optional Preamble Detect)|     | (Extracted Bits)|
+-------------------+     +----------------------+     +--------------------------+     +----------------+

GNU Radio Block Parameters (Illustrative):

• UHD USRP Source:
  • Center Frequency: e.g., 433.92e6
  • Sample Rate: e.g., 2e6
  • Gain: e.g., 30
• Frequency Xlating FIR Filter:
  • Decimation: e.g., 10 (reduces sample rate to 2e5)
  • Bandwidth: e.g., 50e3 (typical FSK bandwidth)
  • Taps: Generated by fir_design.py or similar.
• Quadrature Demod:
  • Gain: Needs tuning. For FSK, this is related to the frequency deviation.
• Low Pass Filter:
  • Cutoff Frequency: e.g., 50e3
  • Transition Width: e.g., 10e3
  • Decimation: 1 (if previous decimation was not enough)
• Binary Slicer:
  • Threshold: Often 0.0 for centered signals, or tuned.
• Clock Recovery MM:
  • Omega: Estimated bit rate (e.g., 1 / bit_duration).
  • Gain Omega, Gain Mu: Tuning parameters.
  • Max Bits: Maximum number of bits to track.
• Correlate Access Code Tag:
  • Access Code: A hexadecimal string representing the known preamble (e.g., 0x10101010).
  • Tag: packet_len or packet_start

How to Use for Protocol Hunting:

1. Start Broad: Begin with a wide bandwidth and lower gain to identify the signal.
2. Isolate: Narrow down the frequency and increase gain to get a clean signal.
3. Guess Modulation: Based on visual cues (waterfall, constellation), pick a likely demodulator (ASK, FSK, PSK).
4. Tune: Adjust filter bandwidths, demodulator gains, and slicer thresholds.
5. Extract Bits: Use a File Sink to save the output of the Binary Slicer or Clock Recovery MM.
6. Analyze in Python: Load the saved bitstream into a Python script for deeper analysis as shown in the previous section.
7. Iterate: If the bitstream doesn't show clear patterns, revisit the demodulation and tuning steps. Try different modulation schemes.

Advanced GRC Technique: Packetization

Once a preamble and a basic bit structure are identified, you can use GNU Radio's packet handling blocks to automatically segment transmissions.

• Packet Encoder (in Transmitter): Adds preamble, sync word, length, and CRC.
• Packet Decoder (in Receiver): Detects preamble/sync, extracts length, checks CRC, and outputs the payload.

For hunting, you'd typically use Correlate Access Code Tag to find the start and then process the subsequent bits as a "packet." You might then look for a length field or assume a fixed length.

6) Visual Examples When Applicable

Visualizations are critical for understanding RF signals and their patterns.

Waterfall Display:

A waterfall shows signal strength over time and frequency. For protocol hunting, it's the primary tool for initial observation.

Frequency (e.g., MHz)
^
|     ...     ######
|     ...     ######
|     ...################...
|     ...#    # ###### #...
|     ...#    # ###### #...
|     ...#    # ###### #...
+--------------------------------> Time

• Interpretation: Horizontal streaks represent continuous signals. Vertical "blips" or distinct shapes indicate bursts or packets. The color intensity represents signal strength. Repeating patterns in the waterfall can suggest structured transmissions.

Bit Patterns (Raw):

This represents a sequence of binary digits as captured after demodulation and slicing.

10101010 11001010 01011010 00001111

• Interpretation: This raw stream is what we analyze for structure. We look for repeating sequences, fixed-length blocks, and transitions.

Byte Layouts:

Once bits are grouped into bytes (8 bits), we can look for structure in terms of bytes.

Byte 0: 10101010  (Hex: 0xAA)
Byte 1: 11001010  (Hex: 0xCA)
Byte 2: 01011010  (Hex: 0x5A)
Byte 3: 00001111  (Hex: 0x0F)

• Interpretation: We can convert these to hexadecimal or decimal for easier comparison. Repeating byte sequences are strong indicators of protocol elements (e.g., addresses, commands).

ASCII Signal Diagrams (Conceptual):

This is a high-level representation of how data might be encoded in time.

Example 1: Simple ASK (On-Off Keying)

Bit:   1   0   1   1   0   0   1   0
Time:  ---- ---- ---- ---- ---- ---- ---- ----
Signal:___   _   ___   ___   _   _   ___   _
       (High) (Low) (High) (High) (Low) (Low) (High) (Low)

• Interpretation: High amplitude = '1', Low amplitude = '0'.

Example 2: FSK (Frequency Shift Keying)

Bit:   1   0   1   1   0   0   1   0
Time:  ---- ---- ---- ---- ---- ---- ---- ----
Signal:~~~~ ~~~~ ~~~~ ~~~~ ~~~~ ~~~~ ~~~~ ~~~~
       (Freq A) (Freq B) (Freq A) (Freq A) (Freq B) (Freq B) (Freq A) (Freq B)

• Interpretation: Different frequencies represent '1' and '0'.

Example 3: Manchester Encoding

Manchester encoding has a transition in the middle of each bit period.

Bit:   1   0   1   0
Time:  ---- ---- ---- ----
Signal:__/\___/\___/\___/\_
       (Low-High) (High-Low) (Low-High) (High-Low)

• Interpretation: A transition from low to high in the middle represents '1'. A transition from high to low represents '0' (or vice-versa depending on the variant). This is self-clocking.

Flow Illustration: From Raw Waterfall to Framing Model

graph TD
    A[Raw Waterfall Capture] --> B{Observe Signal Activity};
    B --> C[Identify Signal Parameters (Freq, BW, etc.)];
    C --> D[Capture IQ Data];
    D --> E{Analyze Waterfall};
    E --> F[Hypothesize Modulation (ASK, FSK, PSK?)];
    F --> G[Demodulate Signal (GRC/URH)];
    G --> H[Extract Raw Bitstream];
    H --> I{Analyze Bitstream Patterns};
    I --> J[Look for: Bit Durations, Repetitions, Long Sequences];
    J --> K[Hypothesize Framing Elements: Preamble, Sync Word, Delimiters];
    K --> L[Formulate Probable Framing Model];
    L --> M[Validate Model with More Data];
    M --> N{Refine Model or Revisit Hypothesis};
    N -- If unsuccessful --> I;
    N -- If successful --> O[Protocol Understood];

• Interpretation: This diagram outlines the iterative workflow: observe, capture, demodulate, analyze bits, hypothesize framing, and validate.

7) Defensive and Offensive Implications and Troubleshooting

Defensive Implications:

• Vulnerability Detection: Understanding proprietary protocols allows defenders to identify potential weaknesses. For example, a lack of encryption, weak authentication, or predictable command structures.
• Threat Intelligence: Analyzing the RF protocols used by adversaries or threat actors can provide valuable intelligence about their tactics, techniques, and procedures (TTPs).
• Interference Analysis: Protocol knowledge helps distinguish legitimate traffic from malicious jamming or spoofing attempts.
• System Hardening: By understanding how a device communicates, security teams can implement more effective countermeasures, such as RF jamming detection, signal authentication, or protocol-specific intrusion detection systems.

Offensive Implications (Ethical Context):

• Exploitation of Weaknesses: Once a protocol's framing and data structure are understood, an attacker can craft malformed packets to trigger vulnerabilities, cause denial-of-service, or gain unauthorized access.
• Spoofing and Replay Attacks: Understanding the protocol allows for the creation of spoofed messages or the replaying of captured valid messages.
• Firmware Extraction: In some cases, understanding the communication protocol might be a step towards extracting firmware from devices that use RF for updates or configuration.

Troubleshooting Common Issues:

• Noisy Bitstream:
  • Problem: The extracted bitstream is full of errors, making pattern analysis impossible.
  • Troubleshooting:
    • Gain Settings: Too high gain can cause clipping and saturation; too low can result in a weak signal. Adjust gain carefully.
    • Filtering: Ensure appropriate filtering (e.g., matched filtering, bandpass filtering) is applied.
    • Antenna: Verify the antenna is suitable for the frequency band.
    • Environment: Move to a quieter RF environment or use a shielded enclosure.
    • Modulation/Demodulation Tuning: Revisit the demodulation settings. The Gain in Quadrature Demod or thresholds in Binary Slicer are critical.
• Incorrect Bit Timing:
  • Problem: The Clock Recovery MM block outputs a stream where bits are not properly aligned, or the bit rate is wrong.
  • Troubleshooting:
    • Omega (Bit Rate): This is the most crucial parameter. Estimate the bit rate from dominant bit durations found in Python analysis.
    • Gain Omega and Gain Mu: These control the loop's responsiveness. If the bit rate fluctuates, you might need to adjust these. Start with default values and tune.
    • Clock Drift: If the transmitter's clock drifts significantly, a simple clock recovery might struggle. Advanced techniques might be needed.
• No Clear Preamble/Sync Word:
  • Problem: The Correlate Access Code Tag block doesn't find the expected preamble, or the output is not synchronized.
  • Troubleshooting:
    • Access Code Mismatch: Double-check the hexadecimal representation of your hypothesized preamble.
    • Bit Errors in Preamble: Even a few errors in the preamble can prevent detection. Improve signal quality or consider preamble error correction if the protocol uses it.
    • Preamble Not Found: The preamble might be different from what you assumed, or it might be absent entirely (some protocols use implicit timing).
    • Synchronization Drift: If the preamble is very short, clock drift can cause it to be missed.
• Garbled Payload:
  • Problem: After successful preamble detection, the payload data appears nonsensical.
  • Troubleshooting:
    • Data Encoding: The bits might be encoded (e.g., Manchester, NRZI, Gray code). Try different decoding schemes.
    • Byte Order (Endianness): Are bytes being assembled correctly (little-endian vs. big-endian)?
    • Packet Structure: The assumed packet structure (length field, fixed length, field order) might be wrong.
    • CRC/Checksum Errors: If CRC checks fail, the data is corrupted. This could be due to noise or a deliberate manipulation attempt.

8) Summary

RF protocol hunting is an essential skill for advanced SDR users and security professionals. It's a methodical process that begins with passive observation of the RF spectrum and culminates in a deep understanding of an unknown wireless protocol's framing. By systematically analyzing captured signals, identifying recurring patterns in bitstreams, and formulating/refining hypotheses about the protocol's structure, analysts can move from raw data to actionable insights.

The workflow emphasizes the use of powerful tools like GNU Radio for signal processing and Python for detailed bitstream analysis. Visual aids such as waterfall displays and bit pattern representations are crucial for interpreting the data. While this process can be challenging, a disciplined, iterative approach, combined with a solid understanding of modulation and encoding techniques, is key to success.

The ability to reverse-engineer RF protocols has significant implications for both defensive security (identifying vulnerabilities, understanding threats) and ethical offensive security research (exploring potential attack vectors). Effective troubleshooting involves a deep dive into signal quality, demodulation parameters, and the accuracy of hypothesized framing elements. This chapter has provided a framework for embarking on this complex but rewarding endeavor.

This chapter is educational, lab-oriented, and constrained to lawful, defensive, and controlled research contexts.