Reverse Engineering a Malware Sample: A Practical Introduction

Reverse engineering a binary means recovering intent from a compiled artifact. The source is gone, the symbols are usually stripped, and the person who wrote it was not trying to make your life easier. Everything you learn has to be inferred from what the file imports, what it contains, and how its code is wired together.

The goal is almost never to understand every instruction. It is to answer a narrow question quickly: is this file dangerous, and if so, what does it do to a machine? The pipeline below is ordered by information gained per minute spent, which is the only ordering that matters when you have forty samples in the queue.

1. Identify the file before anything else

The extension is a claim, not a fact. Malware authors have never felt especially bound by naming conventions. Start from the actual format, read out of the magic bytes and headers: PE for Windows, ELF for Linux, Mach-O for macOS. Architecture (x86, x86-64, ARM) decides your disassembler settings.

Two signals matter right away. Size that is wildly out of proportion with the claimed purpose suggests a bundled payload or an embedded installer. A 40 MB "invoice viewer" is not a subtle animal. High entropy in the code sections suggests packing or encryption, which means the real code only exists after it unpacks itself at runtime.

2. Read the import table, the highest signal per second

A program cannot touch the filesystem, the network, the registry, or another process without asking the operating system. Those requests are declared in the import table, and reading it is the fastest way to bound what a binary can possibly do.

Imports are far more useful grouped by capability than read alphabetically:

The discipline that separates useful analysis from noise: presence is not intent. CreateFileA appears in Notepad. What matters is combination. A binary that imports file enumeration, crypto and network I/O together has the shape of ransomware. Any one of those on its own describes a text editor with delusions of grandeur.

3. Extract strings, then filter them hard

Strings are the cheapest source of concrete indicators and by far the noisiest. A typical binary gives up several thousand. About twenty matter. Most of the rest are error messages from a C++ runtime that nobody has thought about in fifteen years.

The ones worth your attention:

• Network indicators. Domains, IPs, URL paths, User-Agent values. These become detection rules directly, with no further work.
• Filesystem and registry paths, especially anything under Run, AppData, Temp or ProgramData.
• Command lines. cmd.exe /c, powershell -enc, vssadmin delete shadows. That last one is behaviour stated in plain text, and it is rarely an accident.
• Build artifacts. PDB paths, framework markers, error messages. Occasionally a developer leaves their username in a debug path, which is always a good day.

Absence tells you something too. A large binary with almost no readable strings is announcing that its strings are encrypted and resolved at runtime, which is itself a signal.

4. Move to functions and control flow

Imports tell you what is possible. Functions tell you what is actually wired together. A disassembler recovers function boundaries, the call graph, and cross-references between code and data.

Three questions carry most of the value:

1. Which functions call the dangerous imports? Cross-reference backwards from CreateRemoteThread and you land in the injection routine directly, without reading a single unrelated function.
2. What runs before main? TLS callbacks and entry-point stubs execute before the visible program logic. It is a traditional hiding place for anti-analysis checks, precisely because most people start reading at main.
3. Which functions are referenced everywhere? A small routine called from forty places is usually a utility. Very often it is the string decryption routine, which is the key that unlocks everything else in the sample.

5. Decompile only the functions that matter

A decompiler such as Ghidra reconstructs C-like pseudocode from machine code. Treat it as a reading aid rather than source recovery. Variable names are invented, types are guessed, and anything built around a large switch statement tends to come out looking like a crime scene.

On the handful of functions you have already identified as interesting, it is decisive. As a first step it is a trap. Ghidra will cheerfully hand you four thousand decompiled functions and your afternoon will not survive the offer. Let the import table and the cross-references pick your targets, then decompile those.

6. Know when static analysis has stopped paying

Static analysis is finished when you can state what the binary is capable of. It will never tell you what it actually did on a given run. Switch to dynamic analysis when:

• The sample is packed and the real payload only ever exists in memory.
• Strings are decrypted at runtime and you need the plaintext values.
• You need the real C2 endpoint, which is often assembled at runtime from fragments scattered across the binary, mostly to annoy you.
• Behaviour depends on environment checks you would rather observe than reconstruct by hand.

The workflow, condensed

Identify the format and check entropy. Read the import table and group it by capability. Extract strings and keep only the concrete indicators. Follow cross-references from the dangerous imports to the functions that use them. Decompile those. Then decide whether the remaining questions need the sample to actually run.

The mistake almost everyone makes while learning this is reading linearly instead of following signal. Every stage above exists to narrow the next one. If you find yourself scrolling through decompiled output hoping something jumps out, you skipped a step.