A Review of Common Practices for Malware Unpacking

---

2.1 Packing purpose

Packing is a technique that is widely used to compress software to reduce in size, often it improves performance, protects intellectual properties, and prevents tamper. One of the main purposes of packing software is to reduce the size which makes the distribution and the download of the executable faster compared to its original size, especially for big software. This is really useful, for software distributed over the internet, which makes it convenient for users.

Another benefit of packing is to protect intellectual properties by hindering third parties from reverse engineering and decompiling the code. This can prevent competitors from stealing the code or creating a similar product. Various techniques of packing can be implemented in order to protect the software.

Malicious users use the packing technique to avoid detection mechanisms by detection software such as Firewalls and Intrusion Detections Systems (IDS) by compressing or encrypting the malicious code. This makes it difficult to be detected by the protection mechanisms.

As further protection packers can implement anti-debugging and anti-tampering techniques on the malware. Malware packed in such mechanisms makes really difficult to be analyzed. Anti-debugging technique detects and tries to prevent debugging attempts by checking for example breakpoint which is a clear indication of debugging. Anti-tampering mechanisms on the other hand tried to detect changes in the code and a possible way can be by computing checksums and digital signatures. If the mentioned actions are detected by the malware, it may terminate or alter its behavior as a protection mechanism to mislead malware analysts.

Finally, the main purpose of malware packers is to increase the chances of successfully infecting a target system, avoiding detection by security measures, making the malware hard to analyze in order to understand how internally it works and mitigate any infection.

2.2 How packers work

During the packing process, the packing software takes as input the original executable file (PE) and generated a new packed file. During the packing process, the packer can encrypt or compress the original PE file or a combination of both. This newly packed file contains the original PE file and the code to unpack it during the execution usually called “stub” which is the new entry point of the executable. The stub is usually designated to unencrypt or decompress the original PE but, in some cases, it can also encrypt and compress back the code.

The following image shows an example of how the executable file changes before and after the packing process:

The newly packed PE file has a different structure than the original PE, trying to analyze this file will result in an unsuccessful attempt as the real code is not extracted yet.

The packer can modify the PE header or remove it completely in an attempt to evade detections by security systems or malware analysts. The packer can also add some additional code as anti-debugging and anti-tampering technique. When the packed PE is executed the first entry point is the stub which extracts the packed code to be unpacked. Once the extractions finishes, the control is passed to the unpacked code to be executed. The following image shows the steps involved during this process:

Understanding how the packing process works and how the PE section can be modified is as important as to keep the system up to date, use strong passwords, and avoid getting files from untrusted sources to avoid getting infected.

The structure of a packed PE file is complex as it is usually customized by malware writers to obfuscate the malicious code. During the analysis of a packer the main objective is to find the Original Entry Point (OEP), which is the starting point of the original malicious code. Identifying the OEP is not always trivial as malware authors can implement techniques to make this step difficult.

Overall, the process of packing malware is constantly evolving, and malware authors are continually developing new techniques to evade detection. As security researchers, it's crucial to keep up with the latest malware-packing methods to detect and prevent.

4.1 Introduction

Malware analysis is a critical aspect of the cybersecurity field as it helps to identify, understand, and mitigate malicious software. Malware can be designed to cause various types of harm, such as stealing sensitive data, damaging systems, or hijacking resources for nefarious purposes. To counter such threats, security professionals use different methods and tools to analyze malware and gain insights into its behavior.

There are three primary approaches to analyzing malware: static analysis, dynamic analysis, and hybrid analysis. Each technique has its own strengths and weaknesses, and security analysts choose the appropriate method based on their objectives and the type of malware they are dealing with.

Static analysis involves analyzing the malware's code without actually executing it. One of the main advantages of static analysis is that it is a non-intrusive method, which means that the malware is not actually run on the system, reducing the risk of contamination and being discovered by the malware writer that his malware is being analyzed. This also prevents future patches by the malware writer in order to make the malware harder to be detected.

Additionally, static analysis can be performed quickly, making it an effective method for identifying known malware. For such purposes, various tools and techniques can be used such as disassembling the executable code, examining metadata, analyzing strings, and checking the digital signature of the file.

Dynamic analysis involves running the malware in a controlled environment, such as a virtual machine or sandbox, to observe its behavior. This technique is useful for detecting malware that is designed to evade detection or that requires specific environmental conditions to activate. Dynamic analysis techniques include monitoring system calls, network traffic, and file system changes.

The dynamic analysis provides detailed insights into the behavior of the malware, allowing analysts to identify the type of malware, its purpose, and the systems it interacts with. However, dynamic analysis can be a time-consuming process, and there is a risk of contamination if the malware is not properly contained in the analysis environment.

During this step, it is important to take all the precautions to safeguard the malware. It is important to execute it with due diligence and make sure the testing environment is isolated. Also, it is important to take a safe copy of the malware or execute continuous snapshots of the testing environment in case the malware deletes itself, overwrites portions of code, or implements additional defence mechanism to void being analyzed.

The hybrid analysis, a combination both static and dynamic analysis techniques, provides a comprehensive understanding of the malware. This technique involves first performing a static analysis to identify the malware's characteristics, followed by a dynamic analysis to observe its behaviour. Hybrid analysis can be an effective method for detecting advanced malware that is designed to evade detection or that uses sophisticated techniques to obfuscate its code.

The initial static analysis and the discovery of information in this phase are really helpful during the dynamic analysis phase. The static analysis gives an overview and an idea of the malware and what to expect from it, static analysis can also reduce the time consumed by the dynamic analysis phase in case some key information are found.

4.2 Static Analysis

One effective method of analyzing malware is through static analysis, which involves examining the code and structure of a malware sample without executing it. One of the primary techniques used in static analysis is disassembly, also known as reverse engineering. Disassembly involves converting the binary code of a program into human-readable assembly code that can be analyzed and understood.

During static analysis, it is possible to extract a wide range of information about malware, including the strings contained in the malware, the system calls used, and the entropy if the malware was packed. This information can help analysts understand the intent of the malware, its capabilities, and potential weaknesses that can be exploited.

“Basic static analysis consists of examining the executable file without viewing the actual instructions. Basic static analysis can confirm whether a file is malicious, provide information about its functionality, and sometimes provide information that will allow you to produce simple network signatures. Basic static analysis is straightforward and can be quick, but it’s largely ineffective against sophisticated malware, and it can miss important behaviors ^[1]”.

In the event of packed malware, the aim at this stage is to get a general idea of what the malware might be, i.e., whether the malware is really packed, whether there are indicators to confirm this, and the type of packing it might implement.

The following packing indicator is taken from DetectItEasy (DIE). With DIE, it is possible to identify packers and retrieve useful information about the malware sample. For instance, when analyzing a malware sample, DIE can identify whether malware is packed or not. In addition, DIE can retrieve the version number of the packer, in this specific case as shown below is 3.93, which can be used to identify known vulnerabilities and weaknesses in the packer itself which is useful during the analysis phase.

A useful feature of DIE is the capability to obtain the number of sections from the packed file. This can provide valuable information on the internal structure of the malware and help to confirm the presence of the packer. In fact, from the malware sample analyzed, DIE found three sections containing UPX in the name.

To further confirm whether the malware is packed or not, it is possible to check the entropy of the file. Since packed files tend to have a high entropy due to the compression and encryption techniques used, checking the entropy can help to confirm the presence of the packer. As shown in the picture there is a spike in the entropy graph reaching the value of 7 which we can consider enough high to say that the malware is packed.

Another advantage of static analysis is that it can be performed on multiple samples simultaneously, allowing for rapid identification of commonalities and patterns across different malware variants. This can help analysts identify and understand larger malware families and the techniques they use to evade detection. This can be performed from the command line for example with the command "strings" and grepping key strings such as "host", "IP" or "socket" in search of an external connection to the command and control or via Yara rules that can identify patterns and advanced metrics.

The subset of findings presented in the image above reveals important information about the behaviour of the analysed malware. One of the notable indicators is the string 'Writing %s section to 0x%p\r\n', which suggests that the unpacked code is written into memory due to the presence of packing. Additionally, the strings 'Getting thread context' and 'Setting thread context' imply that the malware employs threads to perform certain operations. The occurrence of the strings 'Creating process' and 'GetCurrentProcesID' is indicative of the creation of new processes by the malware.

Moreover, the string 'IsDebuggerPreset' raises the possibility that the malware attempts to detect whether it is being analysed and may modify its behaviour accordingly. This underscores the importance of using anti-analysis techniques by malware authors to evade detection and analysis by security researchers. This gives an idea of what the malware might do and how it might behave. This is very useful if, after the first static analysis phase, we proceed to a dynamic analysis, so that we already know what to expect from the malware.

However, there are also limitations to static analysis. For instance, some malware variants use advanced obfuscation techniques to make their code difficult to analyse, such as using anti-disassembly techniques or encrypting critical sections of their code. Additionally, static analysis alone cannot provide a complete understanding of how the malware operates, as it cannot account for the behaviour of the malware in different environments or the impact of the malware on the system.

4.3 Dynamic analysis

Malware has become a significant threat to computer systems and networks, and its proliferation is increasing day by day. To counter this, various techniques have been developed to analyze malware and to understand its behavior, the operations it performs, and its objectives. For such purposes, dynamic analysis is used. It is the process of observing the behavior of a piece of malicious software and understanding its purpose.

The primary goal of dynamic analysis is to understand the behavior of the malware when it is executed. Dynamic analysis can help in identifying the malware's objectives, capabilities, and methods of propagation. In the case of packed malware dynamic analysis can help to find the critical pieces of code such as the stub which is the code used to extract the packed section or the tail jump, a jump instruction pointing to the original malicious code.

Dynamic analysis can also help in identifying the presence of any anti-analysis or evasion techniques used by the malware, in the event of packed malware for example it can monitor the read-and-write system calls of unpacked code and identify changes in the memory sections. It can also determine the malware's impact on the system or network, such as changes in the registry or file system, network communication, and system resource utilization.

The following image is the representation of the memory segment before and after executing the packed malware and it clearly indicates the changes it does and the extraction in memory of the packed code.

In particular with dynamic analysis, it is possible to monitor system calls, by monitoring them it is possible to have a brief understanding, in some cases detailed and precise, of what operations the malware is performing and what can be the purpose of such. For example, if we find the socket() system call, we can assume that the malware most likely communicates with a remote host that could be the command and control (C&C). The purpose of such communication could be to receive remote commands to perform operations, exfiltrate data and send it to the remote host, or even update the malware.

Another advantage of running malware is that one can see the changes it makes to the system. Malware can make different changes to the system such as changes to the file system, registry, and system resources. These changes can, for instance, be intended to persist in the event of a system restart, to hide from analysts or security software, or create DLL files.

With dynamic analysis, it is also possible to analyze multi-threaded or multi-process malware. The creation and execution of these threads or processes are not immediate via static analysis. In fact, it is easier to run the malware, and then dump the threads or the processes it created. Even if most of this information could be found by static analysis, it involves very long analysis times that can be reduced by dynamic analysis, especially in the case of packed malware where the original code is obfuscated.

Dynamic analysis has its limitations, mainly due to the complexity of modern malware. Malware authors implement a variety of techniques to evade detection, such as anti-debugging and anti-virtualization techniques. They also use advanced encryption and obfuscation methods to make the malware more difficult to analyze and they can also be designed to detect the presence of a sandbox or virtual environment. Malware often alters its behavior accordingly in presence of an analysis environment or debuggers leading the analyst in the wrong direction.

4.4 Hybrid (sandbox) analysis

With dynamic analysis, it is possible to analyze the behavior of malware in a controlled environment. The main goal of dynamic analysis is to observe the behavior of malware when executed which otherwise could not be understood. Dynamic analysis can help in identifying the malware's intent, capabilities, and methods of propagation. For example, it can also identify the malware's impact on the system or network, such as changes in the registry or file system, creation of files, creation of scheduled tasks in order to survive a reboot, network communication, and system resource utilization. Dynamic analysis can also help in identifying the presence of any anti-analysis or evasion techniques used by the malware. This is especially useful in presence of thoroughly packed malware or a new packing technique in which unpacking is time-consuming. In this way, it is possible to understand up to a certain degree the analyzed malware and implement a provisional countermeasure.

The hybrid analysis differs from dynamic analysis in its scope. It combines both static and dynamic analysis. Further, it includes an analysis of external elements in which the malware may operate such as the network. Hybrid analysis does not limit to executing the malware, but it also creates a virtualized environment, even with several machines in order to allow the malware to perform the operations it would typically execute in a real-world environment.

However, there are also some limitations to hybrid analysis. For example, some malware may be designed to detect the presence of a VM or sandbox by implementing checks on the number of cores, number of RAM, and other metrics to understand if it is running in a genuine or controlled environment and alter its behaviour accordingly.

5.1 Premise

This chapter will further introduce packers and give some in depth knowledge about internal structure of a Portable Executable (PE) file thave a good understanding of each unpacking steps performed later on. Additional it will present a detailed guide on how to carry out static andynamic analysis, highlighting their key differences as well as their pros and cons. The tools required for each step will be listed and explainein order to provide a comprehensive understanding. The environment setup will also be described so that it can be replicated and the reasoninbehind such a strategy can be understood. The main objective of this chapter is to demonstrate a possible method of unpacking UPX and MPRESpackers in detail and provide insights into the expected outcomes.

5.2 Popular packers

UPX and MPRESS are two popular packing commonly used in malware packing. UPX is an open-source executable packer that can compress executables and DLLs without damaging or changing the original code and it is completely self-contained. It uses a combination of multiple compression algorithms, including LZMA, Huffman coding, and Run Length Encoding (RLE), to compress the code. UPX supports a wide range of executable formats, such as Windows PE and Linux ELF. UPX is widely used by malware authors as it is easy to use and can quickly compress executables.

UPX is a popular packer in the malware development community due to its effectiveness in reducing the size of the executable and the availability of both encryption and compression. However, it is also well-known among security researchers and antivirus vendors, and many have developed signatures and heuristics to detect and analyze UPX-packed executables. As a result, malware developers have increasingly turned to more sophisticated packing algorithms, such as MPRESS, to evade detection.

On the other hand, the MPRESS packing is well known for its ability to hardly compress malware to make it harder to be detected and significantly reduce its size. It achieves this by replacing large sections of the original code with compressed and encrypted versions. The algorithm then adds the stub to compress/decompress the original code at runtime, making it difficult for static analysis tools to detect the true purpose of the malware.

MPRESS uses a variety of advanced techniques to make it difficult for analysts to reverse engineer the packed code. It uses a custom stub to pack and unpack the code, which makes it difficult for anti-virus software to detect and unpack the malware. The algorithm also uses anti-debugging techniques to prevent analysts from debugging the packed code or altering its execution in any way.

One of the main advantages of MPRESS is its speed and efficiency it has. The packing process is relatively fast, and the resulting packed file is highly compressed, which reduces the overall size of the file. This makes it easier for malware authors to distribute the packed file by being stealthier.

It is common among malware writers to create their own custom versions of packers. UPX is an open-source tool, which means that the source code is accessible and can modify it as needed. However, creating a custom version of a packer is not a trivial task and requires a good understanding of assembly language, binary file formats, and compression algorithms. In the case of MPESS is even harder as it is a proprietary packer and creating a custom version requires to reverse engineer it. By doing so malware writers will go against its license agreement.

Although a custom packer may not be as effective as the original packer in terms of compression ratio and speed, the main goal of malware writers is to make the malware less detectable possible and harder to be analyzed. Malware writers often introduce additional peace of code in order to change the malware signature to evade security systems or to deviate from the analyst analysis by introducing anti-analysis techniques.

5.3 Import address table (IAT)

It is very important to introduce the concept of Import Address Table (IAT) to understand all the steps of malware analysis later on. In particular to understand the process of extracting and creating the PE file containing the unpacked code from a packed file.

The Import Address Table (IAT) is a crucial component in PE (Portable Executable) files used on Windows-based systems. The IAT is responsible for storing pointers to functions that are imported from other DLLs (Dynamic-Link Libraries), which is an essential part of allowing a program to call functions that are not part of its own code. This is a fundamental aspect of modular and maintainable programming.

During the resolution stage of a typical Windows PE executable, the header of the executable contains essential metadata that informs the operating system about the symbols from external libraries that the executable relies upon. When the operating system's loader processes the executable, it loads the necessary libraries into memory if they are not already loaded. The loader then assigns the addresses of the imported symbols to specific memory locations as specified by the metadata within the executable.

To clarify the image below, the IMAGE_IMPORT_BY_NAME is a structure within the Import Address Table (IAT). It is used to store the import information for a specific function, including the function name and its corresponding hint value . The IMAGE_IMPORT_BY_NAME structure is employed when the import is performed using the name of the function, rather than its ordinal value, which are the two methods to import functions.

The Import Name Table (INT) stores the names of the imported functions from external DLLs. The Import Address Table (IAT) is responsible for holding the addresses of the imported functions after they have been resolved by the loader. It is an array of function pointers located within the PE file's data section. The IAT is populated by the loader with the actual addresses of the imported functions.

The purpose of the IMAGE_IMPORT_BY_NAME structure is to provide the necessary information for importing a function by its name. It contains the name of the function and its corresponding hint value, which is an index into the export name table of the DLL. The IMAGE_IMPORT_BY_NAME structure is used when the loader needs to resolve an imported function by name rather than by ordinal .

In the context of the IMAGE_IMPORT_BY_NAME structure in the Portable Executable (PE) format, the Hint value is an optional field that provides an additional hint to help identify the imported function. The Hint value is typically an index or position of the function within the export table of the DLL from which it is being imported. It allows for faster and more efficient resolution of imported functions, especially when the import code knows the specific Hint value to look for.

However, in the case of packed executables created by packers, this metadata is often modified. Packers take on the responsibility of performing the resolution stage themselves instead of relying on the operating system's loader.

Packers typically employ techniques to handle the resolution of symbols and imported libraries internally. They analyze the packed executable, identify the required symbols, and dynamically resolve their addresses within the packed code. This allows the packed executable to function without relying on the standard resolution mechanism provided by the operating system's loader. By bypassing the standard resolution stage, packers can modify the structure of the executable and hide important information about dependencies, making it more challenging for reverse engineers and analysis tools to understand the executable's inner workings.

It's worth noting that the resolution stage performed by packers may vary depending on the specific packing technique used and the goals of the packer. Some packers may prioritize minimizing the size of the executable and focus on simple resolution mechanisms, while others may employ more advanced techniques to evade detection and analysis.

The IAT reconstruction is a feature of tools like Scylla ^[6], typically involves scanning the binary of a loaded module (a process's executable image in memory) and identifying patterns that match the standard structure of an IAT. In other words, it's looking for arrays of memory addresses that point to valid locations within loaded DLLs. This process is often needed when analyzing malware or reversing a packed executable, where the IAT may be altered.

Once potential IAT locations are identified, the tool can often rebuild a valid IAT, allowing the binary to be dumped from memory and run as a standalone executable. This can be incredibly useful in malware analysis and similar reverse engineering tasks.

5.4 Environment Setup

For this setup, I opted to use a virtualized environment, specifically VMware Workstation^[2] with Windows 10 Operating System. I chose Windows as several malware samples are readily available, and it is the most widely used and targeted operating system. Additionally, some critical tools are only accessible on Windows. While compatibility layers can allow these tools to be used on other operating systems such as Linux, their proper execution cannot be guaranteed.

To ensure efficient performance, I configured the Operating System with the minimum requirements of 4GB of RAM, 60GB of storage, and 4 cores, as the tools utilized are lightweight and do not require significant computational power. I made certain modifications within the Operating System such as disabling the Windows 10 Virus and Threat Protection. This was necessary to prevent Windows 10 from detecting the malware samples as malicious files and automatically deleting them.

Some other steps can be performed according to the needs:

To Disable Windows Defender

1. Open Windows Security (type Windows Security in the search box)
2. Virus & threat protection → Virus & threat protection settings → Manage settings.
3. Switch Tamper Protection to Off

To permanently disable Real Time Protection

1. Open Local Group Policy Editor (type gpedit in the search box)
2. Computer Configuration → Administrative Templates → Windows Components → Microsoft Defender Antivirus → Real-time Protection
3. Enable Turn off real-time protection.
4. Reboot

To permanently disable Microsoft Defender:

1. Open Local Group Policy Editor (type gpedit in the search box)
2. Computer Configuration → Administrative Templates → Windows Components → Microsoft Defender Antivirus
3. Enable Turn off Microsoft Defender Antivirus
4. Reboot

Disabling or removing the virtual machine's network card is a crucial step to prevent malware from communicating externally and causing unintentional malicious traffic. It is also important to keep the virtualization software, in my case VMware, up to date with the latest version to avoid exploitation of known vulnerabilities that could potentially allow the malware to escape the virtual environment and infect the host machine.

5.5 UPX

For this demonstration of unpacking malware packed with UPX, I will be using the following tools:

• PEStudio ^[3]
• Detect It Easy ^[4]
• IDA (Free) ^[5]
• Scylla ^[6]

Before diving into how and why these tools are used, let's first introduce them and their purpose. PEStudio ^[3] provides information about malware and potential behavior, allowing us to gain an idea of what it might be. The tool also indicates whether a file is malicious or not, drawing from different sources. For example, it may analyze strings within the file that indicate modifications to registry keys, libraries, or imports that usual software do not contain or often used by malware. However, this can also result in many false positives. Additionally, PEStudio ^[3] provides an important indication of the presence of packers through the entropy value, which gives an indication of the file's level of compression.

DetectItEasy is a software tool that provides a straightforward approach to detecting the presence of packers. Its capabilities are similar to those of PEStudio ^[3] in terms of packer detection and using multiple tools can be useful in confirming the presence of packing.

IDA (Free) ^[5], a free version of IDA ^[5] with limited functionality is mainly used to analyze the code inside the malware and to evaluate the malware behavior. In this case, we will use IDA ^[5] to find the tail jump (as described in “Dynamic Analysis” chapter) in the case of UPX and the Original Entry Point (OEP) in the case of MPRESS.

Finally, we can utilize Scylla ^[6] to reconstruct the Import Address Table (IAT) of the unpacked code, dump the unpacked code from memory onto the disk and create a new executable file that includes only the original code.

5.6 MPRESS

Unpacking executable files that have been compressed and protected by the MPRESS packer involves a distinct process compared to the previously discussed UPX packer. While the initial steps to identify the presence of packing in a file remain the same, the subsequent unpacking process and analysis of the MPRESS-packed executables require a different approach.

Similar to the UPX packer, the first step involves conducting a preliminary analysis of the file using tools such as PEStudio ^[3] and DetectItEasy ^[4] to identify indicators of packing. These indicators include reduced numbers of imports, unusually small sections, significant differences between the virtual size and raw size of sections, and other relevant characteristics.

It is important to introduce concept of hardware breakpoint to understand the steps where it is involved and its difference from software breakpoint. A hardware breakpoint is a powerful debugging feature offered by modern processors and debuggers that allows software developers and analysts to pause the execution of a program at a specific memory address or when a particular condition is met. Hardware breakpoint is an essential tool for debugging complex software, analyzing malware, and investigating software vulnerabilities. These breakpoints are typically implemented using specialized registers called debug registers, which are part of the processor's debugging architecture.

To set a hardware breakpoint, the debugger configures one of the available debug registers with the desired memory address or condition. The debug register is then monitored by the processor, which halts program execution when any instruction attempts to read, write, or execute the specified address or the condition is met.

Given this introduction we can proceed to the process of identifying a packed binary with MPRESS which is identical to that of UPX, as shown above. Upon loading the binary packed with MPRESS we receive a similar message to the UPX one, but in this case, clearly stating that the file was packed.

Once the binary had been loaded, we are supposed to face a "pusha" instruction. If this is not the case, we can simply select the start function and insert a breakpoint.

Then, we run the binary, paying particular attention to the Stack View window (usually it is located in the bottom-right corner of the IDA ^[5] window).

Upon stepping over once we can notice that the addresses on the stack had changed (shown in the image below).

At this point, the stack is pointing to start. We have to step over until we come back again to “start” since the start appears twice on the stack, once at 9FF64 and once at 9FF68. In this example, it was enough to do “Step Over” twice.

Note that we started from the "pusha" instruction at 4512D7 and then moved to 4512DE. Then we right clicked on the stack address 19FF64 and followed it in the hex dump.

The “Hex View-1” window changed, and a byte gets highlighted.

At this point, we select the four bytes (DWORD) starting from D7 and put a breakpoint on it. We also set the breakpoint to be a hardware breakpoint with read and write permission (WR), in this case, IDA ^[5] automatically enables these permissions. Although it is possible to view and modify the breakpoint which we don’t need in this case.

At this point we run the binary with the Play button. Once the binary is running, at a certain point it hit the hardware breakpoint. This was done in search of the Original Entry Point (OEP).

IDA ^[5] then detected that RIP point is not in a defined address similarly as sees as of the UPX.

After pressing “Yes”, the code changes, and we are suddenly in a call instruction. When we stepped into it, IDA ^[5] told me the same message about RIP pointing to an undefined address. We cannot use the address of the call instruction (402BA6) as we did for UPX. Instead, we have to use the address pointed out by the call instruction (401B54).

Finally, the address 401B54 can be used as the Original Entry Point (OEP) and unpack with Scylla ^[6]. Once found the OEP the unpacking process with Scylla ^[6] is very much the same as the one seed for the UPX which we will perform to validate its correctness. It is important to remember to restart Scylla ^[6] due to its bug and as seen previously we cannot close IDA ^[5] which was in debugging mode to let Scylla ^[6] use the process in execution.

In order to unpack the steps to follow are the same as seen for UPX:

1. Select the process of IDA ^[5] running the sample.
2. Insert in the OEP field the address found on IDA ^[5].
3. Click “IAT Autosearch”

Upon setting up Scylla ^[6] with the new values for the malware sample, it successfully finds the start address and its size.

At this point, we obtained all the imports by simply clicking “Get Imports”.

Scylla ^{[6]>/sup> shows that 68 imports are found since we own the unpacked binary, we can double-checked if the found imports matched the original number of imports.}

In fact, PEStudio ^[3] had told us that the unpacked binary had 68 imports, so our process was successful. Now to create the unpacked binary, we first create a dump by clicking on "Dump" and saved the new file.

Now we click “Fix Dump” and select the dump file previously created in order to unpack it and create the unpacked binary.

To make further check we can load the newly created binary in PEStudio ^[3] and compared it with the original packed binary to see their differences.

Finally, we could see the differences between the original packed binary and the unpacked binary that we created. Much like UPX, even here, the same value changes, even if the change is not significant. The unpacked version with Scylla ^[6] has 0 imports, which was the same as UPX. It is not a problem as long as Scylla ^[6] has found all the imports during the process.

This concludes the manual unpacking of binaries packed of MPRESS.

7.1 Advanced Packing Techniques

As can be seen, such a complex packer that implements several anti-analysis and evasion techniques cannot be analyzed in a simple way as seen in the previous examples and they require different strategies. The following chapters will discuss advanced strategies for analyzing and understanding the implementation of packed malware with advanced packing strategies.

“More complex packers often involve several layers of unpacking routines, in which the first layer unpacks the second one, which in turn unpacks another routine, until the original code is reconstructed at the end of the chain. Others employ several parallel processes, they interleave the execution of unpacking code with the original code, or even incrementally unpack and re-pack the code on-demand before and after its execution ^[93]”.

Following a subset of advanced unpacking techniques presented in the paper “SoK: Deep Packer Inspection: A Longitudinal Study of the Complexity of Run-Time Packers”:

1. Parallel Unpackers
2. Unpacking Frames
3. Code Visibility

Parallel Unpackers is a strategy where “packers employ several processes in order to unpack the original code ^[9]”. These processes can interact with one another to perform various checks such as synchronization or time measurement to avoid detection and analysis.

Unpacking Frame “is a region of memory in which we observe a sequence of a memory write followed by a memory execution. Traditional run-time packers have one unpacking frame for each layer, because the code is fully unpacked in one layer before the next layers are unprotected. We call these packers single-frame unpackers. However, more complex cases exist in which the code of one layer is reconstructed and executed one piece at a time. These cases involve multiple frames per layer and are called multi-frame packers in our terminology ^[9]”.

Code Visibility is an “advanced multi-frame examples exist that selectively unpack only the portion of code that is actually executed. This approach is used as a mechanism to prevent analysts and tools from easily acquiring a memory dump of the entire content of the binary ^[9]”. There are different types of unpacking mechanism such as: Full -Code Unpacking which is a routine that unpack the entire code and executes it. Incremental Unpacking is an approach where the code in unpacked, if necessary, before being executed but not all the code are unpacked. Shifting-Decode Frames is a strategy where only one frame at a time is visible and unpacked and the previews unpacked frame are repacked to reduce the visibility.

7.2 Deep Dive into PEzoNG and Advanced Malware Packer

PEzoNG is a packer for Windows that implements various evasion techniques and anti-analysis mechanism. “PEzoNG is built with modularity in mind and allows to add new features in a simple way by adding new modules that could implement different techniques with a fine grained detail ^[10]”. In this way it is possible to add new techniques and implement additional feature to add to the packer and develop new evasion strategies. PEzoNG is a polymorphic packer which means that its signature changes from computation to computation which makes Anti Viruses that checks for trivial signature ineffective.

The image below shows an overview of the packing and unpacking overview of PEzoNG:

PEzoNG includes the following modules:

1. Encryption
2. APIs
3. Syscall
4. Evasion
5. PE Loader
6. Shellcode Injeciton

9.1 Introduction

To solve the problem of malware classification and its variant a system named Malwise [14] has been developed. This system classifies malware “using a fast application-level emulator to reverse the code packing transformation, and two flowgraph matching algorithms to perform classification. An exact flowgraph matching algorithm is employed that uses string-based signatures and is able to detect malware with near real-time performance ^[14]”.

The innovative technique that Malwise uses is the use of structuring and decompilation to generate malware signature. Structuring a malware consists in rearranging the code in a more understandable manner which helps to understand patterns and logical flow of it. Decompilation is used to convert the low-level code to a pseudo higher level for a better understanding of it and analysis.

The approach of Malwise to generate the signature is divided in two parts and they consist of “Exact Matching”, Malwise look for the exact match of a control flow graph from a malware database to classify the input binary, and “Approximate Matching” Malwise search for similarity between the input flow graph and the database to classify binaries respecting a given threshold.

9.2 Detecting Packing

The first step is to understand if the malware employs packing techniques. This is done by computing the entropy of the input file as a packed file most like have high entropy compared to a non-packed one. “If the binary is identified as being packed, then the dynamic analysis to perform automated unpacking proceeds. If the binary is not packed, then the static analysis commences immediately ^[14]”.

9.3 Detect Code Extraction (Unpacking Completion)

Malwise detects the OEP by tracking the memory writes ^[15]. The idea behind this technique discussed in Renovo [15] is that regardless the employed packing technique eventually the original code should be preset in the memory to be executed and the instruction pointer must jump to the address (OEP) of the extracted program to execute it at runtime.

Malwise improved this concept in the event of multi-layer packing and can identify precisely in which state the OEP will be revealed. The idea is that “if there exist high entropy packed data that have not been used by the packer during execution, then they remain to be unpacked. To determine if a particular stage of unpacking represents the OEP, the entropy of new or unread memory in the process image is examined. Newly written memory is indicated by the shadow memory for the current stage being unpacked. Unread memory is maintained globally, in a shadow memory for all stages. If the entropy of the analyzed data is low, then it is presumed that no more compressed or encrypted data are left to be unpacked. This heuristically indicates completion of unpacking ^[14]”.

9.4 Static Analysis

The objective of the static analysis is to extract characteristics from the unpacked binary. The characteristics are then transformed in a higher-level representation that are suitable from string matching (signature). For such a purpose Exacts Flow Graph and Approximate Flow Graph matching are employed.

9.5 Exact Flow Graph Matching

Once the characteristics of the input binary has been extracted a higher level of control flow graph is created and its signature is also generated. The signature is then compared with the database of malware signatures (consisted of graph edges and ordered nodes) in search of isomorphism. The figure below shows and example of control flow graph and the signature of it.

9.6 Approximate Flow Graph

Using the approximate flow graph is possible to identify a greater number of malware variants compared to the previews matching technique. In this process structuring is used to generate signatures to compare the similarity with the signatures in the database. “Structuring is the process of recovering high-level structured control flow from a control flow graph. The intuition behind using structuring as a signature is that similarities between malware variants are reflected by variants sharing similar high-level structured control flow ^[14]”.

The reason of performing structuring of code is because malware variants have similarities at a higher-level representation of their structure and control flow. Structuring produces a string of characters representing the higher-level representation of the malware code block. Following an example of the signature generation:

Malwise uses a database containing the signature generate from flow graphs of known malware in string format. To classify an input binary the signature is then compared with the database in search of similarities. “For exact matching, the assignment is based on string equality. "For approximate matching, a greedy assignment is made for the best approximate matching string where the similarity ratio is above 0.9 ^[14]"”.

9.7 Similarity

The greedy match of 0.9 threshold is used to identify potential exact matches and in the final classification a threshold of 0.6 is used to identify malware variants. Both thresholds are used in the result for malware identification and a lower threshold is used for a broader match for flexibility and capture even slightly modified versions of malware.

The problem that arises from this approach is the match with several malwares. For such reason the similarity check process is divided in two stages. The stage is the search for uniqueness, the input malware is compared with the database to check if it’s a unique malware or has been already seen previously by comparing its extracted features. In the second step the remaining duplicated are compared with a lower threshold of 0.6 in search of similarities and malware variants. Only the exact matches or the matches above 0.6 are returned as a result of the searching process.

9.8 Evaluation

The following tables shows the accuracy of Malwise in identifying the OEP of binaries (HOSTNAME.EXE and CALC.EXE) packed with different packers.

From the table we can see that Malwise has a great accuracy in finding the OEP even in cases of several layer of several layer of packing. Malwise failed the unpacking of CALC.EXE only when packed with “napck”, “expressor”, “telock”, and “pespin”. It failed 4 out of 14 packer which is an impressive result, and it was able in most of the cases to reveal more than 98% of the packed instructions.

The performance evaluation shows that thanks to the immersive speed of Malwise, as shown below, is suitable to be integrated into desktop anti malware systems.

10.1Embedded Systems and IoT

With the emergence of IoT and Embedded Systems, malicious users have adapted to exploit these technologies for malicious purposes. In this regard, it is important how much and how packing impacts them.

The difference between Embedded System and IoT lies in connectivity. That is, an IoT device is connected to the Internet e.g., a washing machine or refrigerator that can offer real-time information. On the other hand, an embedded system is like a minicomputer build for a specific use case, equipped with minimal computing power, and it is not connected to the internet.

Normally an embedded system does not have an Internet connection, but an IoT embedder does, so an embedded system can be considered a subset of IoT. So, these types of devices can be used by malicious users to carry out further attacks or perform packing and unpacking of malware.

Theoretically it is possible to carry out cyber-attacks from a compromised IoT device, its only limitation would be the available device resources such as processing power and memory. More specifically if we talk about packing “While packing, in general, helps reduce the size of the stored binary, the unpacking/decryption time increases the total instructions executed and leads to increased power consumption ^[56]”.

10.2 Performance Evaluation

The study experimented with two packers, AES Encryption and UPX, with the addition of a simple anti-debugging feature, time delay, “when single-stepping a process through a debugger the wall clock differences between any two points will grow dramatically when compared to a native execution. Time based anti-debugging software can perform two time checks and infer that it is running under such conditions if unusual difference is detected [56]”.

The experiment was conducted against the embedded board, which consists of Intel XScale PXA255 400MHz CPU, 128MB SDRAM, 32MB Flash ROM, Embedded Linux (Kernel 2.4.19) and g++ was used to compile the executables. Its result proved that UPX several times faster that AES because of the AES decryption overhead also “UPX is better than AES-encryption packing for reducing the startup time and invocation time on embedded systems with limited resources ^[56]”.

Regarding power consumption (W=V·I·t=Volt·Ampere·milliseconds), it was not possible to keep the electric current stable so there is a margin of error of ±2%. UPX showed very low power consumption compared with AES. In this regard, AES can be optimized with power consumption reduction in mind.

The two programs binary that have been choose for these experiments are MD5 hashing algorithm and AES encryption algorithms. Their execution time and power consumption have been measured in different states such as with no packing, with UPX packing and AES packing. The table below shows the data from the experiment and the difference in the measurements.

10.4 IoT Embedded System for malware distribution

IoT Embedded Systems can be infected or used for malware distribution. Through the results obtained, it is possible to see how fast the execution of programs packed specially with UPX are. On average between the two programs (MD5 and AES), the average execution time is about 0.45sec. despite having significantly limited computation power. By compromising a device an attacker can use it to make lateral movement and infect other devices in the network.

This works particularly well in the case of polymorphic malware where each time after compromising a device it is possible to re-pack the malware so that it has a different signature to infect subsequent devices. This would increase the chances of infection and reduce detection by security systems.

10.5 Packing Limitations

Given the reduced computational power, the packing algorithm cannot be the same as in standard computing systems. That is, the packer in an Embedded IoT system cannot implement all the evasion features and advanced anti-debugging techniques such as parallel execution and unpacking through different child processes or threads. This would drastically increase execution times. So, in this regard, malware and packers for these systems have to be crafted in a very specific way keeping in mind power consumption and execution time.