GoAuditParser: The FireEye Audit Swiss Army Knife

As Mandiant consultants, we rely on FireEye Endpoint Security and Redline to collect forensic information from endpoint artifacts, known as audit data, for investigations. Both solutions offer integrated methods to review the collected audit data. However, sometimes we need to work with the data programmatically, such as performing intelligence enrichment, frequency analysis, or extracting specific information from the data in bulk.

We created GoAuditParser and made it open source to help analysts with those needs.

GoAuditParser’s primary purpose is to convert XML data from FireEye Endpoint Security bulk audit requests, triages, and Redline packages into CSV format for various use cases. In CSV format, an analyst can review the data in Microsoft Excel, easily ingest the data into a preferred review platform such as a SIEM, or use any custom tools to work with the data programmatically. Just point GoAuditParser at the audit MANS file or raw XML audit data, and it will extract and/or parse the data to CSV format quickly and efficiently!

Parsing FireEye Audits from XML format to CSV

When audit data is acquired using the aforementioned FireEye tools, it is usually in XML format. Figure 1 depicts an excerpt of a sample FireEye FileItem audit, which contains an enumeration of the MFT and INDX records, in XML format.

Figure 1: Sample FireEye XML FileItem audit

GoAuditParser parses XML audit files dynamically, meaning it does not rely on a static template for retrieving all of the available information in an audit and thus will always attempt to parse any new fields or even new types of audit data sources. As long as the audit data fits the expected XML schema of FireEye audits, it can be parsed.

While the tool has many different command line arguments, it can be run easily with the command line syntax depicted in Figure 2.

> GoAuditParser -i path/to/input -o path/to/output

Figure 2: Simple GoAuditParser usage syntax

After parsing XML audit files with the tool, parsed CSV audit files will be found in the output directory. Figure 3 depicts the parsed output of the sample FireEye XML audit referenced in Figure 1. Please note that some of the columns in the spreadsheets within figures throughout this blog post have been manually excluded for clarity.

Figure 3: Parsed output of a FileItem audit

FireEye Audit Timelines

A key feature of GoAuditParser is the ability to create timelines from parsed audit data. Users can add the -tl flag to the command syntax depicted in Figure 2 to create a combined timeline from all the parsed audit data. Timeline files contain entries with timestamp details, a “Summary” column (more details about this column discussed further down), and a handful of specific attribute columns including “MD5”, “Size”, “User”, etc. Figure 4 depicts a sample audit timeline with multiple audit sources such as FileItem, EventLogItem, and RegistryItem.

Figure 4: A single system’s audit timeline

The timeline “Summary” column contains the important fields for each audit entry. Important audit fields are joined into this column based on the timeline configuration file which can be customized by audit source. This process is summarized for the RegistryItem audit source in Figure 5. Documentation on GoAuditParser configuration files is available on our GitHub page.

Figure 5: Summary of the timeline “Summary” process

Timelines can also be created using audit data from multiple systems. This can be used to perform a type of frequency analysis to identify anomalies, as depicted in Figure 6.

Figure 6: Using frequency analysis on a multi-system audit timeline to hunt for anomalies across multiple systems

Optimizations and Other Features

GoAuditParser can parse FireEye Endpoint Security triage package in less than a minute on average. It has been heavily optimized and has a handful of extra features implemented, including the following:

• Automatically chooses between different parsing methods to optimize speed and memory consumption
• Automatically splits large files into smaller chunks before processing to save time
• Automatically caches progress if you need to pause parsing lots of data and resume later
• Adjustable settings that make it easier to work with parsed audits in Microsoft Excel
• Documented and customizable parsing and timelining configuration files
• Can extract files in bulk from FireEye Endpoint Security file acquisitions
• Many additional small features documented on the project’s GitHub page

Conclusion

By making the GoAuditParser public and open source, we hope to enable those who use our tools with more versatility and customizability in their hunts for evil.

Compiled binaries of GoAuditParser are available for Windows, Mac, and Linux on the FireEye Marketplace. Documentation and the source code are available on Mandiant’s GitHub.