Hacking Series Part 6

Challenge: Investigative Reversing 2

Category: reverse engineering and forensics

We are given a binary named “mystery” and image named “encoded.bmp”. When looking at a hex dump of encoded.bmp, there is no obvious indication of a flag being stored anywhere. In order to understand how mystery influences encoded.bmp, I opened it in IDA.

Mystery opens three files: “flag.txt”, “original.bmp”, and “encoded.bmp”. flag.txt and original.bmp are read from, and encoded.bmp is appended to.

If these files are not found, the next set of assembly instructions deal with error handling in this case, which can be ignored. Next there is a loop that uses a counter from 0 to 2000. A character is read from original.bmp and appended to encoded.bmp without any operations or encoding. This occurs 2000 times, then the program attempts to read 50 chars from flag.txt.

If flag.txt does not contain 50 chars, then error handling occurs for this situation. If it does, the next set of instructions set up counters for two different loops.

The first loop uses a counter from 0 to 49, and the second uses a counter from 0 to 7. That means the 50 characters read from flag.txt are each manipulated 8 times with an unknown logic. The following instructions preform an encoding on each of these characters, then append it to encoded.bmp.

Then, after these two loops are complete, the rest of original.bmp is appended to encoded.bmp. That means that, in total, only 400 (50*8) chars are encoded in encoded.bmp, and the only relevant part of this file we should look at is from bytes 2000 to 2400.

To split encoded.bmp into smaller segments (since the file is very large), I used split -b 2000 encoded.bmp , which split the file into smaller 2000 byte files.

The first 2000 bytes are stored in xaa, the next 2000 in xab, the next 2000 bytes in xac, and so on. Since we need to be looking at only bytes 2000–2400, the only file we need from this entire output is xab.

We now need to figure out exactly how the characters are being encoded so that it can be reversed. In the second loop, each char is encoded by a function called codedChar which uses a series of sar, and, or, and mov instructions to shift the character. Determining exactly how to reverse this function would be a tedious and time consuming task. Instead, in a hint for the challenge, it mentions to look into LSB encoding.

LSB encoding is a type of steganography which uses the least significant bit of 8 different bytes to conceal a relevant byte for our challenge. In this case, it could conceal an ASCII character from the flag in binary.

To get the binary from xab, I used the following command, which dumps it in analyze.txt.

xxd -b -c1 xab | cut -d” “ -f2 | tr -d “\n” > analyze.txt

I then made a Python script to get the LSB from every byte and print the results. In the results, the bytes that I got did not seem to be valid ASCII characters at first. After multiple attempts to find a pattern within the binary, I realized that each byte I got from the LSBs needed to be reversed. Then, they needed to be added to by 5 to get the original character. In the end my Python script looked like this.

The correct flag was printed.

picoCTF{n3xt_0n30000000000000000000000000c5c7dd39}