Hacking Series Part 4

Challenge: vault-door-8

Category: reverse engineering

We are given java source code called VaultDoor8.java. When viewing it’s contents for the first time, it appears to not be formatted correctly as usual java source is. There are also multiple occurrences of commented out code that may be need to be included in the original source. After reformatting the code, it looks like this.

There are two important functions to look at: scramble and switchBits. scramble returns a char array with each char’s bits switched 8 times in a specific order. switchBits swaps the bit value at two specified places of a char, then returns it. When I looked at the commented code of both of these functions, I quickly realized that they were all irrelevant. Some introduced variables that were never used and some called switchBits using parameters that did not satisfy the precondition of p1 < p2. All of these comments can be deleted.

Next, we need to figure out exactly how the bits are swapped in scramble to see if it is possible to reverse the process somewhere. The bits are swapped a total of 8 times in specific places.

The positions represent an index of the bits of the char from 0–7.

All we have to do to get the original chars before the scrambling is take the chars in the expected array and reverse the swapping process. To do this, scramble should look like the following.

Then, switch the a array being used for scramble to the expected array found in checkPassword.

Before compiling and running the program, print the scrambled array, which should now include the original bits belonging to part of the flag. Then add the proper flag format.

picoCTF{s0m3_m0r3_b1t_sh1fTiNg_89eb3994e}