This challenge was on reverse engineering. The binary is 32bit MZ-PE executable for Windows.
Summary: reverse engineering, crypto
File is packed by very easy algorithm. It is xored with 0x2d
After decryption, sample becomes usually C++ compiled file.
This file can crypt and decrypt input files with password (which is got from command line).
From password program calculates a hash and than uses them for xor first dword in encrypted/decrypted file. On each step hash is regenerated and depends on previous decrypted dword.
signed int __cdecl hash_pass(int init_val, unsigned __int8 *pass, unsigned int pass_len)
{
unsigned int len; // [sp+0h] [bp-Ch]@1
int prev; // [sp+8h] [bp-4h]@1
prev = ~init_val;
for ( len = 0; len < pass_len; ++len )
prev = IV[(unsigned __int8)(prev ^ pass[len])] ^ ((unsigned int)prev >> 8);
return ~prev;
}
For decrypt file we need to find first hash. On assignment, we have an encrypted file “Challenge.png”. We know first 4 bytes in PNG header (89504e47), so if we do xor encrypted data and 4bytes from header than we will get first hash.
Header = 89504e47
Encrypted data = 617180C7
Firts hash = E821CE80
Further let’s change it in memory (under dbg) and continue to execute program. That’s it!
So, key:
31cb891dbb83e16aa9ea1c54bd0bf1d1
