Codegate CTF Quals 2014 - dodoCrackme - Reverse 200 - [Team SegFault]

The binary starts with mmap() syscall to allocate a memory of 0x7530 bytes.

.text:00000000004000E0 exit:                                  
.text:00000000004000E0                 mov     eax, 3Ch
.text:00000000004000E5                 mov     edi, 0
.text:00000000004000EA                 syscall
.text:00000000004000EC                 mov     r9d, 0
.text:00000000004000F2                 mov     r8, 0FFFFFFFFFFFFFFFFh
.text:00000000004000F9                 mov     r10d, 22h
.text:00000000004000FF                 mov     edx, 3
.text:0000000000400104                 mov     esi, 7530h
.text:0000000000400109                 mov     edi, 0
.text:000000000040010E                 mov     eax, 9
.text:0000000000400113                 syscall                 ; mmap
.text:0000000000400115                 cmp     rax, 0
.text:0000000000400119                 jle     short exit
.text:000000000040011B                 mov     rbp, rax
.text:000000000040011E                 add     rbp, 3A98h

The allocated memory is then used as stack for performing memory operations. There is lot of lea, inc, dec instructions throughout the binary to compute values. Below is the syscall to write() to print chr(r) and chr(o)

.text:0000000000400276                 lea     rbp, [rbp+8]
.text:000000000040027A                 dec     byte ptr [rbp+0]
.text:000000000040027D                 dec     byte ptr [rbp+0]
.text:0000000000400280                 dec     byte ptr [rbp+0]
.text:0000000000400283                 mov     eax, 1
.text:0000000000400288                 mov     edi, 1
.text:000000000040028D                 mov     rsi, rbp
.text:0000000000400290                 mov     edx, 1
.text:0000000000400295                 syscall                 ; r
.text:0000000000400297                 dec     byte ptr [rbp+0]
.text:000000000040029A                 dec     byte ptr [rbp+0]
.text:000000000040029D                 dec     byte ptr [rbp+0]
.text:00000000004002A0                 mov     eax, 1
.text:00000000004002A5                 mov     edi, 1
.text:00000000004002AA                 mov     rsi, rbp
.text:00000000004002AD                 mov     edx, 1
.text:00000000004002B2                 syscall                 ; o

What I can see is that, value pointed by RBP is being operated upon to create needed characters. Few more system calls prints root@localhost's password: .

[ctf@renorobert codegate14]$ ./crackme_d079a0af0b01789c01d5755c885da4f6 
root@localhost's password: asdfghjkl
Permission denied (password).

Supplying a wrong key prints Permission denied (password). and a maximum of 32 characters are read. Then I decided to break at read() syscall to inspect memory. The idea was to check if all the operations and memory referred by RBP, left the key or any interesting string in the mmap'ed area.

gdb-peda$ break *0x00000000004065AA
gdb-peda$ x/600x $rbp
0x7ffff7ff9b30: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b38: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b40: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b48: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b50: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b58: 0x48 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b60: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b68: 0x34 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b70: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b78: 0x50 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b80: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b88: 0x50 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7ffff7ff9b90: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

All these bytes yields the flag H4PPY_C0DEGaTE_2014_CU_1N_K0RE4