Showing posts with label Hack You CTF. Show all posts
Showing posts with label Hack You CTF. Show all posts

Saturday, January 18, 2014

Hack You CTF 2014 - Crypto 400 - CRYPTONET - [Team SegFault]

Challenge description says about use of strange protocol using RSA cryptosystem. We also have access to the client source code and a pcap file. Reading the code we could see that, client receives n and e value from remote system. This value is used to encrypt the message before sending it. The protocol has following format
Receive:
[2 bytes specifying the size zlib compressed e value][zlib compressed e value] [2 bytes specifying the size zlib compressed n value][zlib compressed e value]
Send:
[2 bytes specifying the size zlib compressed m^e mod n] [zlib compressed m^e mod n]
Analyzing the pcap file we could see that client has communicated with some 19 remote machines. First we must extract the values of e and n for all the communication. Initially I checked if those n values are having some common prime, but all the gcd checks ended up as relatively prime. e value was small, 17. Further reading on use of low public exponent took me to Hastad's Broadcast Attack. Code to solve the challenge using Hastad's Broadcast Attack is below:
#!/usr/bin/env python

from scapy.all import *
from sage.all import *
import zlib
import struct

PA = 24L
packets = rdpcap('packets.pcap')
client = '192.168.1.5'
size = 2 # size of e and n is packed into 2 bytes
list_n = []
list_m = []

for packet in packets:
    if packet[TCP].flags == PA:
       if packet.dst == client:
           src = packet[IP].src
           raw_data = packet[TCP].load

           size_e = struct.unpack('!H', raw_data[:size])[0] 
           e = int(zlib.decompress(raw_data[size: size + size_e]))

           size_n = struct.unpack('!H', raw_data[size + size_e: 2 * size + size_e])[0]
           n = int(zlib.decompress(raw_data[2 * size + size_e: ]))
           list_n.append(n)

        if packet[IP].src == client:
            raw_data = packet[TCP].load
            size_m = struct.unpack('!H', raw_data[:size])[0]
            m = int(zlib.decompress(raw_data[size: size + size_m]))
            list_m.append(m)

e_17 = crt(list_m, list_n)
factors = prime_factors(e_17)
enc_message = 1
for num in factors:
    enc_message *= num

print hex(enc_message).decode('hex')
# 'Secret message! CTF{336b2196a2932c399c0340bc41cd362d}\n'
Flag for the challenge is CTF{336b2196a2932c399c0340bc41cd362d}

Hack You CTF 2014 - Crypto 300 - Matrix - [Team SegFault]

We were given source code of encryptor and an encrypted file flag.wmv.out. The encryption algorithm performs the following operation

[*] Generates a 4x4 matrix as key using randint() function. An secret password is used as seed, which we dont know
[*] File is padded with NUL such that size(file) % 16 == 0
[*] Each 16 bytes of file is converted to 4x4 matrix and multiplied with key
[*] Each element of resultant 4x4 matrix is packed into 2 bytes of data and written to file

First we should find the key to perform decryption. The file extension leaves a clue wmv. Using wmv header as known plain text and encrypted header as cipher text, the key could be found as wmv_header.inverse() * enc_header. Once the key found, we have to write decryption routine such that encrypted file is processed in blocks for 32 bytes in a 4x4 matrix. Below is the code to solve this
#!/usr/bin/env python

from sage.all import *
import struct

wmv_header = [[0x30, 0x26, 0xB2, 0x75], [0x8E, 0x66, 0xCF, 0x11], [0xA6, 0xD9, 0x00, 0xAA], [0x00, 0x62, 0xCE, 0x6C]]

def Str2matrix(s):
    #convert string to 4x4 matrix
    return [map(lambda x : ord(x), list(s[i:i+4])) for i in xrange(0, len(s), 4)]
def Matrix2str(m):
    #convert matrix to string
    return ''.join(map(lambda x : ''.join(map(lambda y : struct.pack('!B', y), x)), m))
def PackedStr2matrix(s):
    matrix = [0] * 16 
    for i in range(0, len(s), 2):
        matrix[i/2] = struct.unpack('!H', s[i:i+2])[0]
    matrix = [matrix[i:i+4] for i in range(0,len(matrix), 4)]
    return matrix
def Multiply(A,B):
    #multiply two 4x4 matrix
    C = [[0 for i in xrange(4)] for j in xrange(4)]
    for i in xrange(4):
        for j in xrange(4):
            for k in xrange(4):
                C[i][j] += A[i][k] * B[k][j]
    return C
        
header =  matrix(wmv_header)
encrypted_wmv = open('flag.wmv.out','rb').read()
size = struct.unpack('!I', encrypted_wmv[:4])
enc_header = encrypted_wmv[4:36]
enc_header = matrix(PackedStr2matrix(enc_header))

# sage: header = matrix([[48, 38, 178, 117], [142, 102, 207, 17], [166, 217, 0, 170], [0, 98, 206, 108]])
# sage: enc_header = matrix([[6025, 10758, 8274, 14059], [10718, 11769, 5025, 15260], [19537, 18796, 14142, 15035], [7648, 8842, 7254, 17852]])
# sage: header.inverse() * enc_header
# [31 51 20  0]
# [53 10  6 45]
# [ 3 13  3 49]
# [17 48 56 31]
# sage: key.inverse()
# [  5732/2519421  96221/5038842 -41017/2519421 -10009/5038842]
# [ 65399/2519421 -67381/5038842  44957/2519421 -44311/5038842]
# [-49681/2519421  22679/5038842 -51064/2519421 128507/5038842]
# [-14660/2519421  10597/5038842  45127/2519421   4501/5038842]

key = header.inverse() * enc_header
key_inverse = key.inverse()

out = open('flag.wmv','wb')
encrypted_wmv = encrypted_wmv[4:]

for i in xrange(0, len(encrypted_wmv), 32):
    unpacked_data = matrix(PackedStr2matrix(encrypted_wmv[i:i+32]))
    decrypted = unpacked_data * key_inverse
    out.write(Matrix2str(decrypted))
out.close()

# CTF{b699a72e2692d16f65ec9626055aa740}
We get a decrypted wmv file which has the flag.
root@sagepc:~ $file flag.wmv
flag.wmv: Microsoft ASF
Flag for the challenege is CTF{b699a72e2692d16f65ec9626055aa740}

Hack You CTF 2014 - Crypto 200 - Hashme - [Team SegFault]

We have access to the source code of a remote service which allows users to register and login.

Register:
[*] Everytime a new login is created, hash of string SALT+login={username}&role=anonymous is computed
[*] Computed hash is appended to login={username}&role=anonymous, encrypted with a key using xor and then encoded using base64
[*] The generated base64 string is our certificate for login

Login:
[*] The user provided certificate is base64 decoded and decrypted using xor
[*] Hash of SALT+login={username}&role=anonymous is computed and checked with the user supplied hash. If it matches, then login is provided
[*] Administrator gets access to flag file

We have to find a way to get administrator access by forging the request. Ok, first we have to find the key used in the remote machine. This can be done by registering a user with long username
client sends: username as "A"*1000 # login={username}&role=anonymous is our known plain text
server sends: XOR(login={username}&role=anonymous + HASH) # cipher text
Using the plain text - cipher text combination, the KEY can be found.
Now we have to find a way to compute valid hash without knowing the SALT. Thanks to my friend who pointed me out the Hash Length Extension attack. The final 32 byte of hash comes from A,B,C and D of used hash algorithm
A = 0x67452301
B = 0xEFCDAB89
C = 0x98BADCFE
D = 0x10325476
X = [int(0xFFFFFFFF * sin(i)) & 0xFFFFFFFF for i in xrange(256)]

for i,ch in enumerate(s):
    k, l = ord(ch), i & 0x1f
    A = (B + ROL(A + F(B,C,D) + X[k], l)) & 0xFFFFFFFF
    B = (C + ROL(B + G(C,D,A) + X[k], l)) & 0xFFFFFFFF
    C = (D + ROL(C + H(D,A,B) + X[k], l)) & 0xFFFFFFFF
    D = (A + ROL(D + I(A,B,C) + X[k], l)) & 0xFFFFFFFF

return ''.join(map(lambda x : hex(x)[2:].strip('L').rjust(8, '0'), [B, A, D, C]))
We can use the HASH(login={username}&role=anonymous) sent by remote service as initial values, then compute the hash values of newly appended string without knowing the SALT. But one value we need to know is the length of SALT. This can be easily bruteforced.
Below is the code for checking login and administrator.
if hashme(SALT + auth_str) == hashsum:  
    data = parse_qs(auth_str, strict_parsing = True)
    print '[+] Welcome, %s!' % data['login'][0]
    if 'administrator' in data['role']:
    flag = open('flag.txt').readline()
    print flag
We need a craft a input as login={username}&role=anonymous&role=administrator so that parse_qs() sets data['role'] to ['anonymous','administrator'] and if 'administrator' in data['role'] is passed. Finally, find the valid hash value for string login={username}&role=anonymous&role=administrator to get the flag. Below is the code used
#!/usr/bin/env python

import base64
from math import sin

def hashme(length, s, state):
    # my secure hash function
    def F(X,Y,Z):
        return ((~X & Z) | (~X & Z)) & 0xFFFFFFFF
    def G(X,Y,Z):
        return ((X & Z) | (~Z & Y)) & 0xFFFFFFFF
    def H(X,Y,Z):
        return (X ^ Y ^ Y) & 0xFFFFFFFF
    def I(X,Y,Z):
        return (Y ^ (~Z | X)) & 0xFFFFFFFF
    def ROL(X,Y):
        return (X << Y | X >> (32 - Y)) & 0xFFFFFFFF

    #A = 0x67452301
    #B = 0xEFCDAB89
    #C = 0x98BADCFE
    #D = 0x10325476
    B = int(hash[:8],16)
    A = int(hash[8:16],16)
    D = int(hash[16:24],16)
    C = int(hash[24:],16)

    X = [int(0xFFFFFFFF * sin(i)) & 0xFFFFFFFF for i in xrange(256)]

    for i,ch in enumerate(s):
        k, l = ord(ch), (i+length) & 0x1f
        A = (B + ROL(A + F(B,C,D) + X[k], l)) & 0xFFFFFFFF
        B = (C + ROL(B + G(C,D,A) + X[k], l)) & 0xFFFFFFFF
        C = (D + ROL(C + H(D,A,B) + X[k], l)) & 0xFFFFFFFF
        D = (A + ROL(D + I(A,B,C) + X[k], l)) & 0xFFFFFFFF

    return ''.join(map(lambda x : hex(x)[2:].strip('L').rjust(8, '0'), [B, A, D, C]))



login = "A"*1000 
user_data = 'login=%s&role=anonymous' % login

encoded_user_data = "RK5yZMJaRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJkUZfC1QZOZsfcZlXz4V1qj5TTm1j2pJnepqTLNKoeqfSlYQa9IahiQhPbSkaYBUTO0mRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJkUZfC1QZOZsfcZlXz4V1qj5TTm1j2pJnepqTLNKoeqfSlYQa9IahiQhPbSkaYBUTO0mRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJkUZfC1QZOZsfcZlXz4V1qj5TTm1j2pJnepqTLNKoeqfSlYQa9IahiQhPbSkaYBUTO0mRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJkUZfC1QZOZsfcZlXz4V1qj5TTm1j2pJnepqTLNKoeqfSlYQa9IahiQhPbSkaYBUTO0mRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJkUZfC1QZOZsfcZlXz4V1qj5TTm1j2pJnepqTLNKoeqfSlYQa9IahiQhPbSkaYBUTO0mRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJkUZfC1QZOZsfcZlXz4V1qj5TTm1j2pJnepqTLNKoeqfSlYQa9IahiQhPbSkaYBUTO0mRRl8LVBk5mx9xmVfPhXWqPlNObWPakmd6mpMs0qh6p9KVhBr0hqGJCE9tKRpgFRM7SZFGXwtUGTmbH3GZV8+Fdao+U05tY9qSZ3qakyzSqHqn0pWEGvSGoYkIT20pGmAVEztJiIqUgB0GMZDU+ldcxAh5N2LaUrM/hIw788YPMU42cruOSc3SaRv/lVRTMODTfQl" 

decoded_user_data = base64.b64decode(encoded_user_data)

key = []
for i in range(len(user_data)):
    key.append(ord(decoded_user_data[i]) ^ ord(user_data[i])) 
key = key[:50]  # sizeof key is 50 bytes

# key
# [40, 193, 21, 13, 172, 103, 4, 88, 61, 108, 17, 37, 167, 45, 60, 135, 36, 30, 127, 84, 151, 233, 184, 12, 120, 244, 206, 43, 8, 220, 171, 43, 13, # 242, 11, 224, 171, 222, 11, 23, 81, 42, 147, 91, 199, 101, 96, 124, 245, 229]

login = "A"
user_data = 'login=%s&role=anonymous' % login
encoded_user_data = "RK5yZMJaRX5PA31AmkxS6EpnEjvimo81G82rT2q5n0k9ym/Tme47IDcT92z2UVJOxNEe+CE5"
decoded_user_data = base64.b64decode(encoded_user_data)

decrypted_data = ''

for i in range(len(decoded_user_data)):
    decrypted_data += chr(ord(decoded_user_data[i]) ^ key[i % len(key)])

target = "&role=administrator"
hash = decrypted_data[-32:]

length_of_salt = 0
length_of_user_data = len(user_data)

for length_of_salt in range(1,30):
    forged_hash = hashme(length_of_salt + length_of_user_data, target, hash)
    string_to_enc = user_data + target + forged_hash
    enc_payload = ''

    for i in range(len(string_to_enc)):
        enc_payload += chr(ord(string_to_enc[i]) ^ key[i % len(key)])

    print "%d : %s" % (length_of_salt, base64.b64encode(enc_payload))
We get successful login and flag, when length of SALT is 20 bytes.
RK5yZMJaRX5PA31AmkxS6EpnEjvimp5+F5irFmm4xkJjm3iU2b9/eCNM8TimAFcdwYca8CU8nQI8OVxbdRefTgzjFi0bMaPejw==
[+] Welcome, A!
CTF{40712b12d4be002e20f51424309a068c}

Hack You CTF 2014 - Crypto 100 - Easy One - [Team SegFault]

Source file of the encryption algorithm was given. Also, we have plain text and cipher text combination for one message
 FILE* input  = fopen(argv[1], "rb");
 FILE* output = fopen(argv[2], "wb");
 char k[] = "CENSORED";
 char c, p, t = 0;
 int i = 0;
 while ((p = fgetc(input)) != EOF) {
     c = (p + (k[i % strlen(k)] ^ t) + i*i) & 0xff;
     t = p;
     i++;
     fputc(c, output);
 }
Find the key:

[*] Algorithm coverts one byte of plain text to one byte of cipher text using equation of the form c = (p + (k[i % len(k)] ^ t) + i*i) mod 256
[*] Using the plain text - cipher text combination, the equation can be written as k[i] = ((c[i] - (i*i) - p[i]) ^ t) & 0xff to find the key. Here c,i,p and t are known values

The key used is VeryLongKeyYouWillNeverGuess. Once the key is found, the decryption algorithm is straight forward. Below is the code
#!/usr/bin/env python

plain_text = open('msg001','r').read().strip()
cipher_text = open('msg001.enc','r').read().strip()

plain_text = [ord(i) for i in plain_text]
cipher_text = [ord(i) for i in cipher_text]

t = 0
key = ''

for i in range(len(plain_text)):
    c = ((cipher_text[i] - (i*i) - plain_text[i]) ^ t) & 0xff
    key += chr(c)
    t = plain_text[i]
#print key

cipher_text = open('msg002.enc','r').read().strip()
key = 'VeryLongKeyYouWillNeverGuess'

key= [ord(i) for i in key]
cipher_text = [ord(i) for i in cipher_text]

t = 0
plain = ''

for i in range(len(cipher_text)):
    c = (cipher_text[i] - (key[i % len(key)] ^ t) - i*i) & 0xff
    plain += chr(c)
    t = c
print plain
Flag for the challenge is CTF{6d5eba48508efb13dc87220879306619}

Friday, October 19, 2012

Hack You CTF - Crypto 300 [Team xbios]

We were given the python source code of the crpto service. The flag is read from a file and used as the salt for S-box. Also during encryption the S-box is once again shuffled with a part of the user input. So to grab the key we have to do the reverse process. First we find the current S-box using printables and non printables from ascii range of 0-127. Then revert back the effect in S-box due to user input. This will give the server copy of the salted S-box. From this copy we have to find the key which is our flag.
Here is the part of code which we used to grab the server copy of salted S-box. Not the best code to read but it worked for us
#!/usr/bin/python

import string
import collections
from socket import *

data_print = string.printable
data_notprint = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x7f"
JUNK = "aa"
NEW_LINE = "\n"
HOST = "127.0.0.1" 
HOST = "93.191.13.142"
PORT = 7777
sbox = [0] * 128

soc = socket(AF_INET,SOCK_DGRAM)
soc.connect((HOST,PORT))

for i in data_print:
    data = JUNK + i + NEW_LINE
    soc.sendto(data, (HOST, PORT))
    f_byte = soc.recvfrom(128)[0][:2] #fetch first byte
    val = ord(f_byte.decode("hex"))
    sbox[int(ord(i))] = val
for i in data_notprint:
    data =JUNK + i + NEW_LINE
    soc.sendto(data, (HOST, PORT))
    f_byte = soc.recvfrom(128)[0][:2] #fetch first byte
    val = ord(f_byte.decode("hex"))
    sbox[int(ord(i))] = val

print "\n[*] Reconstructed LEVEL ONE SBOX"
print sbox

for j in xrange(len(sbox)):  #remove the effect in S-box due to user input
    sbox[j] = (sbox[j] - 1) % 128
sbox[ord('a')],sbox[1] = sbox[1],sbox[ord('a')]
for k in xrange(len(sbox)):
    sbox[k] = (sbox[k] - 1) % 128
sbox[ord('a')],sbox[0] = sbox[0],sbox[ord('a')]

print "\n[*] Reconstructed LEVEL TWO SBOX"
print sbox 
This is the salted S-box from the server:
[86, 3, 13, 122, 14, 2, 75, 28, 29, 5, 77, 19, 34, 6, 74, 8, 83, 38, 127, 41, 40, 15, 1, 31, 89, 88, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 46, 32, 76, 36, 78, 79, 80, 81, 82, 42, 84, 85, 26, 87, 51, 50, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 43, 123, 124, 125, 126, 44, 0, 48, 49, 27, 4, 35, 39, 7, 45, 9, 10, 11, 12, 33, 30, 47, 16, 17, 18, 37, 20, 21, 22, 23, 24, 25]
Once we got this, we have to calculate the key length being used. The server code does a mod operation over the S-box list for each character of key, this actually rotates the S-box as per key length. So from each element in list, subracting its index, we found that most elements differed by 26, which is actually our key length. A simple rotation with key length will show how it works.
[0, 48, 49, 27, 4, 35, 39, 7, 45, 9, 10, 11, 12, 33, 30, 47, 16, 17, 18, 37, 20, 21, 22, 23, 24, 25, 86, 3, 13, 122, 14, 2, 75, 28, 29, 5, 77, 19, 34, 6, 74, 8, 83, 38, 127, 41, 40, 15, 1, 31, 89, 88, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 46, 32, 76, 36, 78, 79, 80, 81, 82, 42, 84, 85, 26, 87, 51, 50, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 43, 123, 124, 125, 126, 44]
As we can see most elements are in its respective positions when the salted S-box is rotated with key length. Next task is to find the key, this is where we got stuck. We couldn't figure out the exact way to find the flag from the available data. Finally we came up with a code which found keys when they are smaller and doesn't have any repeated characters. Heres is the part of the code:
sboxes=[]
for i in range(max_len-1, -1, -1):
    for j in xrange(len(sbox)):
        sbox[j] = (sbox[j] - 1) % 128
    ret = sbox[i] - i
    print "\n[*] SBOX Stages: ",i
    print sbox  
    sboxes.append(sbox[::])   
    sbox[ret],sbox[i] = sbox[i],sbox[ret]

flag = ''
for index,sbox in enumerate(sboxes[::-1]):
    num = sbox[index]-index
    if num > 31 and num < 127:
        flag += chr(num)
The key we got from server is <i**t*1**k3**l**9*e***g*?> ('*' represents unfound characters). Also the stage 0 of reconstructed S-box gives clue of repeating characters. Characters other than repeating ones were in place
[60, 1, 2, 3, 4, 5, 6, 115, 96, 9, 10, 11, 12, 13, 14, 15, 16, 8, 18, 110, 48, 21, 22, 104, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 20, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 0, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 17, 97, 98, 99, 100, 101, 102, 103, 23, 105, 106, 107, 108, 109, 19, 111, 112, 113, 114, 7, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127]

>>> len("<i**t*1**k3**l**9*e***g*?>")
26
>>> chr(115)
's'
>>> chr(96)
'`'
>>> chr(110)
'n'
>>> chr(48)
'0'
>>> chr(104)
'h'
We have partial key and some other data to work with. Then our teammate rdy started looking into this with his Beautiful Mind eyes, hunting for combinations and sensible meanings. We soon figured out the key: <is`th1s`k3y`l0n9`en0ugh?>

Hack You CTF - Reverse 300 [Team xbios]

We were given a binary which was packed. "strings" revealed something like "This file is packed with the LOL executable packer http://upx.sf.net". Unpacking with UPX didn't help. But strace gave the following info
[ctf@renorobert rev300]# strace ./task3.bin 
execve("./task3.bin", ["./task3.bin"], [/* 52 vars */]) = 0
[ Process PID=5086 runs in 32 bit mode. ]
getpid()                                = 5086
gettimeofday({1350641971, 320063}, NULL) = 0
unlink("/tmp/upxDHV4WSUAIXC")           = -1 ENOENT (No such file or directory)
open("/tmp/upxDHV4WSUAIXC", O_RDWR|O_CREAT|O_EXCL, 0700) = 3
ftruncate(3, 9036)                      = 0
old_mmap(NULL, 9036, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = 0xfffffffff770a000
old_mmap(0xf770d000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfffffffff770d000
munmap(0xf770a000, 9036)                = 0
close(3)                                = 0
open("/tmp/upxDHV4WSUAIXC", O_RDONLY)   = 3
getpid()                                = 5086
access("/proc/5086/fd/3", R_OK|X_OK)    = 0
unlink("/tmp/upxDHV4WSUAIXC")           = 0
The binary unpacks itself into /tmp before gettting executed. So the idea was to grab a copy of it. Using objdump we found an entry point into the packed binary. By running the binary in gdb and setting necessary breakpoints, we got the copy of unpacked binary from /tmp before the file is unlinked.
[ctf@renorobert rev300]# objdump -f task3.bin 

task3.bin:     file format elf32-i386
architecture: i386, flags 0x00000102:
EXEC_P, D_PAGED
start address 0x00402273
Once the binary was unpacked, it nicely decompiled into a readable source. Here is the part of the source that we got
void __cdecl sub_8048617(int a1, int a2)
{
  int v2; 
  int v3; 
  if ( a1 != 3 )
  {
    printf("Usage: %s <user> <key>\n", *(_DWORD *)a2);
    exit(1);
  }
  if ( sub_8049A30(*(_DWORD *)(a2 + 4), "hackyou") )
  {
    v3 = *(_DWORD *)(a2 + 4);
    v2 = *(_DWORD *)(a2 + 8);
    if ( sub_8049A00(v2) != 14 )   // length of key checked here
      sub_80485C9();
    if ( *(_BYTE *)(v2 + 4) == 45 )   // key looks like ABCD-ABCD-ABCD
    {
      if ( *(_BYTE *)(v2 + 9) == 45 )
      {
        if ( !sub_804838C(v3, v2) )   //function call to check 1st part of key
          sub_80485C9();
        if ( !sub_804844B(v3, v2 + 5) )   //function call to check 2nd part of key
          sub_80485C9();
        if ( !sub_804850A(v3, v2 + 10) )  //function call to check 3rd part of key
          sub_80485C9();
        sub_80485F0();
      }
    }
    sub_80485C9();
  }
  printf("Err, something went wrong...\n");
  exit(2);
}
signed int __cdecl sub_804838C(int a1, int a2)  //first part of key is checked with this function
{
  int v2; 
  signed int i; 
  char v5[64];
  int v6; 
  int v7; 
  int v8; 
  int v9; 
  v6 = 23;
  v7 = 13;
  v8 = 34;
  v9 = 40;
  v2 = sub_8049A00(a1);
  sub_8048DC4(a1, v2, (int)v5, 64);
  for ( i = 0; i < 4; ++i )
  {
    if ( sub_804831C(v5[*(&v6 + i)]) != *(_BYTE *)(i + a2) ) //key is checked here
      return 0;
  }
  return 1337;
}
After this, the process is straight forward. With gdb and junk input as key, we started tracing through the function calls setting up breakpoints and changing $eip's.
//Check for first part of key
   0x8048418: call   0x804831c 
   0x804841d: add    $0x4,%esp
   0x8048420: mov    0xc(%ebp),%ecx
   0x8048423: mov    -0x54(%ebp),%edx
   0x8048426: add    %edx,%ecx
   0x8048428: movsbl (%ecx),%edx
   0x804842b: cmp    %edx,%eax //key is checked here
   0x804842d: je     0x804843d
   0x8048433: mov    $0x0,%eax
   0x8048438: jmp    0x8048449
   0x804843d: jmp    0x80483f1
   0x804843f: mov    $0x539,%eax
   0x8048444: jmp    0x8048449
   0x8048449: leave  
   0x804844a: ret    

(gdb) p $eax
$4 = 107
(gdb) p $eax
$5 = 101
(gdb) p $eax
$6 = 99
(gdb) p $eax
$9 = 99
This way, we followed the three function calls and extracted the final key kecc-hack-yo0u