XPM Challenge - Cyber Odyssey 2025 Finals

XPM Challenge - Cyber Odyssey 2025 Finals

December 26, 2025

I am incredibly proud to have participated in the Cyber Odyssey 2025 Finals. A huge thanks to the organizers for putting together such a high-caliber event; the challenges were thoughtfully designed and offered a fantastic learning environment.

Our team, !BTATABNINA, secured 1st place and took the grand prize of $8,000 USD! While we had a successful run, I didn’t manage to solve the “XPM” reverse engineering challenge during the heat of the competition, so I’ve spent time upsolving it post-event to fully understand the logic behind it.

Attachments


1) Overview

The challenge binary reads a hidden XPM file (flag.xpm), generates two intermediate artifacts (header.bin and pixels.bin), and then runs an obfuscated VM. My goal was to reproduce the final image without executing the binary. I did this by:

  • Decrypting header.bin with the XXTEA VM.
  • Reversing the second VM to decode pixels.bin.
  • Reassembling the final pixel grid using the recovered palette.

2) Tooling

  • IDA Pro: Decompiler, xrefs, and data extraction.
  • Python 3: Key derivation, XXTEA decryption, pixel decoding.
  • Pillow (PIL): Image assembly and PNG output.
  • CLI helpers: file, strings, xxd to get quick context.

3) Files and Artifacts

chall        # static ELF (stripped) - was given
header.bin   # 5920 bytes (encrypted palette + metadata) - was given
pixels.bin   # 311812 bytes (obfuscated pixels) - was given
recovered.png  # final reconstructed image (137x569) - recovered flag

4) Initial Recon and Entry Point

I started with quick recon:

  • file chall shows a statically linked, stripped 64‑bit ELF.
  • strings chall revealed helpful hints: “Saved header.bin”, “pixels.bin”, and “Don’t debug me!”.

Next, I opened chall in IDA and followed the entrypoint. The _start stub calls __libc_start_main, passing a function at sub_401810 as the program entry (i.e., main).

Entry-point snippet:

401e71  mov     rdi, offset sub_401810
401e78  call    sub_403200   ; __libc_start_main

Inside sub_401810, there’s a ptrace anti-debug check (the “Don’t debug me!” string is referenced here), but since I was not executing the binary, I simply noted it and moved on.


5) High-level Binary Behavior

After confirming sub_401810 is the main function, I traced its control flow. The structure is:

  1. Read and parse XPM data into a qword_649340‑style structure.
  2. Derive a 4‑word key (calls to sub_402DF0, sub_402910, sub_402D60, sub_402880).
  3. Invoke sub_402A80 to generate header.bin.
  4. Run sub_401F80 to decode pixel data (pixels.bin).

Key disassembly excerpt:

401919  call    sub_402DF0
40191e  mov     ebx, eax
401920  call    sub_402910
401925  mov     r11d, eax
401928  call    sub_402D60
40192d  mov     r10d, eax
401930  call    sub_402880
401953  call    sub_402A80
40199d  call    sub_401F80

This gave me the two targets to reverse:

  • sub_402A80 → header encryption/decryption
  • sub_401F80 → pixel decoding

6) VM #1: XXTEA Decryptor (sub_402A80)

sub_402A80 is a bytecode VM. The bytecode is stored at 0x4E6520 (length 0x400). Dumping the bytes and searching for constants revealed the TEA delta 0x9E3779B9 in little-endian form (b9 79 37 9e).

That strongly suggested XXTEA. I confirmed the VM’s core math matched the standard XXTEA formula:

mx = (((z >> 5) ^ (y << 2)) + ((y >> 3) ^ (z << 4))) ^
     ((sum ^ y) + (k[(p & 3) ^ e] ^ z));

I verified the VM behavior by reimplementing XXTEA in Python and matching outputs for random test vectors.

IDA Pro decompilation (sub_402A80)

unsigned __int64 __fastcall sub_402A80(__int64 a1, int a2, __int64 a3, double a4)
{
  char *v4; // rdi
  int *v5; // rsi
  unsigned __int8 v6; // al
  int v7; // eax
  int v8; // eax
  __int64 v9; // rax
  int *v10; // rax
  int v11; // edx
  __int64 v12; // rax
  int v13; // eax
  bool v14; // al
  bool v15; // al
  unsigned __int64 result; // rax
  __int64 v17; // rax
  __int64 v18; // [rsp+8h] [rbp-180h] BYREF
  int v19; // [rsp+14h] [rbp-174h] BYREF
  __int64 v20; // [rsp+18h] [rbp-170h] BYREF
  _BYTE v21[312]; // [rsp+20h] [rbp-168h] BYREF
  unsigned __int64 v22; // [rsp+158h] [rbp-30h]

  v20 = a1;
  v4 = (char *)&unk_4E6520;
  v19 = a2;
  v5 = (int *)v21;
  v18 = a3;
  v22 = __readfsqword(0x28u);
LABEL_2:
  while ( 2 )
  {
    v6 = *v4;
    while ( 1 )
    {
      do
      {
        while ( 1 )
        {
LABEL_3:
          if ( v6 == 0xFF )
          {
            while ( 1 )
              ;
          }
          if ( v6 <= 0x70u )
            break;
          switch ( *v4 )
          {
            case 113:
            case -66:
              v7 = *v5 - *(v5 - 2);
              ++v4;
              v5 -= 2;
              *v5 = v7;
              goto LABEL_2;
            case 122:
              ++v4;
              *(_QWORD *)v5 = (unsigned int)*v5;
              goto LABEL_2;
            case 125:
            case -71:
              ++v4;
              *v5 = **(_DWORD **)v5;
              goto LABEL_2;
            case -128:
              ++v4;
              *(v5 - 2) <<= *v5;
              v5 -= 2;
              goto LABEL_2;
            case -124:
              ++v4;
              *((_QWORD *)v5 - 1) += *(_QWORD *)v5;
              v5 -= 2;
              goto LABEL_2;
            case -103:
              goto LABEL_10;
            case -100:
              v9 = *(_QWORD *)v5 * *((_QWORD *)v5 - 1);
              ++v4;
              v5 -= 2;
              *(_QWORD *)v5 = v9;
              goto LABEL_2;
            case -97:
              v8 = *(_DWORD *)(v4 + 1);
              switch ( v8 )
              {
                case 1:
                  *((_QWORD *)v5 + 1) = &v19;
                  break;
                case 2:
                  *((_QWORD *)v5 + 1) = &v18;
                  break;
                case 0:
                  *((_QWORD *)v5 + 1) = &v20;
                  break;
              }
              goto LABEL_7;
            case -89:
              v10 = *(int **)v5;
              v11 = *(v5 - 2);
              ++v4;
              v5 -= 4;
              *v10 = v11;
              goto LABEL_2;
            case -64:
              ++v4;
              *(_QWORD *)v5 = *v5;
              goto LABEL_2;
            case -56:
              ++v4;
              v15 = *v5 > *(v5 - 2);
              v5 -= 2;
              *v5 = v15;
              goto LABEL_2;
            case -55:
              ++v4;
              v14 = *(v5 - 2) < (unsigned int)*v5;
              v5 -= 2;
              *v5 = v14;
              goto LABEL_2;
            case -48:
              ++v4;
              v13 = *v5 / *(v5 - 2);
              v5 -= 2;
              *v5 = v13;
              goto LABEL_2;
            case -26:
              v12 = *(int *)(v4 + 1);
              v5 += 2;
              v4 += 5;
              *(_QWORD *)v5 = &v21[v12 + 256];
              goto LABEL_2;
            case -22:
              goto LABEL_6;
            case -8:
            case -2:
              ++v4;
              goto LABEL_2;
            default:
              continue;
          }
        }
      }
      while ( v6 > 0x5Fu );
      if ( v6 > 0x29u )
        break;
      switch ( v6 )
      {
        case 7u:
          v4 += *(int *)(v4 + 1) + 1;
          goto LABEL_2;
        case 0x1Au:
          v17 = *(_QWORD *)(v4 + 1);
          v5 += 2;
          v4 += 9;
          *(_QWORD *)v5 = v17;
          goto LABEL_2;
        case 4u:
LABEL_10:
          ++v4;
          *(v5 - 2) += *v5;
          v5 -= 2;
          goto LABEL_2;
      }
    }
    switch ( *v4 )
    {
      case '*':
        ++v4;
        *(v5 - 2) ^= *v5;
        v5 -= 2;
        continue;
      case ':':
        ++v4;
        *(_QWORD *)v5 = **(_QWORD **)v5;
        continue;
      case 'Y':
        ++v4;
        *(v5 - 2) = (unsigned int)*v5 >> *(v5 - 2);
        goto LABEL_36;
      case 'Z':
LABEL_6:
        v5[2] = *(_DWORD *)(v4 + 1);
LABEL_7:
        v5 += 2;
        v4 += 5;
        continue;
      case '[':
        result = __readfsqword(0x28u) ^ v22;
        if ( result )
          sub_462F80(v4, v5, a4);
        return result;
      case '^':
        ++v4;
        *(v5 - 2) &= *v5;
        v5 -= 2;
        continue;
      case '_':
        if ( *v5 )
          v4 += *(int *)(v4 + 1) + 1;
        else
          v4 += 5;
LABEL_36:
        v5 -= 2;
        continue;
      default:
        goto LABEL_3;
    }
  }
}

7) Key Derivation Functions

The XXTEA key is derived by four deterministic functions. I reimplemented each function exactly as IDA decompiled them, then combined their outputs:

key = [
  0xcf522dcc,
  0x40450656,
  0x9f0f2ba6,
  0x83636502
]

These 4 dwords are used directly as the XXTEA key (little‑endian order).


8) Decrypting header.bin

header.bin is 5920 bytes → 1480 dwords.

I used standard XXTEA decryption:

DELTA = 0x9e3779b9

def mx(z,y,sum_,k,p,e):
    return (((z>>5) ^ (y<<2)) + ((y>>3) ^ (z<<4))) ^ \
           ((sum_ ^ y) + (k[(p & 3) ^ e] ^ z))

def xxtea_decrypt(v,k):
    v=v.copy()
    n=len(v)
    rounds = 6 + 52//n
    sum_ = (rounds * DELTA) & 0xffffffff
    y = v[0]
    while sum_:
        e = (sum_ >> 2) & 3
        for p in range(n-1,0,-1):
            z = v[p-1]
            v[p] = (v[p] - mx(z,y,sum_,k,p,e)) & 0xffffffff
            y = v[p]
        z = v[n-1]
        v[0] = (v[0] - mx(z,y,sum_,k,0,e)) & 0xffffffff
        y = v[0]
        sum_ = (sum_ - DELTA) & 0xffffffff
    return v

The decrypted output is structured and includes the palette.


9) Header Layout and Palette Recovery

The decrypted header begins with four dwords:

width  = 569
height = 137
ncolors = 164
cpp = 2

Colors are stored in ncolors entries of size 0x24 bytes, starting at offset 0x10.

  • entry[0:cpp] = XPM symbol (ignored for reconstruction)
  • entry[4:4+32] = ASCII color string (e.g., #7B6DBE)

Example extraction:

for i in range(ncolors):
    off = 0x10 + i*0x24
    color = pt[off+4:off+4+32].split(b"\x00",1)[0].decode("ascii")

I used these color strings to build:

  1. palette[index] = (r, g, b)
  2. hash_to_idx[djb2(color_string)] = index

10) VM #2: Pixel Decoder (sub_401F80)

sub_401F80 is a second bytecode interpreter with a stack-based VM. The bytecode lives at 0x4E6120 (length 0x400). I mapped opcodes by aligning the bytecode with the decompiled interpreter logic.

Key VM behaviors:

  • djb2 hash used to map color strings to indices
  • LCG RNG (same constants as libc rand()):
seed = (1103515245 * seed + 12345) & 0x7fffffff
  • Fisher‑Yates shuffle of indices

The djb2 hash appears explicitly in the VM call handler:

v38 = 5381;
while (*v53) {
    v38 = v38*33 + *v53;
    ++v53;
}

This confirmed the palette hash mapping logic.

IDA Pro decompilation (sub_401F80)

unsigned __int64 __fastcall sub_401F80(__int64 a1, __int64 a2, __int64 a3, __int64 a4, int a5, int a6)
{
  unsigned __int8 *v6; // rbp
  __m128i si128; // xmm0
  char *v8; // rbx
  __int64 v9; // rdi
  unsigned __int8 v10; // al
  __int64 v11; // rsi
  __int64 v12; // rax
  int v13; // eax
  _DWORD *v14; // rax
  _QWORD *v15; // rax
  char *v16; // rax
  int v17; // eax
  bool v18; // al
  unsigned __int64 result; // rax
  bool v20; // al
  __int64 *v21; // rax
  int v22; // eax
  _DWORD *v23; // rax
  __int64 v24; // rax
  int v25; // ecx
  __int64 v26; // rdi
  __int64 v27; // rsi
  __m128i v28; // xmm1
  __int64 v29; // rax
  __int64 v30; // rdx
  __m128i v31; // xmm2
  signed int v32; // eax
  int v33; // eax
  unsigned int v34; // ecx
  _DWORD *v35; // rax
  __int64 v36; // rax
  int v37; // eax
  int v38; // ecx
  _BYTE *v39; // rsi
  int v40; // eax
  char v41[264]; // [rsp+0h] [rbp-228h] BYREF
  int v42; // [rsp+108h] [rbp-120h]
  __int64 v43; // [rsp+110h] [rbp-118h]
  __int64 v44; // [rsp+120h] [rbp-108h]
  __int64 v45; // [rsp+130h] [rbp-F8h]
  int v46; // [rsp+138h] [rbp-F0h]
  __int64 v47; // [rsp+140h] [rbp-E8h]
  __int64 v48; // [rsp+148h] [rbp-E0h]
  __int64 v49; // [rsp+158h] [rbp-D0h]
  __int64 v50; // [rsp+190h] [rbp-98h]
  __int64 v51; // [rsp+198h] [rbp-90h]
  int v52; // [rsp+1A4h] [rbp-84h]
  _BYTE *v53; // [rsp+1A8h] [rbp-80h]
  int v54; // [rsp+1B4h] [rbp-74h]
  int v55; // [rsp+1C0h] [rbp-68h]
  __int64 v56; // [rsp+1D0h] [rbp-58h]
  __int64 v57; // [rsp+1D8h] [rbp-50h]
  __int64 v58; // [rsp+1E0h] [rbp-48h]
  __int64 v59; // [rsp+1E8h] [rbp-40h]
  __int64 v60; // [rsp+1F0h] [rbp-38h]
  unsigned __int64 v61; // [rsp+1F8h] [rbp-30h]

  v6 = (unsigned __int8 *)&unk_4E6120;
  si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
  v61 = __readfsqword(0x28u);
  v8 = v41;
  while ( 2 )
  {
    while ( 1 )
    {
      v9 = *v6;
      v10 = *v6;
      v11 = (unsigned __int8)(v9 - 47);
LABEL_3:
      if ( v10 > 0x15u )
        break;
      if ( !v10 )
      {
        while ( 1 )
          ;
      }
      switch ( v10 )
      {
        case 0u:
        case 2u:
        case 4u:
        case 5u:
        case 6u:
        case 7u:
        case 8u:
        case 9u:
        case 0xAu:
        case 0xBu:
        case 0xCu:
        case 0xDu:
        case 0xEu:
        case 0xFu:
        case 0x10u:
        case 0x11u:
        case 0x13u:
          goto LABEL_3;
        case 1u:
          v22 = *(_DWORD *)v8 * *((_DWORD *)v8 - 2);
          ++v6;
          v8 -= 8;
          *(_DWORD *)v8 = v22;
          continue;
        case 3u:
          ++v6;
          *((_DWORD *)v8 - 2) ^= *(_DWORD *)v8;
          v8 -= 8;
          continue;
        case 0x12u:
          v23 = *(_DWORD **)v8;
          LODWORD(a3) = *((_DWORD *)v8 - 2);
          ++v6;
          v8 -= 16;
          *v23 = a3;
          continue;
        case 0x14u:
          switch ( *(_DWORD *)(v6 + 1) )
          {
            case 1:
              dword_4E8990 = v42;
              goto LABEL_17;
            case 2:
              v24 = sub_42F880(v43, v11, *(double *)si128.m128i_i64);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              v44 = v24;
              goto LABEL_17;
            case 3:
              v25 = v46;
              v26 = v45;
              LODWORD(v27) = v46 - 1;
              if ( v46 <= 0 )
                goto LABEL_67;
              if ( (unsigned int)v27 <= 2 )
              {
                v32 = 0;
              }
              else
              {
                v28 = _mm_load_si128((const __m128i *)&xmmword_4B9750);
                v29 = v45;
                v30 = v45 + 16LL * ((unsigned int)v46 >> 2);
                do
                {
                  v31 = v28;
                  v29 += 16LL;
                  v28 = _mm_add_epi32(v28, si128);
                  *(__m128i *)(v29 - 16) = v31;
                }
                while ( v30 != v29 );
                v32 = v25 & 0xFFFFFFFC;
                if ( (v25 & 3) == 0 )
                  goto LABEL_55;
              }
              *(_DWORD *)(v26 + 4LL * v32) = v32;
              a3 = 4LL * v32;
              a5 = v32 + 1;
              if ( v25 <= v32 + 1 )
              {
LABEL_67:
                if ( (int)v27 <= 0 )
                  goto LABEL_17;
              }
              else
              {
                v33 = v32 + 2;
                *(_DWORD *)(v26 + a3 + 4) = a5;
                if ( v25 > v33 )
                  *(_DWORD *)(v26 + a3 + 8) = v33;
              }
LABEL_55:
              v34 = dword_4E8990;
              v27 = (int)v27;
              do
              {
                a5 = v27 + 1;
                a6 = *(_DWORD *)(v26 + 4 * v27);
                v34 = (1103515245 * v34 + 12345) & 0x7FFFFFFF;
                v35 = (_DWORD *)(v26 + 4LL * (v34 % ((int)v27 + 1)));
                LODWORD(a3) = *v35;
                *(_DWORD *)(v26 + 4 * v27--) = *v35;
                *v35 = a6;
              }
              while ( (int)v27 > 0 );
              dword_4E8990 = v34;
LABEL_17:
              v6 += 5;
              break;
            case 4:
              v36 = sub_4219C0(v47, v48);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              v49 = v36;
              goto LABEL_17;
            case 5:
              v37 = sub_401140(v50, v51, *(double *)si128.m128i_i64);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              v52 = v37;
              goto LABEL_17;
            case 6:
              v38 = 5381;
              v39 = v53 + 1;
              v40 = (char)*v53;
              if ( *v53 )
              {
                do
                {
                  ++v39;
                  LODWORD(a3) = 32 * v38;
                  v38 += 32 * v38 + v40;
                  v40 = (char)*(v39 - 1);
                }
                while ( *(v39 - 1) );
              }
              v54 = v38;
              goto LABEL_17;
            case 7:
              dword_4E8990 = (1103515245 * dword_4E8990 + 12345) & 0x7FFFFFFF;
              v55 = dword_4E8990;
              goto LABEL_17;
            case 8:
              sub_421AC0(v56, 4LL, 1LL, v57);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              goto LABEL_17;
            case 9:
              sub_421320(v58);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              goto LABEL_17;
            case 0xA:
              sub_42FE70(v59);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              goto LABEL_17;
            case 0xB:
              sub_411AE0(v60, v11, a3, v9 - 47, a5, a6, v41[0]);
              si128 = _mm_load_si128((const __m128i *)&xmmword_4B9760);
              goto LABEL_17;
            default:
              goto LABEL_17;
          }
          break;
        case 0x15u:
          if ( *(_DWORD *)v8 )
            v6 += *(int *)(v6 + 1) + 1;
          else
            v6 += 5;
LABEL_21:
          v8 -= 8;
          continue;
        default:
          goto LABEL_4;
      }
    }
LABEL_4:
    switch ( (char)v9 )
    {
      case 47:
      case 103:
        v12 = *(_QWORD *)(v6 + 1);
        v8 += 8;
        v6 += 9;
        *(_QWORD *)v8 = v12;
        continue;
      case 51:
        ++v6;
        LODWORD(a3) = *(_DWORD *)v8 % *((_DWORD *)v8 - 2);
        v8 -= 8;
        *(_DWORD *)v8 = a3;
        continue;
      case 62:
        v21 = (__int64 *)*((_QWORD *)v8 - 1);
        a3 = *(_QWORD *)v8;
        ++v6;
        v8 -= 16;
        *v21 = a3;
        continue;
      case 69:
        ++v6;
        v20 = *(_DWORD *)v8 < *((_DWORD *)v8 - 2);
        v8 -= 8;
        *(_DWORD *)v8 = v20;
        continue;
      case 91:
      case -78:
        ++v6;
        continue;
      case 107:
        v17 = *(_DWORD *)(v6 + 1);
        if ( v17 )
        {
          if ( v17 == 1 )
            *((_QWORD *)v8 + 1) = &unk_4E9A40;
        }
        else
        {
          *((_QWORD *)v8 + 1) = &qword_649340;
        }
        goto LABEL_16;
      case 123:
      case -58:
        ++v6;
        *((_QWORD *)v8 - 1) += *(_QWORD *)v8;
        v8 -= 8;
        continue;
      case 126:
        v6 += *(int *)(v6 + 1) + 1;
        continue;
      case -113:
        ++v6;
        v18 = *((_DWORD *)v8 - 2) == *(_DWORD *)v8;
        v8 -= 8;
        *(_DWORD *)v8 = v18;
        continue;
      case -109:
        v15 = *(_QWORD **)v8;
        a3 = *((_QWORD *)v8 - 1);
        ++v6;
        v8 -= 16;
        *v15 = a3;
        continue;
      case -106:
        ++v6;
        *(_QWORD *)v8 = *(int *)v8;
        continue;
      case -103:
      case -55:
        ++v6;
        *(_DWORD *)v8 = **(_DWORD **)v8;
        continue;
      case -88:
        *((_DWORD *)v8 + 2) = *(_DWORD *)(v6 + 1);
        goto LABEL_16;
      case -83:
        ++v6;
        *((_DWORD *)v8 - 2) += *(_DWORD *)v8;
        v8 -= 8;
        continue;
      case -81:
        v14 = (_DWORD *)*((_QWORD *)v8 - 1);
        LODWORD(a3) = *(_DWORD *)v8;
        ++v6;
        v8 -= 16;
        *v14 = a3;
        continue;
      case -77:
        ++v6;
        *((_QWORD *)v8 - 1) *= *(_QWORD *)v8;
        goto LABEL_21;
      case -73:
        v16 = &off_4E6800[*(int *)(v6 + 1)];
        v8 += 8;
        v6 += 5;
        *(_QWORD *)v8 = v16;
        continue;
      case -32:
        ++v6;
        LODWORD(a3) = *(_DWORD *)v8 % *((_DWORD *)v8 - 2);
        v13 = *(_DWORD *)v8 / *((_DWORD *)v8 - 2);
        v8 -= 8;
        *(_DWORD *)v8 = v13;
        continue;
      case -14:
        *((_QWORD *)v8 + 1) = &v41[*(int *)(v6 + 1) + 256];
LABEL_16:
        v8 += 8;
        goto LABEL_17;
      case -13:
        ++v6;
        *(_QWORD *)v8 = **(_QWORD **)v8;
        continue;
      case -7:
        result = __readfsqword(0x28u) ^ v61;
        if ( result )
          sub_462F80(v9, v11, *(double *)si128.m128i_i64);
        return result;
      case -6:
        ++v6;
        *((_QWORD *)v8 - 1) += *(int *)v8;
        v8 -= 8;
        continue;
      default:
        goto LABEL_3;
    }
  }
}

11) Pixel Decoding Algorithm

I interpreted pixels.bin as an array of 32‑bit words:

N = len(pixels.bin) / 4 = 77953

Since:

77953 = 137 * 569

the pixel matrix is 137x569.

Seed initialization

The VM uses:

seed = N + ncolors = 77953 + 164 = 78117

Shuffle

perm = list(range(N))
for i in range(N-1, 0, -1):
    seed = (1103515245*seed + 12345) & 0x7fffffff
    j = seed % (i+1)
    perm[i], perm[j] = perm[j], perm[i]

Decode each pixel word

for i, w in enumerate(words):
    seed = (1103515245*seed + 12345) & 0x7fffffff
    h = (w ^ seed) & 0xffffffff
    idxs[perm[i]] = hash_to_idx[h]

At this point, idxs is a palette index array of length N.


12) Image Reconstruction

The VM uses column‑major ordering, so reconstruction uses:

row = j % height
col = j // height
pixel[row * width + col] = idxs[j]

Where:

width = 137
height = 569

Mapping each index to the RGB palette yields to the image recovered.png.


13) Reproduction Scripts

Below is the full combined script I used to reproduce the image end‑to‑end:

import struct
from PIL import Image

def rol32(v,r):
    return ((v<<r)&0xffffffff) | ((v&0xffffffff)>>(32-r))

def sub_402880():
    v0=0; v1=0; v2=0; v3=0; v4=(0xffffffff & -559038737)
    while True:
        while True:
            v5=rol32(v4,13)
            v6=(74565 * ((v5+v3)&0xffffffff)) & 0xffffffff
            v7=(v6 ^ v2) & 0xffffffff
            v8=(v1 + ((v7<<7)&0xffffffff)) & 0xffffffff
            v1=(v8 ^ v5) & 0xffffffff
            v9=(v6 - (v7>>3)) & 0xffffffff
            v3=(v7 + 5*v8) & 0xffffffff
            v2=(v8 ^ ((v8 ^ v5)<<11)) & 0xffffffff
            if v0 & 1: break
            v0+=1
            v4=(v9 + (v9 ^ v2) + 1) & 0xffffffff
            if v0==1330: return (16716599*v4) & 0xffffffff
        v0+=1
        v4=((v9 ^ (v3+v1)) + 1) & 0xffffffff
        if v0==1330: return (16716599*v4) & 0xffffffff

def sub_402910():
    v0=0; v1=0; v2=0; v3=0; v4=263171840 & 0xffffffff
    while True:
        while True:
            v5=rol32(v4,13)
            v6=(74565 * ((v5+v3)&0xffffffff)) & 0xffffffff
            v7=(v6 ^ v2) & 0xffffffff
            v8=(v1 + ((v7<<7)&0xffffffff)) & 0xffffffff
            v1=(v8 ^ v5) & 0xffffffff
            v9=(v6 - (v7>>3)) & 0xffffffff
            v3=(v7 + 5*v8) & 0xffffffff
            v2=(v8 ^ ((v8 ^ v5)<<10)) & 0xffffffff
            if v0 & 1: break
            v0+=1
            v4=(v9 + (v9 ^ v2) + 1) & 0xffffffff
            if v0==10000: return (16716599*v4) & 0xffffffff
        v0+=1
        v4=((v9 ^ (v3+v1)) + 1) & 0xffffffff
        if v0==10000: return (16716599*v4) & 0xffffffff

def sub_402D60():
    v0=0; v1=0; v2=0; v3=0; v4=(-559030611) & 0xffffffff
    while True:
        while True:
            v5=rol32(v4,13)
            v6=(74565 * ((v5+v3)&0xffffffff)) & 0xffffffff
            v7=(v6 ^ v2) & 0xffffffff
            v8=(v1 + ((v7<<7)&0xffffffff)) & 0xffffffff
            v1=(v8 ^ v5) & 0xffffffff
            v9=(v6 - (v7>>4)) & 0xffffffff
            v3=(v7 + 5*v8) & 0xffffffff
            v2=(v8 ^ ((v8 ^ v5)<<11)) & 0xffffffff
            if v0 & 1: break
            v0+=1
            v4=(v9 + (v9 ^ v2) + 1) & 0xffffffff
            if v0==10000: return (16716599*v4) & 0xffffffff
        v0+=1
        v4=((v9 ^ (v3+v1)) + 1) & 0xffffffff
        if v0==10000: return (16716599*v4) & 0xffffffff

def sub_402DF0():
    v0=0; v1=0; v2=0; v3=0; v4=(-889275714) & 0xffffffff
    while True:
        while True:
            v5=rol32(v4,13)
            v6=(74565 * ((v5+v3)&0xffffffff)) & 0xffffffff
            v7=(v6 ^ v2) & 0xffffffff
            v8=(v1 + ((v7<<7)&0xffffffff)) & 0xffffffff
            v1=(v8 ^ v5) & 0xffffffff
            v9=(v6 - (v7>>3)) & 0xffffffff
            v3=(v7 + 5*v8) & 0xffffffff
            v2=(v8 ^ ((v8 ^ v5)<<11)) & 0xffffffff
            if v0 & 1: break
            v0+=1
            v4=(v9 + (v9 ^ v2) + 1) & 0xffffffff
            if v0==10000: return (16716599*v4) & 0xffffffff
        v0+=1
        v4=((v9 ^ (v3+v1)) + 1) & 0xffffffff
        if v0==10000: return (16716599*v4) & 0xffffffff

key = [sub_402DF0(), sub_402910(), sub_402D60(), sub_402880()]

DELTA = 0x9e3779b9

def mx(z,y,sum_,k,p,e):
    return (((z>>5) ^ (y<<2)) + ((y>>3) ^ (z<<4))) ^ \
           ((sum_ ^ y) + (k[(p & 3) ^ e] ^ z))

def xxtea_decrypt(v,k):
    v=v.copy(); n=len(v)
    rounds = 6 + 52//n
    sum_ = (rounds * DELTA) & 0xffffffff
    y = v[0]
    while sum_:
        e = (sum_ >> 2) & 3
        for p in range(n-1,0,-1):
            z = v[p-1]
            v[p] = (v[p] - mx(z,y,sum_,k,p,e)) & 0xffffffff
            y = v[p]
        z = v[n-1]
        v[0] = (v[0] - mx(z,y,sum_,k,0,e)) & 0xffffffff
        y = v[0]
        sum_ = (sum_ - DELTA) & 0xffffffff
    return v

# Decrypt header
ct = open("header.bin","rb").read()
words = list(struct.unpack("<%dI" % (len(ct)//4), ct))
pt_words = xxtea_decrypt(words, key)
pt = struct.pack("<%dI" % len(pt_words), *pt_words)

ncolors = struct.unpack_from("<I", pt, 8)[0]

def djb2(s):
    h = 5381
    for c in s.encode("ascii"):
        h = (h*33 + c) & 0xffffffff
    return h

palette = []
hash_to_idx = {}
for i in range(ncolors):
    off = 0x10 + i*0x24
    color = pt[off+4:off+4+32].split(b"\x00",1)[0].decode("ascii")
    if color.startswith("#"):
        r = int(color[1:3],16)
        g = int(color[3:5],16)
        b = int(color[5:7],16)
    else:
        r=g=b=0
    palette.append((r,g,b))
    hash_to_idx[djb2(color)] = i

# Decode pixels.bin
b = open("pixels.bin","rb").read()
words = list(struct.unpack("<%dI" % (len(b)//4), b))
N = len(words)
seed = N + ncolors

# permutation
perm = list(range(N))
for i in range(N-1,0,-1):
    seed = (1103515245 * seed + 12345) & 0x7fffffff
    j = seed % (i+1)
    perm[i], perm[j] = perm[j], perm[i]

idxs = [0]*N
for i, w in enumerate(words):
    seed = (1103515245 * seed + 12345) & 0x7fffffff
    h = (w ^ seed) & 0xffffffff
    idxs[perm[i]] = hash_to_idx[h]

height = 569
width = 137
rgb = [None]*N
for j in range(N):
    row = j % height
    col = j // height
    rgb[row*width + col] = palette[idxs[j]]

img = Image.new("RGB", (width, height))
img.putdata(rgb)
img.save("recovered.png")

Result

The script above produces:

flag