Return Oriented Programming Series Introduction

This post is divided into two sections:

  1. Purpose
  2. Introduction to Return Oriented Programming (ROP)


Return Oriented Programming seems like arcane magic to many. It involves low-level understanding of systems, and can very quickly become overwhelming for the new eye. This series aims to teach ROP in a reader-friendly way. I will cover areas just enough so that you can get the gist of them, and it will be up to you, the reader, to dig in further and learn more.

You don’t need to learn the entire mechanics of swimming in order to swim, do you? A basic understanding is more than enough. Now, if you want to become a competitive or a good swimmer, you learn more about the different techniques used, exercises to do, etc. It’s the same with everything else. So, grab a coffee, tea, or whatever is your thing, sit back, relax, and enjoy.

The series will cover Capture The Flag (CTF) competitions, wargames, and real-world exploits. The aim is to teach the reader, via write-ups, how to exploit binaries by using ROP.

A surgeon doesn’t start his learning journey by opening up a human and figuring out what goes where. The future surgeon first learns about the body, its concepts. Then, she starts operating on frogs, mice, and other small animals, with the help of others. Eventually, she starts operating on actual humans with the assistance of others. Finally, she takes lead in performing surgery, while still receiving help.

I bring up surgeons as a way to demonstrate that all learning is the same. You will, at first, need a lot of assistance. Once you become proficient enough, you will be able to do things by yourself, although you might still need the help of others; be it through searching online or collaborating with someone. There’s nothing bad to it. This series is to help you out with the initial steps, with not too much hand-holding, but sufficient explanations.

Introduction to Return Oriented Programming

The simple idea:

Return Oriented Programming (ROP) or Return-To-Libc, is a binary exploitation technique where program flow is manipulated by utilizing available functions in order to achieve a particular goal. ROP is able to bypass security mechanisms such as a Non-Executable Stack due to the fact that it lives off the land, off what’s already available.

The details:

The name ROP is given because developers utilize a series of Assembly instructions which end with RET (return) that perform a particular operation and then transfer (or return) control to the return address located in the stack.

These series of Assembly instructions are called gadgets. Gadgets can be used by the developer in order to form a chain of commands that helps them achieve their goal. For example, if the developer was dealing with a 64-bit binary and wanted to overwrite the argument passed to system(), she would look for a POP RDI; RET; gadget.

If this all looks confusing, don’t worry! I will provide more details below.

The name Return-To-Libc is given because exploit developers utilize libc functions, such as system(), which are available to the binary, in order to overwrite return addresses and alter program flow. For example, the program might be using the fgets() function to do some operation, and we can overwrite the call to fgets() with a call to system(), so that when the program tries to call fgets(), it will actually call system().

Prerequisite Knowledge Reference:

The following are some concepts that are important to understand in order to do ROP. I will provide useful bits that you can reference back to when in need of help. That said, for a better understanding, look them up in your favorite search engine and learn more.


What follows are some useful bits that you can reference back to when reverse engineering or developing an exploit.


The stack is a Last-In-First-Out (LIFO) system, just like a real stack of plates or trays is.


If we look at the function read(int fd, void *buf, size_t count); then the file descriptor (fd) is the red plate, the buffer (buf) is the orange plate, and count is the yellow plate.

When the computer reads this function from the stack, it will grab (or pop) the yellow plate (count) before it grabs the orange plate (buf).

This will make even more sense when you think in terms of stack frames.

Stack Frames

Visualizing a stack frame will help more than just text. Below is the structure of a stack frame.

+———————-+ <- start of function’s stack frame
| Parameters    | <- function parameters
| …                  | <- there are as many blocks as there are parameters
| …                  |
| ret addr          | <- the return address; where the function returns
| frame ptr       | <- frame pointer; address of the stack pointer before the function is called
| variables        | <- function’s local variables

If we look at a real function, let’s say read(), then it will look like this:

+———————-+ <- start of read()’s stack frame
| size_t count  |
| void *buf      |
| int fd              |
| [ret address] |
| frame pointer |
| local vars       |

Visualizing stack frames in such a way is useful when using or overwriting a function. In fact, you can write your exploit script in such a way that your payload follows this structure; making it easier to understand what you’re doing.

System Calls

The best references for system calls are Linux Syscall Reference and the MAN command.

The former will not only show you the structure of the system call, but also link you to the MAN page.

Calling Conventions

32-bit and 64-bit systems differ in how function arguments are passed. Below a quick reference.


Parameter order: eax, ebx, ecx, edx, esi, edi, ebp

Where EAX contains the Syscall ID.


Parameter order: rdi, rsi, rdx, rcx, r8, r9

I recommend you make a virtual (and perhaps even physical) sticky note of the parameter order for both 32-bit and 64-bit. You will find yourself referencing back to it often, until you can recall it from memory.


Segments are portions of the virtual address space of a program. They contain different information and have different permissions, such as Read-Write-Allocate-Execute.

Let’s look at two segments, and the rest are up to you, the reader, to look up.

Data Segment:

  • Stores data (global variables)
  • Composed of: .data, .bss, .rodata
  • Permissions: read-write (RW), .rodata is read-only (RO)
  • Accessed through normal registers (eax, ebx, ecx, edx)

Text Segment:

  • Stores code
  • Read only (RO) and executable (X)
  • Instruction Pointer / Program Counter points to current instruction
  • Libraries possess code segment
  • Instruction pointer may jump to library code

Understanding segments is important in cases when you might want to write to memory. Knowing where you’re able to write and where is most reliable is critical to writing a reliable exploit.

Moving Forward

I will be expanding this post based on reader feedback. That may mean covering certain areas in more depth, introducing other concepts, and/or changing wording. Suggestions are always welcome.

One thought on “Return Oriented Programming Series Introduction”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s