I started studying software security, and I'm having trouble getting what buffer overflow attack and ROP attack are.
From what I understand is,
Buffer overflow attack:
When a buffer has a certain size, fill the buffer and an add additional code so that the attacker can execute another function in the code or his/her own shellcode.
ROP attack:
Give a certain input which can override the return address, so that the attacker can control the flow.
But what's the exact difference between the two?
I feel like both just give an excessive input to override the area which is not supposed to be approached.
For example, if I have a code of
1 #include <stdio.h>
2
3 void check(){
4 printf("overflow occurs!\n");
5 }
6
7 int main(int argc, char* argv[]){
8 char buffer[256];
9 gets(buffer);
10 printf("%s\n", buffer);
11 return 0;
12 }
and try to execute the function check() by giving a certain input to gets() function.
Is this a ROP attack or a buffer overflow attack?
A ROP attack is one kind of payload you can deliver via a buffer-overflow vulnerability, for buffers on the stack. (Overflowing other buffers could let you overwrite other data, e.g. in a struct or nearby other globals, but not take control of the program-counter.)
A buffer overflow is when incorrect bounds checking or handling of implicit-length data (e.g. strcpy or strcat) lets malicious input write memory past the end of an array. This gets interesting when the array was allocated on the call-stack, so one of the things following it is the return address of this function.
(In theory overwriting a static variable past the end of a static array could be useful as an exploit, and that would also be a buffer overflow. But usually a buffer overflow implies a buffer on the stack, allowing the attacker to control the return address. And thus to gain control of the instruction pointer.)
As well as a new return address, your malicious data will include more data which will be in memory below and above that return address. Part of this is the payload. Controlling the return address alone is usually not sufficient: in most processes there isn't anywhere you can jump to that (without other inputs) will execve a shell listening on a TCP port, for example.
Traditionally your payload would be machine-code ("shellcode"), and the return address would be the stack address where you knew that payload would land. (+- a NOP slide so you didn't have to get it exactly right).
Stack ASLR and non-executable stacks have made the traditional shellcode injection method of exploiting a buffer overflow impossible in normal modern programs. "Buffer overflow attack" used to (I think) imply shellcode injection, because there was no need to look for more complicated attacks. But that's no longer true.
A ROP attack is when the payload is a sequence of return addresses and data to be popped by pop instructions, and/or some strings like "/bin/sh". The first return address in the payload sends execution to some already-existing bytes at a known address in an executable page.
and try to execute the function
check()by giving a certain input togets()function.
The code for check() already exists in the target program, so the simplest attack would be a ROP attack.
This is the absolute simplest form of ROP attack, where code to do exactly what you want exists at a single known address, without needing any "function args". So it makes a good example to introduce the topic.
Is this a ROP attack or a buffer overflow attack?
It's both. It's a buffer overflow to inject a ROP payload.
If the program was compiled with -z execstack -no-pie, you could also choose to inject e.g. x86 shellcode that did mov eax, imm32 / jmp eax to jump to the known absolute address of check. In that case it would be a buffer overflow but not a ROP attack; it would be a code-injection attack.
(You might not call it "shellcode" because the purpose isn't to run a shell replacing the program, but rather to do something using the existing code of the program. But terminology is often used sloppily, so I think many people would call any injectable machine code "shellcode" regardless of what it does.)
Buffer overflow attack:
When a buffer has a certain size, fill the buffer and an add additional code so that the attacker can execute another function in the code or his/her own shellcode.
The "in the code" option would be a ROP attack. You point the return address at code which is already in memory.
The "or his/her own shellcode" option would be a code-injection attack. You point the return address at the buffer you just overflowed. (Either directly or via a ret2reg ROP attack to defeat stack ASLR, by looking for a jmp esp gadget on x86 for example.)
This "Buffer Overflow" definition is still slightly too narrow: it excludes overwriting some other critical variable (like bool user_authenticated) without overwriting a return address.
But yes, code injection and ROP attacks are the 2 main ways, with code injection normally made impossible by non-executable stack memory.