C program and subprocess

I wrote this simple C program to explain a more hard problem with the same characteristics.

#include <stdio.h>

int main(int argc, char *argv[])
{
    int n;
    while (1){
        scanf("%d", &n);
        printf("%d\n", n);
    }
    return 0;
}

and it works as expected.

I also wrote a subprocess script to interact with this program:

from subprocess import Popen, PIPE, STDOUT

process = Popen("./a.out", stdin=PIPE, stdout=PIPE, stderr=STDOUT)

# sending a byte
process.stdin.write(b'3')
process.stdin.flush()

# reading the echo of the number
print(process.stdout.readline())

process.stdin.close()

The problem is that, if I run my python script, the execution is freezed on the readline(). In fact, if I interrupt the script I get:

/tmp » python script.py
^CTraceback (most recent call last):
  File "/tmp/script.py", line 10, in <module>
    print(process.stdout.readline())
          ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

If I change my python script in:

from subprocess import Popen, PIPE, STDOUT

process = Popen("./a.out", stdin=PIPE, stdout=PIPE, stderr=STDOUT)

with process.stdin as pipe:
    pipe.write(b"3")
    pipe.flush()

# reading the echo of the number
print(process.stdout.readline())

# sending another num:
pipe.write(b"4")
pipe.flush()

process.stdin.close()

I got this output:

» python script.py
b'3\n'
Traceback (most recent call last):
  File "/tmp/script.py", line 13, in <module>
    pipe.write(b"4")
ValueError: write to closed file

That means that the first input is sent correctly, and also the read was done.

I really can't find something that explain this behaviour; can someone help me understanding? Thanks in advance

[EDIT]: since there are many points to clearify, I added this edit. I'm training on exploitation of buffer overflow vuln using the rop technique and I'm writing a python script to achieve that. To exploit this vuln, because of ASLR, I need to discover the libc address and make the program restart without terminating. Since the script will be executed on a target machine, I dont know which libraries will be avaiable, then I'm going to use subprocess because it's built-in in python. Without going into details, the attack send a sequence of bytes on the first scanf aim to leak libc base address and restart the program; then a second payload is sent to obtain a shell with which I will communicate in interactive mode.

That's why:

I can only use built-in libraries
I have to send bytes and cannot append ending \n: my payload would not be aligned or may leeds to fails
I need to keep open the stdin open
I cannot change the C-code

Solution

Change these:

Send a separator between the numbers read by the C program. scanf(3) accepts any non-digit byte as separator. For easiest buffering, send a newline (e.g. .write(b'42\n')) from Python. Without a separator, scanf(3) will wait for more digits indefinitely.
After each write (both in C and Python), flush the output.

This works for me:

#include <stdio.h>

int main(int argc, char *argv[])
{
    int n;
    while (1){
        scanf("%d", &n);
        printf("%d\n", n);
        fflush(stdout);  /* I've added this line only. */
    }
    return 0;
}

import subprocess

p = subprocess.Popen(
    ('./a.out',), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
try:
  print('A'); p.stdin.write(b'42 '); p.stdin.flush()
  print('B'); print(repr(p.stdout.readline()));
  print('C'); p.stdin.write(b'43\n'); p.stdin.flush()
  print('D'); print(repr(p.stdout.readline()));
finally:
  print('E'); print(p.kill())

The reason why your original C program works when run interactively within the terminal window is that in C the output is automatically flushed when a newline (\n) is written to the terminal. Thus printf("%d\n", n); does an implicit fflush(stdout); in the end.

The reason why your original C program doesn't work when run from Python with subprocess is that it writes its output to a pipe (rather than to a terminal), and there is no autoflush to a pipe. What happens is that the Python program is waiting for bytes, and the C program doesn't write those bytes to the pipe, but it is waiting for more bytes (in the next scanf), so both programs are waiting for the other indefinitely. (However, there would be a partial autoflush after a few KiB (typically 8192 bytes) of output. But a single decimal number is too short to trigger that.)

If it's not possible to change the C program, then you should use a terminal device instead of a pipe for communication between the C and the Python program. The pty Python module can create the terminal device, this works for me with your original C program:

import os, pty, subprocess

master_fd, slave_fd = pty.openpty()
p = subprocess.Popen(
    ('./a.out',), stdin=slave_fd, stdout=slave_fd,
    preexec_fn=lambda: os.close(master_fd))
try:
  os.close(slave_fd)
  master = os.fdopen(master_fd, 'rb+', buffering=0)
  print('A'); master.write(b'42\n'); master.flush()
  print('B'); print(repr(master.readline()));
  print('C'); master.write(b'43\n'); master.flush()
  print('D'); print(repr(master.readline()));
finally:
  print('E'); print(p.kill())

If you don't want to send newlines from Python, here is a solution without them, it works for me:

import os, pty, subprocess, termios

master_fd, slave_fd = pty.openpty()
ts = termios.tcgetattr(master_fd)
ts[3] &= ~(termios.ICANON | termios.ECHO)
termios.tcsetattr(master_fd, termios.TCSANOW, ts)
p = subprocess.Popen(
    ('./a.out',), stdin=slave_fd, stdout=slave_fd,
    preexec_fn=lambda: os.close(master_fd))
try:
  os.close(slave_fd)
  master = os.fdopen(master_fd, 'rb+', buffering=0)
  print('A'); master.write(b'42 '); master.flush()
  print('B'); print(repr(master.readline()));
  print('C'); master.write(b'43\t'); master.flush()
  print('D'); print(repr(master.readline()));
finally:
  print('E'); print(p.kill())