how to read multiple serial ports in realtime in C or Python

I need to read multiple (at least 2) serial ports (currently two ports on a FT2232H module connected through USB).

I am using it to monitor a serial connections, so the two ports have their RX connected in parallel to RX and TX of the serial I need to monitor.

Setup is very similar to this.

I am setting up ports like this:

#define waitTime   0

int start_dev(const int speed, const char *dev) {
    int fd = open(dev, O_RDWR | O_NOCTTY |O_NONBLOCK| O_NDELAY);
    int isBlockingMode, parity = 0;
    struct termios tty;

    isBlockingMode = 0;
    if (waitTime < 0 || waitTime > 255)
        isBlockingMode = 1;

    memset (&tty, 0, sizeof tty);
    if (tcgetattr (fd, &tty) != 0) {
        /* save current serial port settings */
        printf("__LINE__ = %d, error %s\n", __LINE__, strerror(errno));
        exit(1);
    }

    cfsetospeed (&tty, speed);
    cfsetispeed (&tty, speed);

    tty.c_cflag = (tty.c_cflag & ~CSIZE) | CS8;     // 8-bit chars
    // disable IGNBRK for mismatched speed tests; otherwise receive break
    // as \000 chars
    tty.c_iflag &= ~IGNBRK;         // disable break processing
    tty.c_lflag = 0;                // no signaling chars, no echo,
                                    // no canonical processing
    tty.c_oflag = 0;                // no remapping, no delays
    tty.c_cc[VMIN]  = (1 == isBlockingMode) ? 1 : 0;            // read doesn't block
    tty.c_cc[VTIME] = (1 == isBlockingMode) ? 0 : waitTime;     // in unit of 100 milli-sec for set timeout value

    tty.c_iflag &= ~(IXON | IXOFF | IXANY); // shut off xon/xoff ctrl

    tty.c_cflag |= (CLOCAL | CREAD);        // ignore modem controls,
                                            // enable reading
    tty.c_cflag &= ~(PARENB | PARODD);      // shut off parity
    tty.c_cflag |= parity;
    tty.c_cflag &= ~CSTOPB;
    tty.c_cflag &= ~CRTSCTS;

    if (tcsetattr (fd, TCSANOW, &tty) != 0) {
        printf("__LINE__ = %d, error %s\n", __LINE__, strerror(errno));
        exit(1);
    }
    return fd;
}

... and currently I have this code for reading (I also tried with select()):

...
    for (running=1; running;) {
        for (int*p=devs; p<end; p++) {
            char b[256];
            int n = read(*p, b, sizeof(b));
            if (n > 0) {
                for (int i=0; i<n; i++) {
                    ...
                }
            }
        }
    }
...

This is obviously highly suboptimal because it doesn't suspend waiting for chars.

Problem is I experience some kind of buffering because when two processes exchange data on a tight loop I often see a few requests together and then the corresponding answers (1b6f is request and 19 is the empty answer):

1b6f
19
1b6f
19
1b6f
19
1b6f
191919
1b6f1b6f1b6f
19191919
1b6f1b6f1b6f1b6f
1b6f1b6f1b6f
191919

I also tried using python (pyserial), but I get similar results.

How should I proceed to ensure correct timings are enforced?

Note: I am not very interested in precise timing, but sequence should be preserved (i.e.: I would like to avoid seeing an answer before the request).

Solution

In my opinion what you're trying to do, which is if I understood correctly a kind of port sniffer to identify the transactions exchanged on a serial link is not feasible with USB-to-serial converters and a conventional OS, unless you're running at slow baudrates.

The USB port will always introduce a certain latency (probably tens of milliseconds), and you'll have to put the unpredictability of the OS on top of that.

Since you have two ports you could try to run two separate threads and timestamp each chunk of data received. That might help improve things but I'm not sure it will allow you to clearly follow the sequence.

If you had real (legacy) serial ports, and a not very loaded OS maybe you could do it somehow.

But if what you want is a serial port sniffer on the cheap you can try something like this solution. If you do forwarding on your ports you'll know what is coming from where at all times. Of course, you need to have access to either side of the communication.

If you don't have that luxury, I guess it would be quite easy to get what you want with almost any kind of microcontroller.

EDIT: Another idea could be to use a dual serial port to USB converter. Since both ports are served by the same chip, somehow I think it's likely that you can follow the sequence with one of those. I have access to this one if you post a full working snippet of your code I can test it next week, if you're curious to know.