Python Shell: writing one byte and reading decimal data from a serial port

Question

Problem Description: I am trying to retrieve all data stored on a device using rs232.

How?: For this specific device, I need to:

• Step 1: send one byte (0x80) - uInt8 (8-bit unsigned integer) through a serial port (COM5) and expect to receive 0x81 in response.
• Step 2: send a second byte (0x81) and expect to receive all data stored, line by line.

Note: The device is automatically transmitting the data it has. It only accepts one byte at a time (in my case, sending two bytes at a time is enough to retrieve the data --- Step 1 and Step 2).

Device Testing: Before running my experiment, I first tested whether my serial connection is working. First, I tested the serial connection with a terminal emulation program called RealTerm: Serial/TCP Terminal. Second, I run a Matlab test using shell commands. I could retrieve all data stored on the device for both tests.

---

What have I tried?: I have tried to write a Python Script and a Node.js Script. Unfortunately, both scripts did not work, both Scripts were giving 0x00 whenever I send 0x80 (I failed to pass Step 1). I am not sure where is the issue though! (I have been trying for 7 days)

Today, I thought about running an experiment using Python Shell instead of a Script.

>>> import serial
>>> rs232 = serial.Serial(
...     port = 'COM5',
...     baudrate = 115200,
...     bytesize = serial.EIGHTBITS,
...     parity = serial.PARITY_NONE,
...     stopbits = serial.STOPBITS_ONE,
...     timeout=1
...     )
>>> rs232.write(0x80)
128
>>> rs232.read(size=1)
b'\x87'
>>> rs232.read(size=2) 
b'\x87\x87'
>>> rs232.read(size=5) 
b'\x87\x87\x87\x87\x87'

This gave me some hope because I could receive something back from the device in Step 1 experiment. Though, I am not sure why am I receiving b'\x87' (ord(rs232.read(size=1)) -> 135) instead of 0x80. Also, rs232.read(size=5) gives same values!! Well, I am new to embedded programming. I am sorry about any confusion here.

I also read something about using a buffer but didn't get the idea of how can I use it for my experiment (Step 1 and Step 2).

---

What am I expecting? I would like to be able to write a Python Script instead of using Python Shell to retrieve all data stored in the device and save it into a CSV file, line by line.

workable solution based on all feedbacks: After taking all advices in the answers of this question, I ended up having the following working with some an issue I would appreciate to someone help me to fix it.

import serial, time

rs232 = serial.Serial(
    port = 'COM6', 
    baudrate = 115200, 
    bytesize = serial.EIGHTBITS,
    parity = serial.PARITY_NONE,
    stopbits = serial.STOPBITS_ONE,
    timeout=5,
    write_timeout=5
    )

rs232.is_open
rs232.set_buffer_size(rx_size = 2000000, tx_size = 2000000)

# cleanup RX/TX buffer
rs232.rts=True
rs232.reset_input_buffer()
rs232.reset_output_buffer()

time.sleep(2)
rs232.write(bytes([0x80, 0x82, 0x83]))

time.sleep(5)
while True:
    myBytes = rs232.readline()

    data.append(myBytes)
    print(myBytes)
    if myBytes == b'\r\x83\x87':
        break

Answer 1

My suggestion:

1. make sure that connection settings are really the same (I would compare them to Matlab)
2. send correct commands: the first is b'\x80' and the second b'\x82' from your screenshots, although here you are writing b'\x81'
3. add 5s timeout on both write and read, so you will definitely know if something is waiting
4. reset in and out buffers as was suggested in the comments
5. use logging to see the timings of each command
6. add additional sleep to see if anything changes
7. as a last resort you could try with a different library https://pyvisa.readthedocs.io/en/latest/index.html

My test script would be something like this (run with python -u to avoid buffering of log output):

import logging
import serial
import time
from pathlib import Path

logging.basicConfig(level='INFO', format='%(asctime)s - %(message)s')

logging.info('open')
rs232 = serial.Serial(
    port='COM5',
    baudrate=115200,
    bytesize=serial.EIGHTBITS,
    parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,
    timeout=5,
    write_timeout=5,
)

logging.info('reset buffers')
rs232.reset_input_buffer()
rs232.reset_output_buffer()

logging.info('write first')
rs232.write(b'\x80')

# time.sleep(1)  # see if uncommenting changes anything

logging.info('read first')
reply = rs232.read(size=1)
logging.info(reply)

logging.info('write second')
rs232.write(b'\x82')

# time.sleep(1)  # see if uncommenting changes anything

logging.info('read second')
# reply = rs232.read_until(expected=b'\x0a\x0d')  # read single line
reply = rs232.read_until(expected=b'\x80')
Path('result.csv').write_text(reply.decode().replace('\n\r', '\r\n'))
logging.info(reply)

logging.info('close')
rs232.close()