How I can read more than 16384 bytes using OpenSSL TLS?

Question

I'm trying to read a big chunk of data using OpenSSL TLS sockets, and I'm always stuck at 16384 being read. How I can read more?

SSL_CTX* ctx;
int server;
SSL* ssl;
int bytes;
std::string result;
std::vector<char> buffer(999999999);

ctx = InitCTX();
server = OpenConnection();
ssl = SSL_new(ctx);
SSL_set_fd(ssl, server);
if (SSL_connect(ssl) != -1)
{
    std::string msg = 0; //request here
    SSL_write(ssl, msg.c_str(), msg.size());
    bytes = SSL_read(ssl, &buffer[0], buffer.size());
}

result.append(buffer.cbegin(), buffer.cend());

Answer 1

The TLS protocol encapsulates data in records that are individually encrypted and authenticated. Records have a maximum payload of 16 kB (minus a few bytes), and SSL_read() will only process one record at a time.

I suggest you change the size of buffer to 16384 bytes to match. Note that allocating ~1 GB as you did is way too much anyway, as that amount of memory would then potentially not be available to other processes.

Then, as rustyx mentioned in the comments, just read more in a loop. If the other side can respond with multiple records, it would be good if it would somehow send the size of the response in the first record, so you would know how much to read.