Reading and Writing more than 4096 bytes to and from STDIN

Question

I'm trying to implement a here document like the one used in bash (in C), but I noticed that I can't give it a line that's longer than 4096 bytes, I looked it up and It was because of the max size of the Kernel I/O queues so my process never receives the full input that I type in the shell. Meanwhile the bash's heredoc can handle that. is there a way to get around this using C.

with any word that exceeds 4096 characters only the 4096 characters gets printed or the input gets corrupted

#include <unistd.h>

int main()
{
    ssize_t r_bytes;
    char buf[42];
    while (r_bytes = read(0, buf, 42))
    {
        write(1, buf, r_bytes);
    }
    return (0);
}

1-Compile and execute this code. 2-Give it this input:

<start>herrslcrwhjsyxlnuvxoofkmnnqnqplyyomfekjdvifzmzqsjjffjlqgvxrmhplzfaywnsptodrwygvhtplduhivfefqefkwcrficiatatbvlpzikietxdfekfnqvgazseylfagktlaseeedbbeajmnvshwkzvlfyhdoswwzzdesnaphprclhihfdckjcpepytllszcucuyvgqlfreyknejkjbthgociqxzellhzgdqkpnglnaocygxqifkjikcrepkcosxclwqlhhgzndoqroocniwrjaoumuvlvtvdrsyaupmkldzhvurnzgwhpvsyzrqzxgtlythlqflciocyepzbxyepegfxifkuopqrivsnhzqhmjmbrdkoxwrgpmdntoqlksjxwrfdadkjzjvyknulhlhmseykfpqnpvfpfbxncefcbfzdfndkwskdusuhylklqcxnmbibgmerllmzgmorqpqawbiexnbgkuaxbiucrkfsxndhdifgrfiuyanximidvvfaqnurmsnvfnwahuvpyjjvpukbrkemqotmvnvqcwivueqisxqvrfunmaaeuguqpullxgyuypepxgkrmgpfxlihmazubjmnlifqoyfidhpvvpyraddjlxoalrhsrpviqkhmcszkluelumfpukwynrmlhdvnwhhukorlpqzouzakphkgorumfqabjxrtghrvynqazsoahewbhpujofcokzeapdlpkgzrrdnmmwintaoowbzctqnxatvfecwnirxaacqmsrbecdgikauqosygzyxkyfikryohidyslrizbwnnhbiwnqbbgqylhvkmtnazfdzvweucoqdovuzfuqlywxdfkmzgsssnprekjfnutlmhbdvtcngeitphqywvtmppptaxyielehppdxjgzuvwybjealhkjjlgibgjcysgvmgfvgqpnpmadtjlngmsxgvolvdlzkfyfqoujjiegunytjkxornyruaxgpqqjhqyfwymvyglnccfbdslqdbrqpmwtonklsajnuaemvkqwjuscectjwbvohfvuecltlkhszplqxipaymbtxjztjofubjblqrrantnfcsoirrvkwnvmwnwqpdsysqsokmwylnvvryhnaatnlbmvtfhbquzmnrukbvqbvwjdqrdgthrqpnjebcokbroruogaswulugwxkrrmbrkskqhlgbufanzafdsbgcqcjxntmczmzrsjqjfcpmetvjirmcpmfmobfslagzlvoppuachoyxbrgqejrdbykeczesigwwcmjdmvvzudouderjbhpihwaxhizaxejmckwnwgmhcgylephrukykldufcobjqrmiiqnxznmlnzpdbxgyxfnwdtgjeupvxdnzyvivpuvrfulypwzfgxmnomohfcvcyuhimpeelkwaurwcgcthvsfpppurrercewdfhryyyytzowizhtlzohewpqxjmacpgkhvjzopuoggswmowcbfhkdpkvbjajqjmqfqbovgiaojsfyaqzqmngemgytcsvsqfdpbcehxrcleqmguhnuipoqecwralfehnxrevtmgqbiqpfighujwpgqqfcumppeamwasfqkeochbptmezrmttngijtwawvojpmpkzozkkxzuxzggvcswowkjmtvtmzwptwolaflkddwbkmzxvnclqzsikajmyihcatjrnvqixbfnmpvigmmwmepppebohsidrquffvkmjxtuchozqbdcowfqpnrhxzrkxozpqfojuybxwcerlfmaveslyfnkvkphotyrdukcelzszlqaqyayvuzljfsgrltqherbwabxslbwxlrfnundvzvchgpllcenakkwaejdejpfqiarsomxgcoplpdrqdudcaopdxjziyvuqeolkqqopwzixmhpyaabjuoplqourpuzphjfyvvwoxlssgkkqrwewzikvkdgfwgtzkennzhlnuakywfiiwqvpsfqgkowunzfblrxiwqibulojvrjnbhgjookrazjzrhmhqxnxjmaupdczcwlhmhvaepgvdksckettpzqhkfbmszbqbqidnjnwcrloqduucjdgvisrlixrwsqtdrackbbwbfslpgrkycycqnbccebvcuivfkyejwwcfpuaqmuacfrcbjtmvwvcryuzlbvuwagcggiwwzxxbicvnwbafuubeqjuvrdcizwbnagdntjypfolexpesovyrupoxlnhtdcnhyppxoniqdpsfyhlarshtdgwbtbkjtklvdpztdumykphalytjaefwfhwhhnpqcwfcruhmmxoqngnvtfbxwfmfdkhivhitsdecqehowjmgxmdipdtsiwuxivdjmmwrkotqrdhwtnzfbhemboqkqkodaabxkedzkpxriuurifgfjhrtxpqlaoolqgbkdqyaatgnsdzhawpfhvwgtqjsdeopuwvgolouofwwzbhpeybvpglwwqjcjlqpsobgojoijblingffcrsydqcjjzlwoeklhxzecitrtihakcktvfnfblmhejybnsihzhmcgxayxoruyvnecnbbfpwieoejtfzjhjczduuldcflvwzjzruxlckvpyivtnyzfzdhfzptwjjggaeqsbgkkynqjpsuobflnaqbwayslrsvrgrbtjzhkbtmvicoxbdfexnuwzbvamumswnloymrjojujwtmcudbhcxvbxbcwzrzhjlrgoerptgczhldyskvqpqxxpuywrtltwyzkbpdmzgcaulbdygqfeoflgwnhtuaelesgjaawskmsccgyoqraabxppuueyfbdvuhvotgbswrtyfxliwkewxzpoetubihymgbmqnmaseemmonsgkbfsomawfjgxqqrjbfxpyissubkeafseucdpwasafhkkhsdygbgeypjesxeyafbdctkgorlcknmjtnqpjbkwwyztqailtefgjmkgujpynkfmtqnjvdqjgttnvejsvzahddzedysitrwumidnufktcozdfgshwzicuykokdjxszhcsimalzijyvylkaxdweafmtfgtkaepteyafqijbonhgxrqnkxwtpceajseopimpkvouzajzrrtbpiymsqxnrkmpejokpancnmmcincaddywnozsknikimutldarwabrzgvymatbamsbwaikbwatvqbpvfsdeafycximswbehmyzwzarqvcjtzyofhfjxagtrehvssqsbtowueuizqjuyjrulldgxtjtkirekhwaxhdfcnbwhyqtuinxpxddkvoiveoenwsgixpqqxupuunrgyddowemsegsovkigdwxgefzvmiuuafynnxukixpsnmiwszojlmcxvopwdazpcuxetclgatgzmvmeecppggaqhobatlyynzhdygxiatrktpgntmkqcjkuxmdlroounnxcebepchuokotvmqfctmtjsnplgjmeobzbwhxekvlbknvwughuppzuecumvqjylqkhlmpbbhmgrenotwyyquqhgrrboqvjskaczyhnukqogmqbngkrvsakcppveykzuqmsbxdfxjulofmpnilynqzzgdxbdjjzhskrthbwzjzugjsutgpgkromaccbjqitrhxlahzaygcvztjycifokrbxugvdmwxqqxibhgckxwhabbjsmnpencsplrxjnfanqtdgrruiomesapwfdvsotvkgqgfxeelavujnigyedrhwksraopenosfgbrdvogtudmxecvrvxvjjuaodjvajlgazeqsyemyysxpnilklpvzciwidtjebaczuhezsresvglzivbbodxakeighfawfanvdzzpadrcisfsqyhfbuyurppqdfekrolbzxvquulxhnwwndndirrcysbywooygizgbuiezholvuoqkyiorttcmzpseippnxkuilfsrxfknzkvzlqmdlupqkxdjxtiuxjghikocnpkocoauecaofnsyouizpbgvytkpxrfalhvpqymbxqetmozufwmcbycsalporhqzlfpbeqgffteuqeatujlrjyjhjjjhj<end>helloworld

and the output is everything until the <end> or a 4096 charater string that is corrupted.

Answer 1

To handle input lines longer than 4096 bytes in C, you can switch the terminal to Non-Canonical mode (also known as Raw mode).

In Non-Canonical mode, input is available immediately without waiting for a newline, allowing you to process input that exceeds the typical buffer size.

Look at your code modified as explained above:

#include <unistd.h>
#include <termios.h>
#include <stdlib.h>
#include <stdio.h>

void setRawMode(int fd) {
    struct termios term;

    tcgetattr(fd, &term);
    term.c_lflag &= ~(ICANON | ECHO);
    tcsetattr(fd, TCSANOW, &term);
}

void setCanonicalMode(int fd) {
    struct termios term;

    tcgetattr(fd, &term);
    term.c_lflag |= (ICANON | ECHO);
    tcsetattr(fd, TCSANOW, &term);
}

int main() {
    setRawMode(STDIN_FILENO);

    ssize_t r_bytes;
    char    buf[64];

    while ((r_bytes = read(STDIN_FILENO, buf, sizeof(buf) - 1)) > 0) {
        buf[r_bytes] = '\0'; 
        printf("%s", buf);
    }

    setCanonicalMode(STDIN_FILENO);

    return 0;
}