I want to write compiler with C# for the first time and I somehow lost what to do about its buffering! My reference is Compilers, Principles, Techniques and Tools, and it says:
Because of the amount of time taken to process characters and the large number of characters that must be processed during the compilation of a large source program, specialized buffering techniques have been developed to reduce the amount of overhead required to process a single input character.An important scheme involves two buffers that are alternately reloaded,Each buffer is of the same size N, and N is usually the size of a disk block,e.g., 4096 bytes. Using one system read command we can read N characters into a buffer, rather than using one system call per character. If fewer than N characters remain in the input file, then a special character, represented by eof,marks the end of the source file and is different from any possible character of the source program.
and it also said in this book that we put eof at the end of each buffer to realize that we reach the end of buffer.and it has two pointers forward and lexemBegine that points to the lexeme in buffer! my problem is that I don't know how to create this buffer ? should I make array or buffer with the size N in sourceBuffer class and then how can I read file from StreamReader and put N characters of source file into array ? what is the problem if I read characters from source file instead?
Seems you reference the last edition of "Compilers: Principles, Techniques, and Tools" from 1986. (But even at that time the quoted part was already outdated).
In modern programming languages like C# (or more precisely in its I/O library) this kind of buffering is already implemented (in a robust, tested, high performance way).
Just use StreamReader which does all this work for you. Then just read character after character until you found a complete token, then process your tokens as described in this excellent book.