Search code examples
filegoiocharacterbyte

Processing a file byte by byte in go


Similar to this question, what is the idiomatic way to process a file one byte at a time in go?

Put another way, is there a better way to write the following?

file, err := ioutil.ReadFile(filename)
file_string = string(file)
for i, c := range file_string {
    // -snip-
}

Solution

  • You're reading the entire file as a string (not bytes) at once, then processing it rune-by-rune (not byte-by-byte). To literally read the file one byte at a time (which is very likely not what you want), you must do exactly that:

    f, err := os.Open("path")
    if err != nil {
        panic(err)
    }
    
    b := make([]byte, 1)
    
    for {
        err := f.Read(b)
        if err != nil && !errors.Is(err, io.EOF) {
            fmt.Println(err)
            break
        }
    
        // process the one byte b[0]
    
        if err != nil {
            // end of file
            break
        }
    }
    

    However, what you more likely want is to read the file efficiently, and process it one byte at a time. For this, you should use a buffered reader:

    f, err := os.Open("path")
    if err != nil {
        panic(err)
    }
    
    br := bufio.NewReader(f)
    
    // infinite loop
    for {
    
        b,err := br.ReadByte()
    
        if err != nil && !errors.Is(err, io.EOF) {
            fmt.Println(err)
            break
        }
    
        // process the one byte b
    
        if err != nil {
            // end of file
            break
        }
    }