Search code examples
gostring-literals

How to use Golang custom scanner string literals and expand memory to load entire file into memory?


I have been trying to figure out how to implement what I originally thought would be a simple program. I have a text file of quotations that are all separated by ‘$$’

I want the program to parse the quotation file and randomly select 3 quotes to display and standard output.

There are 1022 quotes in the file.

When I attempt to split the file I get this error: missing '

I can’t seem to figure out how to assign $$ with a string literal, I keep getting:
missing '

This is the custom scanner:

onDollarSign := func(data []byte, atEOF bool) (advance int, token []byte, err error) {  
    for i := 0; i < len(data); i++ { 
        //if data[i] == "$$" {              # this is what I did originally
        //if data[i:i+2] == "$$" {    # (mismatched types []byte and string)
        //if data[i:i+2] == `$$` {    # throws (mismatched types []byte and string)
        // below throws syntax error: unexpected $ AND missing '
        if data[1:i+2] == '$$' {   
            return i + 1, data[:i], nil  
        }  
    }  

The string literal works fine if I only use one $.

For some reason only 71 quotations are loaded into the quotes slice. I'm not sure how to expand. To allow all 1022 quotes to be stored in memory.

I've been having a really difficult time trying to figure out how to do this. this is what I have right now:

package main
import (  
    "bufio"  
    "fmt"  
    "log"  
    "math/rand"  
    "os"  
    "time"  
)  

func main() {  
    rand.Seed(time.Now().UnixNano()) // Try changing this number!  
    quote_file, err := os.Open("/Users/bryan/Dropbox/quotes_file.txt")  
    if err != nil {  
        log.Fatal(err)  
    }  
    scanner := bufio.NewScanner(quote_file)  
    // define split function  
    onDollarSign := func(data []byte, atEOF bool) (advance int, token []byte, err error) {  
        for i := 0; i < len(data); i++ {  
            if data[i] == '$$' {  
                return i + 1, data[:i], nil  
            }  
        }  
        fmt.Print(data)  
        return 0, data, bufio.ErrFinalToken  
    }  
    scanner.Split(onDollarSign)  
    var quotes []string  

    // I think this will scan the file and append all the parsed quotes into quotes  
    for scanner.Scan() {  
        quotes = append(quotes, scanner.Text())  

    }  
    if err := scanner.Err(); err != nil {  
        fmt.Fprintln(os.Stderr, "reading input:", err)  
    }  
    fmt.Print(len(quotes))  
    fmt.Println("quote 1:", quotes[rand.Intn(len(quotes))])  
    fmt.Println("quote 2:", quotes[rand.Intn(len(quotes))])  
    fmt.Println("quote 3:", quotes[rand.Intn(len(quotes))])  
}  

Solution

  • Using a scanner if you end up reading the whole file anyway is kind of convoluted. I'd read the whole file and then simply split it into the list of quotes:

    package main
    
    import (
        "bytes"
        "io/ioutil"
        "log"
        "math/rand"
        "os"
    )
    
    func main() {
        // Slurp file.
        contents, err := ioutil.ReadFile("/Users/bryan/Dropbox/quotes_file.txt")
        if err != nil {
                log.Fatal(err)
        }
    
        // Split the quotes
        separator := []byte("$$") // Convert string to []byte
        quotes := bytes.Split(contents, separator)
    
        // Select three random quotes and write them to stdout
        for i := 0; i < 3; i++ {
                n := rand.Intn(len(quotes))
                quote := quotes[n]
    
                os.Stdout.Write(quote)
                os.Stdout.Write([]byte{'\n'}) // new line, if necessary
        }
    }
    

    Using a scanner would make sense if you selected three quotes before reading the file; then you can stop reading after you have reached the last quote.