Just doing some simple benchmark on Bytestring and String. The code load a files of 10,000,000 lines, each an integer; and then convert each of the strings into the integer. Turns out the Prelude.read is much slower than ByteString.readInt.
I am wondering what is the reason for the inefficiency. Meanwhile, I am also not sure which part of the profiling report corresponds to the time cost of loading files (The data file is about 75 MB).
Here is the code for the test:
import System.Environment
import System.IO
import qualified Data.ByteString.Lazy.Char8 as LC
main :: IO ()
main = do
xs <- getArgs
let file = xs !! 0
inputIo <- readFile file
let iIo = map readInt . linesStr $ inputIo
let sIo = sum iIo
inputIoBs <- LC.readFile file
let iIoBs = map readIntBs . linesBs $ inputIoBs
let sIoBs = sum iIoBs
print [sIo, sIoBs]
linesStr = lines
linesBs = LC.lines
readInt :: String -> Int
readInt x = read x :: Int
readIntBs :: LC.ByteString -> Int
readIntBs bs = case LC.readInt bs of
Nothing -> error "Not an integer"
Just (x, _) -> x
The code is compiled and executed as:
> ghc -o strO2 -O2 --make Str.hs -prof -auto-all -caf-all -rtsopts
> ./strO2 a.dat +RTS -K500M -p
Note "a.dat" is at aforementioned format and about 75MB. The profiling result is:
strO2 +RTS -K500M -p -RTS a.dat
total time = 116.41 secs (116411 ticks @ 1000 us, 1 processor)
total alloc = 117,350,372,624 bytes (excludes profiling overheads)
COST CENTRE MODULE %time %alloc
readInt Main 86.9 74.6
main.iIo Main 8.7 9.5
main Main 2.9 13.5
main.iIoBs Main 0.6 1.9
individual inherited
COST CENTRE MODULE no. entries %time %alloc %time %alloc
MAIN MAIN 54 0 0.0 0.0 100.0 100.0
main Main 109 0 2.9 13.5 100.0 100.0
main.iIoBs Main 116 1 0.6 1.9 1.3 2.4
readIntBs Main 118 10000000 0.7 0.5 0.7 0.5
main.sIoBs Main 115 1 0.0 0.0 0.0 0.0
main.sIo Main 113 1 0.2 0.0 0.2 0.0
main.iIo Main 111 1 8.7 9.5 95.6 84.1
readInt Main 114 10000000 86.9 74.6 86.9 74.6
main.file Main 110 1 0.0 0.0 0.0 0.0
CAF:main1 Main 106 0 0.0 0.0 0.0 0.0
main Main 108 1 0.0 0.0 0.0 0.0
CAF:linesBs Main 105 0 0.0 0.0 0.0 0.0
linesBs Main 117 1 0.0 0.0 0.0 0.0
CAF:linesStr Main 104 0 0.0 0.0 0.0 0.0
linesStr Main 112 1 0.0 0.0 0.0 0.0
CAF GHC.Conc.Signal 100 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding 93 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding.Iconv 91 0 0.0 0.0 0.0 0.0
CAF GHC.IO.FD 86 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Handle.FD 84 0 0.0 0.0 0.0 0.0
CAF Text.Read.Lex 70 0 0.0 0.0 0.0 0.0
Edit:
The input file "a.dat" are 10,000,000 lines of numbers:
1
2
3
...
10000000
Following the discussion I replaced "a.dat" by 10,000,000 lines of 1s, which does not affect the above performance observation:
1
1
...
1
read is doing a much harder job than readInt. For example, compare:
> map read ["(100)", " 100", "- 100"] :: [Int]
[100,100,-100]
> map readInt ["(100)", " 100", "- 100"]
[Nothing,Nothing,Nothing]
read is essentially parsing Haskell. Combined with the fact that it's consuming linked lists, it's no surprise at all that it's really very slow indeed.