I am considering embedding Lua in C++ app (running under FreeBSD 8.2). But benchmarking revealed poor performance in some cases. Specifically when Lua tries to convert strings to numbers and compare strings, it becomes slower, and worse, ruins scalability (8 cores perform worse than one!). I now think it is locale, because when I avoid auto-conversion everything works fine. But for real life I will need string comparisons and number conversions. How can I:
isolate Lua from locale, i.e. ensure that none of Lua's functions use locale indirectly. For instance can I provide my own conversion and comparison functions?
or disable locale altogether. I tried setlocale (LC_ALL, "C"), it works ok (locale changes), but bottleneck remains
Update:
following suggestion by lhf I jumped right into Lua library code. What I found is dozens of places where (officially) locale-dependent functions are used. To remove all of them would cost too much effort, there must be a better way. I tried to measure which of them do not scale. I also added some other commonly used functions, as well as some of my own interest (Lua interpreter creation and destruction, setting global variable, etc). Results follow. The correct percentage must be 700%, i.e. 7 threads must perform 7 times better than 1 thread:
nop: 824% (1:106867300/7:881101495)
sprintf %f: 57% (1:2093975/7:1203949)
sprintf %.14g: 51% (1:2503818/7:1278312)
sprintf %.14lf: 73% (1:2134432/7:1576657)
sprintf %lf: 64% (1:2083480/7:1340885)
sprintf %d: 601% (1:6388005/7:38426161)
sscanf %s: 181% (1:8484822/7:15439285)
sscanf %f: 712% (1:3722659/7:26511335)
lua_cycle: 677% (1:113483/7:768936)
set_global: 715% (1:1506045/7:10780282)
set_get_global: 605% (1:2814992/7:17044081)
strcoll: 670% (1:38361144/7:257300597)
getenv: 681% (1:8526168/7:58131030)
isdigit: 695% (1:106894420/7:743529202)
isalpha: 662% (1:80771002/7:535055196)
isalpha(r): 638% (1:78232353/7:499207555)
strtol: 694% (1:16865106/7:117208528)
strtod: 749% (1:16727244/7:125323881)
time: 168% (1:727666/7:1225499)
gettimeofday: 162% (1:727549/7:1183433)
figures change from run to run, but big picture remains consistent: sprintf double conversions perform worse than on single thread. time and gettimeofday scale badly. sscanf with %s also scales poorly which is quite surprising, but not an issue in my case.
At last it probably was not locale at all. I changed Lua conversion from sprintf to some simplified hand-made code and everything works fine so far..
BTW, first benchmark was run on linux desktop and showed nothing so strange. I was surprised by its FreeBSD behaviour.
To avoid locales in string comparison, change strcoll to strcmp in lvm.c. To avoid locales in string-to-number conversions, change the definition of lua_str2number in luaconf.h to avoid strtod. (Note however that supplying your own strtod is not an easy task.) You can also remove trydecpoint in llex.c.