Search code examples
stringsizelimittoit

Esp32 string length limit


I found that the length of the line in app that runs on esp32 is limited. Below is an application that generates a string:

main :
  loop ::= 2459
  text  := ""
  stamp := "1234567890"
  loop.repeat :
    text += stamp
  print ("$text.size")  

If loop ::= 2458 everything is fine:

micrcx@micrcx-desktop:~/toit_apps/Hsm3/tests$ toit run test_str.toit
2021-06-02T17:03:43.237309Z: <process initiated>
24580
2021-06-02T17:03:44.575860Z: <process terminated - exit code: 0>

If loop ::= 2459, a memory allocation error appears:

micrcx@micrcx-desktop:~/toit_apps/Hsm3/tests$ toit run test_str.toit
2021-06-02T17:04:03.507773Z: <process initiated>
2021-06-02T17:04:04.907154Z: <stack trace>
EXCEPTION error. 
ALLOCATION_FAILED
  0: string.+                  <sdk>/core/string.toit:274:5
  1: main.<block>              /home/micrcx/toit_apps/Hsm3/tests/test_str.toit:6:10
  2: SmallInteger_.repeat      <sdk>/core/numbers.toit:209:3
  3: main                      /home/micrcx/toit_apps/Hsm3/tests/test_str.toit:5:8
  4: __entry__.<lambda>        <sdk>/core/entry.toit:46:20
2021-06-02T17:04:04.954257Z: <process terminated - exit code: 1>

All these errors are of course not fatal, it's just nice to know the real numbers, for example, the maximum length of a string, list, hash maps & etc, in order to write an app that at one point will not fall because the size of the json file has exceeded the allowable number.


Solution

  • There aren't any real size limitations on strings, lists, maps, etc.

    The system just tries to allocate space for these objects, and reports an error when there isn't enough memory left. This, obviously, imposes fundamental limits on all objects: The ESP32 has at most 520K RAM, so no data structures can every exceed that size.

    However, there are many more restrictions that come into play. IRAM memory, for example, has different properties than DRAM, and isn't well suited for normal data. Similarly, there is some memory that is used by the system.

    For some data structures, like strings, it's also important that the memory is in a continuous region. Fragmentation can lead to cases where a bigger string just can't be allocated anymore because all free memory is in smaller holes where the string wouldn't fit.

    Note, that using + to build up a string also means that there is always a copy of the previous string around, which doubles the space needed for the program.

    Here is an example, to show that strings aren't fundamentally limited to ~25K:

    main :
      loop ::= 4500
      stamp := "1234567890"
      text := stamp * loop
      print "$text.size"
    

    The * operator on strings preallocates the required size, and thus doesn't need any other copy (as in the loop).