Out-of-memory issues are undetected

[matthijskooijman]

When the Arduino runs low on memory, malloc starts failing and the String class starts silently dropping its contents. In practice, this means that the parser.ino example running on a Uno gives empty values for all String fields. I suspect that memory is just enough, since no other random behaviour happens (which is typical when memory is exhausted and the stack starts to overwrite the heap and global variables), but malloc starts to fail a bit earlier (to leave some space for the stack).

After compilation, there are still 800 bytes of free RAM. About 300 are taken up by the huge data struct, but the other 500 are possibly taken up by stack variables? This might warrant some investigation to try and reduce the memory usage.

Additionally, it would be good if memory errors could be handled more gracefully. I'm not sure if the String class properly returns this error condition (it might have some kind of "isvalid" method), but worst case we can just check the String length value and return an error if it does not match.

[HermenB]

Hi Matthijs,

Thanks for the explanation given here (the stack overwriting the heap)!
I am working with an Arduino Uno and ran into this issue myself. The memory restriction of an Uno is very tight (2 kB). In a previous program I used sscanfto parse the telegram, and sprintfto write to SD, and the design is prone to bugs, probably because of memory issues, too. Then I found your library and since it seemed to be written explicitly for Arduino, I decided to try and use it. But alas, for Arduino Uno the memory footprint is still too big:(

The problem might be solved by parsing the telegram line-by-line, since one whole telegram is about 650 bytes (or more if more fields are available). From experience, I know the Arduino compiler starts to complain if less than 500 bytes are left for local variables, so using a fixed char buffer[BUFSIZE] instead of String would probably bring the free memory above that threshold.

But I guess that might require a complete refactoring of your code, and who nowadays works with a bare Arduino Uno?! I'm sure it works fine on ESP and the like...

But if you feel challenged to get it working on an ancient UNO, I'd much appreciate it!
Regards,

Hermen