[rfc] file system benchmark

[vincent-d]

Contribution description

This is based on #8315 and #8239.
This provide an app to benchmark the file systems using vfs.

Results

I've run the test on native with the emulated mtd device for spiffs and littlefs with the following results.

spiffs

Binary size:

   text	   data	    bss	    dec	    hex
  87874	    792	  49440	 138106	  21b7a

Timings:

benchmarking file system: spiffs
[BEGIN] Format test...
Test time: 69146us
[END] Format test
[BEGIN] Mount test...
Test time: 16786us
[END] Mount test
[BEGIN] Write test...
File #0, 400 bytes written in: 60569us
File #1, 400 bytes written in: 54963us
File #2, 400 bytes written in: 59451us
File #3, 400 bytes written in: 53766us
File #4, 400 bytes written in: 58093us
File #5, 400 bytes written in: 56973us
File #6, 400 bytes written in: 55487us
File #7, 400 bytes written in: 53980us
File #8, 400 bytes written in: 56722us
File #9, 400 bytes written in: 55598us
File #10, 400 bytes written in: 53592us
File #11, 400 bytes written in: 54107us
File #12, 400 bytes written in: 55462us
File #13, 400 bytes written in: 56342us
File #14, 400 bytes written in: 54442us
File #15, 400 bytes written in: 55705us
File #16, 400 bytes written in: 55280us
File #17, 400 bytes written in: 55397us
File #18, 400 bytes written in: 54621us
File #19, 400 bytes written in: 55357us
Mean time: 55795us
[END] Write test
[BEGIN] Read test...
File #0, 4000 bytes read in: 661us
File #1, 4000 bytes read in: 659us
File #2, 4000 bytes read in: 722us
File #3, 4000 bytes read in: 663us
File #4, 4000 bytes read in: 662us
File #5, 4000 bytes read in: 658us
File #6, 4000 bytes read in: 660us
File #7, 4000 bytes read in: 934us
File #8, 4000 bytes read in: 1039us
File #9, 4000 bytes read in: 1063us
File #10, 4000 bytes read in: 934us
File #11, 4000 bytes read in: 949us
File #12, 4000 bytes read in: 943us
File #13, 4000 bytes read in: 687us
File #14, 4000 bytes read in: 666us
File #15, 4000 bytes read in: 664us
File #16, 4000 bytes read in: 647us
File #17, 4000 bytes read in: 678us
File #18, 4000 bytes read in: 651us
File #19, 4000 bytes read in: 836us
Mean time: 768us
[END] Read test
[BEGIN] Remove test...
File #0, test time: 6857us
File #1, test time: 222us
File #2, test time: 220us
File #3, test time: 217us
File #4, test time: 219us
File #5, test time: 219us
File #6, test time: 219us
File #7, test time: 216us
File #8, test time: 221us
File #9, test time: 219us
File #10, test time: 225us
File #11, test time: 217us
File #12, test time: 220us
File #13, test time: 219us
File #14, test time: 217us
File #15, test time: 220us
File #16, test time: 231us
File #17, test time: 219us
File #18, test time: 216us
File #19, test time: 217us
Mean time: 551us
[END] Remove test
[BEGIN] Unmount test...
Test time: 3us
[END] Unmount test

littlefs

Binary size:

   text	   data	    bss	    dec	    hex	
  69561	    796	  49024	 119381	  1d255

Timings:

benchmarking file system: littlefs
[BEGIN] Format test...
Test time: 1178us
[END] Format test
[BEGIN] Mount test...
Test time: 35us
[END] Mount test
[BEGIN] Write test...
File #0, 400 bytes written in: 6103us
File #1, 400 bytes written in: 6440us
File #2, 400 bytes written in: 7328us
File #3, 400 bytes written in: 6891us
File #4, 400 bytes written in: 6089us
File #5, 400 bytes written in: 6173us
File #6, 400 bytes written in: 6036us
File #7, 400 bytes written in: 6308us
File #8, 400 bytes written in: 6899us
File #9, 400 bytes written in: 6145us
File #10, 400 bytes written in: 6027us
File #11, 400 bytes written in: 6046us
File #12, 400 bytes written in: 6128us
File #13, 400 bytes written in: 7743us
File #14, 400 bytes written in: 7992us
File #15, 400 bytes written in: 7394us
File #16, 400 bytes written in: 6756us
File #17, 400 bytes written in: 7549us
File #18, 400 bytes written in: 8485us
File #19, 400 bytes written in: 6799us
Mean time: 6766us
[END] Write test
[BEGIN] Read test...
File #0, 4000 bytes read in: 160us
File #1, 4000 bytes read in: 157us
File #2, 4000 bytes read in: 170us
File #3, 4000 bytes read in: 158us
File #4, 4000 bytes read in: 159us
File #5, 4000 bytes read in: 157us
File #6, 4000 bytes read in: 145us
File #7, 4000 bytes read in: 127us
File #8, 4000 bytes read in: 134us
File #9, 4000 bytes read in: 128us
File #10, 4000 bytes read in: 127us
File #11, 4000 bytes read in: 129us
File #12, 4000 bytes read in: 137us
File #13, 4000 bytes read in: 142us
File #14, 4000 bytes read in: 150us
File #15, 4000 bytes read in: 143us
File #16, 4000 bytes read in: 156us
File #17, 4000 bytes read in: 134us
File #18, 4000 bytes read in: 134us
File #19, 4000 bytes read in: 134us
Mean time: 144us
[END] Read test
[BEGIN] Remove test...
File #0, test time: 734us
File #1, test time: 731us
File #2, test time: 735us
File #3, test time: 744us
File #4, test time: 728us
File #5, test time: 746us
File #6, test time: 414us
File #7, test time: 426us
File #8, test time: 419us
File #9, test time: 407us
File #10, test time: 399us
File #11, test time: 403us
File #12, test time: 394us
File #13, test time: 394us
File #14, test time: 398us
File #15, test time: 423us
File #16, test time: 399us
File #17, test time: 386us
File #18, test time: 381us
File #19, test time: 386us
Mean time: 502us
[END] Remove test
[BEGIN] Unmount test...
Test time: 2us
[END] Unmount test

Differences

• Opening/writing 4000B/closing a file is about 8 times faster with littlefs than with spiffs
• Opening/reading 4000B/closing a file is about 5 times faster with littlefs than with spiffs
• Formatting is about 500 times faster with littlefs than with spiffs
• Mounting is about 60 times faster with littlefs than with spiffs

[vincent-d]

I ran the test on real hardware and the differences are even bigger, the performance test goes in favor of littlefs.

[miri64]

#8315 and #8239 where merged. Needs rebase.

[vincent-d]

Removed WIP, I think this is usable enough as-is.

I used it to test #8715 too.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.