Sanitycheck shows "FAILED: failed" for successful test: tests/kernel/fifo/fifo_api/kernel.fifo

[mike-scott]

Describe the bug
Occasionally in a sanitycheck run on real HW, we see the tests/kernel/fifo/fifo_api/kernel.fifo shown as "failed", but looking at the actual test log shows "PROJECT EXECUTION SUCCESSFUL"

CI Log for sanitycheck run on nrf52_blenano2: https://ci.foundries.io/projects/zephyr/builds/5367/sanity-test-nrf52/artifacts/console.log

To Reproduce
Hard to reproduce, but if you rinse and repeat the following sanitycheck command long enough you should see the failure described above:

sanitycheck --platform nrf52_blenano2 --inline-logs --outdir /tmp/outdir --enable-slow --verbose --no-clean --disable-size-report --load-tests /tmp/test-list --test-only --device-testing --device-serial /dev/ttyACM0 --west-flash=--board-id=109300003131a6610c36303333363445a5a5a5a597969906 -e kernel

Expected behavior
Test showing "PROJECT EXECUTION SUCCESSFUL" should not be called "FAILED".

Impact
CI inconsistency

Screenshots or console output
https://ci.foundries.io/projects/zephyr/builds/5367/sanity-test-nrf52/artifacts/console.log

Environment (please complete the following information):

• OS: Linux
• Toolchain: Zephyr SDK
• Commit SHA or Version used: 6000a62

[nashif]

That log is not really useful, need the actual test log. Sometimes a test passes just fine but before it is terminated an exception happens which causes the failure..

[nashif]

ping?

[nashif]

@mike-scott ping?

[nashif]

can't reproduce and no additional details.