🐛 Bug: --timeout 0 with unref'd timers causes process exit

[rolfl]

Background context - SSCCE to follow. Bear with me, this behaviour is somewhat bizarre, so explaining it is complicated....

When using WebStorm as an IDE, it can run the mocha tests in "run" mode or "debug" mode. If the tests are constructed in a particular way (specifically - using promises) In "run" mode, everything works fine. In debug mode, all the tests are ignored.... in WebStorm, it looks like:

It took me a lot of trial-and-error to identify the "root cause", but it boils down to the --timeout 0 that webstorm adds when doing the node --inspect-brk part of debugging. The timeout 0 is added to allow breakpoints to pause the code without causing the tests to time out.

The problem can be explained with the following test case:

'use strict';


function slowThing() {
  return new Promise(resolve => setTimeout(resolve, 10).unref());
}

describe('This has lots of 10ms tests', function() {
  it('10ms', () => slowThing());
});

Note, all the test does, is wait 10ms, and then succeed.

There are 2 key features in the above....

1. it returns a promise in the it(...) test.
2. it does an unref() on the setTimeout()

If you run the above normally, it succeeds node ./node_modules/mocha/bin/_mocha --ui bdd tests/sscce.spec.js:

  This has lots of 10ms tests
    ✓ 10ms

  1 passing (34ms)

If you add --timeout 0 it completes successfully, but does not actually run the test: node ./node_modules/mocha/bin/_mocha --timeout 0 --ui bdd tests/sscce.spec.js && echo pass gives:

  This has lots of 10ms tests
pass

The same is true (it also fails) if you run in debug mode (--inspect).

BUT, if you add a second --timeout 0 it will then work correctly: node ./node_modules/mocha/bin/_mocha --timeout 0 --timeout 0 --ui bdd tests/sscce.spec.js && echo pass

  This has lots of 10ms tests
    ✓ 10ms

  1 passing (18ms)
pass

My guess is that the code that waits for Promise-based tests to complete is not actively waiting on the promise, so the node system discovers there are no events to prevent system exit, so it exits...

But, for the life of me, I don't understand why adding multiple --timeout 0 flags makes it work.....

Oh, and, if you use WebStorm, you end up having to add 2 --timeout 0 flags in your configuration, so that when you run the tests it has 2 timeouts, and when you debug, it has 3.

Edited to add versions:

rolf@rolf-vb:~/bmix/debugsscce$ node --version
v8.15.1
rolf@rolf-vb:~/bmix/debugsscce$ npm list mocha
debugsscce@1.0.0 /home/rolf/bmix/debugsscce
└── mocha@6.0.2

[plroebuck]

@boneskull, look at this...

[juergba]

Try to remove all --timeout flags while debugging, incl. the one inserted by your IDE.

Update in v6.0.0: --no-timeout is implied when invoking Mocha using debug flags. It is equivalent to --timeout 0. --timeout 99999999 is no longer needed.

[rolfl]

@juergba - I should note that I can't remove the --timeout 0 that WebStorm injects.... I'll initiate communication with them about it. Having said that, that would still just be a workaround for the bug, not a fix. The bug is clearly in the --timeout 0 causing tests to be ignored.

Also, if I run with the flag --no-timeout , mocha skips the tests anyway.... so it's an issue with --no-timeout too.

rolf@rolf-vb:~/bmix/debugsscce$ node ./node_modules/.bin/mocha tests/sscce.spec.js


  This has lots of 10ms tests
    ✓ 10ms


  1 passing (35ms)

rolf@rolf-vb:~/bmix/debugsscce$ node ./node_modules/.bin/mocha --no-timeout tests/sscce.spec.js


  This has lots of 10ms tests
rolf@rolf-vb:~/bmix/debugsscce$

[boneskull]

--no-timeout and --timeout 0 are equivalent.

Do your tests get ignored outside of webstorm when using --inspect-brk?

cc @segrey

[boneskull]

Ah, looks like you answered that.

[boneskull]

The problem is with unref(). You said this doesn't happen in Mocha v5.x?

[boneskull]

Seems like it works in v5.2.0 when using --timeout 9999999, but not --timeout 0, so there's obviously some further problem in the timeout-handling code.

[boneskull]

(@segrey, nevermind, this doesn't have anything to do with webstorm)

[boneskull]

So it looks like because timeouts are disabled within Mocha, and unref() was used, at that point in the stack there's nothing left in the event loop to do, so it looks like node just wants to quit.

The Promise is a red herring; this can be reproduced trivially using --timeout 0 and the following test:

it('just kind of stops', function(done) {
  setTimeout(done, 10).unref();
});