Add Jest to lookup

[targos]

Jest is a very popular testing framework (> 4.4M downloads). Node.js releases have broken it more than once, latest being 11.11.0: nodejs/node#25963 (comment).

Support for Yarn was a necessary step to add Jest but unfortunately it is not enough.
Here is the (truncated) output I get with the latest version of CITGM:

$ citgm jest --yarn
info:    starting            | jest                
info:    lookup              | jest                
info:    lookup-notfound     | jest                
info:    lookup-githead      | https://github.com/facebook/jest/archive/b16789230fd45056a7f2fa199bae06c7a1780deb.tar.gz
info:    jest npm:           | Downloading project: https://github.com/facebook/jest/archive/b16789230fd45056a7f2fa199bae06c7a1780deb.tar.gz
info:    jest npm:           | Project downloaded -.tgz
info:    jest yarn:          | yarn install started
info:    jest yarn:          | yarn install successfully completed
info:    jest yarn:          | test suite started  
error:   failure             | The canary is dead: 
error:   failing module(s)   |                     
error:   module name:        | jest                
error:   version:            | 24.1.0              
error:   error:              | The canary is dead: 
error:   error:              | undefinedyarn install v1.13.0                                                                                                                         
error:                       | [1/5] Validating package.json...                                                                                                                      
error:                       | [2/5] Resolving packages...                                                                                                                           
error:                       | [3/5] Fetching packages...                                                                                                                            
error:                       | info fsevents@1.2.7: The platform "linux" is incompatible with this module.                                                                           
error:                       | info "fsevents@1.2.7" is an optional dependency and failed compatibility check. Excluding it from installation.                                       
error:                       | [4/5] Linking dependencies...                                                                                                                         
error:                       | [5/5] Building fresh packages...                                                                                                                      
error:                       | $ opencollective postinstall && yarn build                                                                                                            
error:                       |                                                                                                                                                       
error:                       | *** Thank you for using jest! ***                                                                                                                     
error:                       |                                                                                                                                                       
error:                       | Please consider donating to our open collective                                                                                                       
error:                       | to help us maintain this package.                                                                                                                     
error:                       |                                                                                                                                                       
error:                       | https://opencollective.com/jest/donate                                                                                                                
error:                       |                                                                                                                                                       
error:                       | ***                                                                                                                                                   
error:                       |                                                                                                                                                       
error:                       | yarn run v1.13.0                                                                                                                                      
error:                       | $ node ./scripts/build.js                                                                                                                             
error:                       | Building packages                                                                                                                                     
error:                       | babel-jest................................................................ DONE                                                                       
error:                       | babel-plugin-jest-hoist................................................... DONE                                                                       
error:                       | babel-preset-jest......................................................... DONE                                                                       
error:                       | diff-sequences............................................................ DONE                                                                       
error:                       | eslint-config-fb-strict................................................... DONE                                                                       
error:                       | expect.................................................................... DONE                                                                       
error:                       | jest...................................................................... DONE                                                                       
error:                       | jest-changed-files........................................................ DONE                                                                       
error:                       | jest-circus............................................................... DONE                                                                       
error:                       | jest-cli.................................................................. DONE                                                                       
error:                       | jest-config............................................................... DONE                                                                       
error:                       | jest-diff................................................................. DONE                                                                       
error:                       | jest-docblock............................................................. DONE                                                                       
error:                       | jest-each................................................................. DONE                                                                       
error:                       | jest-environment-jsdom.................................................... DONE                                                                       
error:                       | jest-environment-node..................................................... DONE                                                                       
error:                       | jest-get-type............................................................. DONE                                                                       
error:                       | jest-haste-map............................................................ DONE                                                                       
error:                       | jest-jasmine2............................................................. DONE                                                                       
error:                       | jest-leak-detector........................................................ DONE                                                                       
error:                       | jest-matcher-utils........................................................ DONE                                                                       
error:                       | jest-message-util......................................................... DONE                                                                       
error:                       | jest-mock................................................................. DONE                                                                       
error:                       | jest-phabricator.......................................................... DONE                                                                       
error:                       | jest-regex-util........................................................... DONE                                                                       
error:                       | jest-repl................................................................. DONE                                                                       
error:                       | jest-resolve.............................................................. DONE                                                                       
error:                       | jest-resolve-dependencies................................................. DONE                                                                       
error:                       | jest-runner............................................................... DONE                                                                       
error:                       | jest-runtime.............................................................. DONE                                                                       
error:                       | jest-serializer........................................................... DONE                                                                       
error:                       | jest-snapshot............................................................. DONE                                                                       
error:                       | jest-util................................................................. DONE                                                                       
error:                       | jest-validate............................................................. DONE                                                                       
error:                       | jest-watcher.............................................................. DONE                                                                       
error:                       | jest-worker............................................................... DONE                                                                       
error:                       | pretty-format............................................................. DONE                                                                       
error:                       |                                                                                                                                                       
error:                       | Building browser packages                                                                                                                             
error:                       | jest-mock................................................................. DONE                                                                       
error:                       | pretty-format............................................................. DONE                                                                       
error:                       | expect.................................................................... DONE                                                                       
error:                       | Done in 11.11s.                                                                                                                                       
error:                       | Done in 28.16s.                                                                                                                                       
error:                       | yarn run v1.13.0                                                                                                                                      
error:                       | $ yarn typecheck && yarn lint && yarn jest                                                                                                            
error:                       | $ flow check --include-warnings                                                                                                                       
error:                       | Found 0 errors                                                                                                                                        
error:                       | $ eslint . --cache --report-unused-disable-directives --ext js,md                                                                                     
error:                       | $ node ./packages/jest-cli/bin/jest.js                                                                                                                
error:                       | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.                                                                  
error:                       | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.                                                                  
error:                       |                                                                                                                                                       
error:                       | FAIL packages/jest-diff/src/__tests__/diff.test.js                                                                                                    
error:                       | ● oneline strings                                                                                                                                     
error:                       |                                                                                                                                                       
error:                       | expect(value).toMatchSnapshot()                                                                                                                       
error:                       |                                                                                                                                                       
error:                       | Received value does not match stored snapshot "oneline strings 1".                                                                                    
error:                       |                                                                                                                                                       
error:                       | - Snapshot                                                                                                                                            
error:                       | + Received                                                                                                                                            
error:                       |                                                                                                                                                       
error:                       | - "<green>- Expected</>                                                                                                                               
error:                       | - <red>+ Received</>                                                                                                                                  
error:                       | + "- Expected                                                                                                                                         
error:                       | + + Received                                                                                                                                          
error:                       |                                                                                                                                                       
error:                       | - <green>- ab</>                                                                                                                                      
error:                       | - <red>+ aa</>"                                                                                                                                       
error:                       | + - ab                                                                                                                                                
error:                       | + + aa"                                                                                                                                               
error:                       |                                                                                                                                                       
error:                       | at Object.toMatchSnapshot (packages/jest-diff/src/__tests__/diff.test.js:88:28)                                                                       
error:                       |                                                                                                                                                       
error:                       | ● oneline strings                                                                                                                                     
error:                       |                                                                                                                                                       
error:                       | expect(value).toMatchSnapshot()                                                                                                                       
error:                       |                                                                                                                                                       
error:                       | Received value does not match stored snapshot "oneline strings 2".                                                                                    
error:                       |                                                                                                                                                       
error:                       | - Snapshot                                                                                                                                            
error:                       | + Received                                                                                                                                            
error:                       |                                                                                                                                                       
error:                       | - "<green>- Expected</>                                                                                                                               
error:                       | - <red>+ Received</>                                                                                                                                  
error:                       | + "- Expected                                                                                                                                         
error:                       | + + Received                                                                                                                                          
error:                       |                                                                                                                                                       
error:                       | - <green>- 123456789</>                                                                                                                               
error:                       | - <red>+ 234567890</>"                                                                                                                                
error:                       | + - 123456789                                                                                                                                         
error:                       | + + 234567890"                                                                                                                                        
error:                       |                                                                                                                                                       
error:                       | at Object.toMatchSnapshot (packages/jest-diff/src/__tests__/diff.test.js:89:42)
...

There's obviously something wrong with colors.

/cc @SimenB

[SimenB]

Thanks for opening up this issue! Yeah, something is wrong with colors. I tried a FORCE_COLOR=1 in #560 (comment), but that was too much (I can't remember the details). I'll try to find the time either today or this weekend to look into it

[BridgeAR]

@SimenB there are two open PRs in Node.js that might improve the situation for Jest (I did not check what issue it is really about):

nodejs/node#26261
nodejs/node#26485

Update: Thinking about it again, it will likely not have any impact.

[SimenB]

Oh, cool!

I think the issue is with how citgm spawns the subprocesses - it does not handle colors correctly. IIRC setting FORCE_COLOR made everything colorized, even things that shouldn't be. But that's just a shot in the dark, I haven't investigated

[BridgeAR]

FORCE_COLOR is indeed going to activate colors on everything, so this won't help.

Can you point me to the code in JEST that adds the coloring? I am just trying to wrap my head around what's really going on.

[SimenB]

We do it in a bunch of places - just grep for chalk.

The test failure in the snippet above is since this serializer does not find any ansi escapes: https://github.com/facebook/jest/blob/b63b279ca832613d2685611d3f98ee175250e9ac/packages/pretty-format/src/plugins/ConvertAnsi.ts
Code adding these specific colors: https://github.com/facebook/jest/blob/b63b279ca832613d2685611d3f98ee175250e9ac/packages/jest-diff/src/diffStrings.ts

[BridgeAR]

The most likely reason I see that the colors are not returned are that the output stream used by CITGM on our CI is that it's not a tty. Chalk is going to check if colors or supported and one of those checks is if it's a tty or not. When using FORCE_COLOR it will apply to all output streams. So even for streams that are indeed not meant to output colors.

[targos]

Can we simulate/pretend to be a tty stream from citgm?

[BridgeAR]

@targos that might be possible.

[BridgeAR]

@SimenB what about adding a few libraries that use jest as testing framework? That way we would at least have noticed the recent issue that broke jest and we would probably detect other similar things as well.

[SimenB]

That works for me! E.g. React #625

[BridgeAR]

@SimenB that sadly fails on Node.js master. It would have to be something that succeeds with master.

[SimenB]

Ah, ok. We can do https://github.com/jest-community/eslint-plugin-jest? And remove it whenever a more fitting project using Jest (or Jest itself) is added to the lookup?

[BridgeAR]

@SimenB if it passes the tests with Node.js master, that should be fine. Would you be so kind and open a PR?

[SimenB]

Yup it passes (at least 12.0.0-nightly20190313e6fa50e953), with a YARN_IGNORE_ENGINES=true env variable