npm in Node 6.2.0 MSI does not work in Windows Docker container

[StefanScherer]

• Version: 6.2.0
  • Platform: Windows Server 2016 TP5, Microsoft Windows [Version 10.0.14300]
  • Subsystem: Windows Server Core Docker Container

Node.js 6.x has a problem running inside a Windows Docker container. While filling out this template I drilled down that the problem seems to be npm 3.x. You can find the Dockerfiles to build the Windows Docker images with Node at https://github.com/StefanScherer/dockerfiles-windows/tree/master/node

Node 6.2.0:

Running an interactive Windows Docker container with Node.js 6.2.0 shows an error at npm:

C:\> docker run -it stefanscherer/node-windows:6 cmd

Microsoft Windows [Version 10.0.14300]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\>node --version
v6.2.0

C:\>npm --version
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3

But the file is there:

C:\>dir "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js"
 Volume in drive C has no label.
 Volume Serial Number is BA9B-FAE0

 Directory of C:\Program Files\nodejs\node_modules\npm\bin

05/06/2016  10:01 PM             2,073 npm-cli.js
               1 File(s)          2,073 bytes
               0 Dir(s)  21,217,292,288 bytes free

C:\>type "C:\Program Files\nodejs\node_modules\npm\package.json"
{
  "version": "3.8.9",
  "name": "npm",
  "description": "a package manager for JavaScript",

Node 4.4.5:

Running an interactive Windows Docker container with Node.js 4.4.5 works fine.

C:\> docker run -it stefanscherer/node-windows:4 cmd

Microsoft Windows [Version 10.0.14300]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\>node --version
v4.4.5

C:\>npm --version
npm info it worked if it ends with ok
npm info using npm@2.15.5
npm info using node@v4.4.5
npm info ok
2.15.5

Also the onbuild version works on Windows for Node.js apps with a Dockerfile like this:

FROM stefanscherer/node-windows:4-onbuild

and

docker build -t example .

Or even running inside a nanoserver container eg. on Windows 10 Insider 14352 with Docker installed:

FROM stefanscherer/node-windows:4-nano-onbuild

and

docker build --isolation=hyperv -t example .

[joaocgreis]

cc @nodejs/docker

@StefanScherer I can't help much with docker, but there are some things you could try. First, does node "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" --version work? npm is a cmd script that should call exactly that.

If not, I guess investigating permissions may make sense. You can use a tool like AccessEnum or icacls on the command line. I think node stats all the directories up to a file in some cases, so try some thing like

icacls "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js"
icacls "C:\Program Files\nodejs\node_modules\npm\bin"
icacls "C:\Program Files\nodejs\node_modules\npm"
icacls "C:\Program Files\nodejs\node_modules"
icacls "C:\Program Files\nodejs"
icacls "C:\Program Files"

and compare that with

icacls "C:\Program Files\nodejs\node.exe"
whoami /groups

[StefanScherer]

@joaocgreis Thank you, but inside a Windows container all processes are running as containeradmin (something like root in Linux containers).

I tested a little more and found out that the problem is not npm, but the node.exe 6 interpreter. I've installed Node.js 6.2.0 MSI inside a Windows container and checked the versions. As described above node.exe works, but npm doesn't work:

Microsoft Windows [Version 10.0.14300]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\>node --version
v6.2.0

C:\>npm.cmd --version
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3

Then I also downloaded just the Node.js 4.4.5 Exe file 64bit and copied it over the node.exe 6.2.0. Now both node.exe and npm work:

C:\>copy node445.exe "\Program Files\nodejs\node.exe"
Overwrite \Program Files\nodejs\node.exe? (Yes/No/All): y
        1 file(s) copied.

C:\>node --version
v4.4.5

C:\>npm.cmd --version
npm info it worked if it ends with ok
npm info using npm@3.8.9
npm info using node@v4.4.5
npm info ok
3.8.9

As you can see with node.exe 4.4.5 I'm able to run npm.cmd in version 3.8.9. So there seems to be a code change that is causing the problem.

[joaocgreis]

If you download node.exe 6.2.0 and replace, does it work? Just to make sure it's not a problem with the bundled exe. Can you test both 32 and 64 bit? Thanks!

[StefanScherer]

In the meantime I've added my test Dockerfile to download some more versions and then run the Windows container interactive with

FROM windowsservercore

ENV NPM_CONFIG_LOGLEVEL info
ENV NODE_VERSION 6.2.0
ENV NODE_SHA256 bd1d766e15f75f67befa76738bc0212ee3016444eb0396b29c6fc319658e21a2

RUN powershell -Command \
    wget -Uri https://nodejs.org/dist/v%NODE_VERSION%/node-v%NODE_VERSION%-x64.msi -OutFile node.msi -UseBasicParsing ; \
    if ((Get-FileHash node.msi -Algorithm sha256).Hash -ne $env:NODE_SHA256) {exit 1} ; \
    Start-Process -FilePath msiexec -ArgumentList /q, /i, node.msi -Wait ; \
    Remove-Item -Path node.msi

RUN powershell -command wget -uri https://nodejs.org/dist/v4.4.5/win-x64/node.exe -outfile c:\node4.4.5.exe
RUN powershell -command wget -uri https://nodejs.org/dist/v5.11.1/win-x64/node.exe -outfile c:\node5.11.1.exe
RUN powershell -command wget -uri https://nodejs.org/dist/v6.0.0/win-x64/node.exe -outfile c:\node6.0.0.exe
RUN powershell -command wget -uri https://nodejs.org/dist/v6.1.0/win-x64/node.exe -outfile c:\node6.1.0.exe

CMD [ "node.exe" ]

docker build -t nodetest .
docker run -it nodetest cmd

C:\>node --version
v6.2.0

C:\>copy node5.11.1.exe "\Program Files\nodejs\node.exe"
Overwrite \Program Files\nodejs\node.exe? (Yes/No/All): y
        1 file(s) copied.

C:\>npm.cmd --version
npm info it worked if it ends with ok
npm info using npm@3.8.9
npm info using node@v5.11.1
npm info ok
3.8.9

C:\>copy node6.0.0.exe "\Program Files\nodejs\node.exe"
Overwrite \Program Files\nodejs\node.exe? (Yes/No/All): y
        1 file(s) copied.

C:\>npm.cmd --version
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:165:25)
    at Function.Module._resolveFilename (module.js:436:25)
    at Function.Module._load (module.js:386:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:159:18)
    at node.js:444:3
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:165:25)
    at Function.Module._resolveFilename (module.js:436:25)
    at Function.Module._load (module.js:386:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:159:18)
    at node.js:444:3

It seems that beginning with Node 6 the problem occurs.

I only test the 64bit version as eg. nanoserver containers are only able to run 64bit binaries. My tests are in windowsservercore containers, but 32bit is not a real option.

[joaocgreis]

Looks like a regression. Can it run other javascript files?

[StefanScherer]

Doesn't look like, neither npm start nor node app.js work with Node.js 6.2.0 exe file:

Microsoft Windows [Version 10.0.14300]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\app>node --version
v4.4.4

C:\app>copy \node6.2.0.exe "\Program Files\nodejs\node.exe"
Overwrite \Program Files\nodejs\node.exe? (Yes/No/All): y
        1 file(s) copied.

C:\app>node --version
v6.2.0

C:\app>npm start
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3

C:\app>node app.js
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, realpath 'C:\app\app.js'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:449:3

[StefanScherer]

Always the problem with binding.realpath(pathModule._makeLong(path), options.encoding);

[StefanScherer]

Digging deeper inside a container. Overwritten the fs.realpathSync functions manually:

C:\app>node
> const binding = process.binding('fs');
undefined
>
(To exit, press ^C again or type .exit)
>

C:\app>node
> 'use strict';
'use strict'
> const constants = process.binding('constants').fs;
undefined
> const util = require('util');
undefined
> const pathModule = require('path');
undefined
> const binding = process.binding('fs');
undefined
> fs.realpathSync = function realpathSync(path,options) {
... if (!options)
...   options = {};
... else if (typeof options === 'string')
...   optinos = {encoding: options};
... else if (typeof options !== 'object')
...   throw new TypeError("asdfsdf");
... console.log('rps: path',path);
... console.log('rps: options',options);
... console.log(pathModule._makeLong(path));
... return binding.realpath(pathModule._makeLong(path), options.encoding);
... };
[Function: realpathSync]
> require('./app.js');
rps: path C:\app\app.js
rps: options {}
\\?\C:\app\app.js
Error: ENOENT: no such file or directory, realpath 'C:\app\app.js'
    at Error (native)
    at Object.realpathSync (repl:11:16)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:438:25)
    at Function.Module._load (module.js:388:25)
    at Module.require (module.js:468:17)
    at require (internal/module.js:20:19)
    at repl:1:1
    at REPLServer.defaultEval (repl.js:272:27)
    at bound (domain.js:280:14)
>

Found #6861 which sounds very related

[addaleax]

#6861 fails with EISDIR and appears to be related to certain kinds of network drives, the only connection I see is that both probably are results of the fs.realpath change in v6. (There’s also a typo, optinos, in your REPL excerpt above, but I’m not sure that’s relevant here).

[StefanScherer]

Tried a similar command node -p "fs.realpathSync('C:\windows')" like in #6861 this shows the error very fast:

Microsoft Windows [Version 10.0.14300]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\>node -p "fs.realpathSync('C:\windows')"
fs.js:1568
  return binding.realpath(pathModule._makeLong(path), options.encoding);
                 ^
Error: ENOENT: no such file or directory, realpath 'C:\windows'
    at Error (native)
    at Object.realpathSync (fs.js:1568:18)
    at [eval]:1:4
    at Object.exports.runInThisContext (vm.js:54:17)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:541:32)
    at node.js:333:29
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

C:\>dir windows
 Volume in drive C has no label.
 Volume Serial Number is BA9B-FAE0

 Directory of C:\windows

05/30/2016  12:57 PM    <DIR>          .
05/30/2016  12:57 PM    <DIR>          ..

I think both issues are related as uv_fs_realpath does not work properly with mapped drives/pathes. A windows container's C: drive is of course mapped to another directory at the host. No UNC, but in any kind a mapped directory.

Don't know how to help, any advice? Run the unit tests of libuv inside a container?

[saghul]

Similar to #7192, I suspect this might have to do with reparse point handling (or lack thereof ) on the realpath implementation.