bug: overrides property only honored when running install the first time

[lukekarrys]

Opening a new issue since #4232 is getting crowded with other possibly unreleated bug reports. But this one I have confirmed.

From: #4232 (comment)

---

Note that the INITIAL install will abide by the override rules set, and the subsequent installs (e.g., run npm install twice) will ignore overrides.

I can confirm this is the behavior in the latest npm@8.19.2. This can be reproduced easily with the following package.json:

{
  "name": "test",
  "version": "1.0.0",
  "engines": {
    "npm": ">=8.3.0"
  },
  "dependencies": {
    "json-server": "^0.17.0"
  },
  "overrides": {
    "json-server": {
      "package-json": "7.0.0"
    }
  }
}

1. npm install in the folder containing only the above package.json --> 0 vulnerabilities
2. Subsequent npm install right after the previous (so node_modules and package-lock.json exists) --> 5 vulnerabilities
3. npm update --> 0 vulnerabilities
4. rm -rf node_modules/ && npm install --> 5 vulnerabilities
5. rm package-lock.json && npm install --> 5 vulnerabilities
6. rm -rf node_modules/ && rm package-lock.json && npm install --> 0 vulnerabilities

From the above it can be concluded that the overrides property is only honored when running npm install first time (i.e. without package-lock.json and node_modules present) and when running npm update.

[AlonNavon]

I can confirm that I have the same issue on the latest npm@9.6.6.

This is really disruptive for our CI. Anyone working with a committed package-lock.json isn't running the overrides for the first time. So in a sense, it's not functional for them.
Moreover, since it reverts to the overridden version quietly, a lot of people probably have no idea that the vulnerable library they've overridden has come back to haunt them.

I think this bug should be prioritized.

[jakubmazanec]

I too think this bug should be a priority. overrides is critical functionality for our team and this bug makes lockfile practically useless.

[bnbdr]

This reproduced for me on 9.6.6, but works as expected when specifying the override 'globally':

{
  "name": "test",
  "version": "1.0.0",
  "dependencies": {
    "json-server": "0.17.0"
  },
  "overrides": {
    "package-json": "7.0.0"
  }
}

[ecl1ps]

Hi, this problem is still present in npm 9.6.7. It causes huge issues for a long time and can cause security vulnerabilities for developers unaware of this bug.

I have created a more thorough repro case.

Initial state
Monorepo package package1 depends on cssstyle@2.2.0 and it depends on cssom@~0.3.6.
After running the initial npm install the lockfile contains:

        "node_modules/cssom": {
            "version": "0.3.8",
        },
        "node_modules/cssstyle": {
            "version": "2.2.0",
            "dependencies": {
                "cssom": "~0.3.6"
            },
        }

After adding an override and running the npm install there are no changes to the lockfile nor the installed packages. The override is completely ignored.

After changing the version of the cssstyle to 2.3.0 and running the npm install the are the following changes:

        "node_modules/cssom": {
            "version": "0.3.8",
        },
        "packages/package1/node_modules/cssstyle": {
            "version": "2.3.0",
            "dependencies": {
                "cssom": "~0.3.6"
            },
        }

Now the override is still not honored and on top of that the cssstyle is moved under packages/package1/node_modules/ for no apparent reason. It causes a duplication of this package in the monorepo. And that is another problem with some packages that require you to have only one version of it in the repo (like react and many others).
Without having the override set up this move wouldn't occur.

The only workaround we found to work is to uninstall and then install that package again when touching the override or the package's version. Which is really really cumbersome in large monorepos.

Running npm uninstall cssstyle --workspace packages/package1 && npm install cssstyle@2.3.0 --workspace packages/package1 finally produces the correct outcome.

        "node_modules/cssom": {
            "version": "0.3.6",
        },
        "node_modules/cssstyle": {
            "version": "2.3.0",
            "dependencies": {
                "cssom": "~0.3.6"
            },
        }

I hope this helps to move this issue further. Feel free to ask for additional info.

[AlonNavon]

Also, I'd like to pile on that as this issue can cause packages to silently revert to vulnerable versions, IMHO this is not a regular bug, but a security issue, that should also be fixed on the latest version 8.
It's fair to assume that the major reason for tinkering with an internal dependency is a security issue, and these silent reverts are a security incident waiting to happen.