Don't cache NPM packages with version set to "*" or which aren't in package-lock

Hey guys,

One issue I’m running into a decent amount is related to the build cache. I maintain a private node package that contains a lot of core functionality shared between a few of my sites, like UI components, a bit of shared logic, and an API service to talk to my backend. I mean, who wants to write that stuff more than once? Not me!

Because I fully control this package and don’t have to worry about unexpected breaking changes, I don’t feel I need to lock it to a specific version, so I keep the package out of my package-lock and set the version to “*” in my package.json.

I did some updates to all my sites and to my core package today, and after doing some beta branch deploys with extra logging enabled, I found that they were still running the old version of the core package. One was several versions out of date. This is a pretty common thing for me to find.

The purpose of marking a package as using “*” (and keeping it out of package-lock) is to make sure it’s always running the very latest version at build time. While caching may be useful for other packages which are listed in the package-lock, caching those that aren’t creates unexpected behavior.

I understand the reason for the build cache is to help aleviate some of the strain on the system when building sites, but it seems to me that a configuration like mine, where the package is set to use version “*” or where a package is specifically kept out of package-lock, should be pretty rare and put a relatively small strain on the system overall if those packages were reinstalled every time. This is especially true when you consider that the current way to get around this is to wait for the new build to start, cancel it (or wait until it completes) then clear the entire build cache and re-run the build.

It would probably end up being more efficient on your end to allow packages with this configuration to avoid the build cache. Consider the two possible workflows:

  1. Push changes to git
  2. Build starts and finishes
  3. “Ah, this isn’t right.”
  4. Clear entire build cache and rebuild
    Result: two full builds (maybe 1.5 if I cancel), plus the entire cache being cleared and all packages being reinstalled from scratch

versus

  1. Push changes to git
  2. You reinstall the one package with this config, use the cache for the rest
  3. “This looks good.”
    Result: One build with one package install, and everything else coming out of the cache

And while you might think this setup would incentivize people to use “*” versions as a means of bypassing the build cache whenever they want, it really wouldn’t since doing so for all packages would actually lead to a ton of breaking changes and be a very bad idea. This would only ever make sense for a subset of packages fully controlled by the owners of the sites on which they’re being used.

As an engineer myself, I realize “this would probably only ever be used by a small subset of users” is the dumbest possible argument for why a feature should be implemented, but in this case I think it makes sense since you’d see at least some of the build-rebuild cycle listed above go away.

Just a thought!

Hey @MikeBeas, sorry to be slow to get back to you!

We’ve talked about this internally and we think your best bet is to actually file an issue on

so we can get the right people the right information about this and can decide whether this is something we want to work on! thank you so much for bringing this up.

Thanks, I’ll do that!

1 Like

great! thank you :netliheart: