Functions and node_modules

I’m testing locally with netlify dev and my function works perfectly. When I commit to Github, my deploy fails because it can’t find a required dependency under functions/myfunc. Obviously I’ve got node_modules in .gitignore and I assumed that Netlify would run npm i automatically on build for functions. Since it isn’t, how do I correct this?

Hi @cfjedimaster,

One option is to include your dependencies in your root package.json. If you have a second package.json in your functions directory then you can chain their installation as part of your build command using &&.

Ok… so… are you saying use a package.json at project root with a script that runs npm i for the function subdirectory?

Also - I tried adding netlify-lambda to my project. In my root package.json, I then tried netlify-lambda build functions'. This is documented here: https://www.netlify.com/docs/functions/#tools-for-building-javascript-functions. Doing so gives me a new error:

netlify-lambda Error: Function source folder (specified in netlify-lambda serve/build command)
9:18:39 AM:       and publish folder (specified in netlify.toml)
9:18:39 AM:       should be different.

As an update - I’ve now tried specifying ./functions in my site settings. I have that same value in netlify.toml, but maybe it will help? Currently my deploy is enqueued so I’m waiting.

hi raymond!

Ok… so… are you saying use a package.json at project root with a script that runs npm i for the function subdirectory?

yes. infact i like to do this as a prebuild or preinstall hook:

{
  "scripts": {
    "prebuild" : "cd functions/myfunction && npm i",
   "build": "gatsby build"
  }
}

you can also stick it inside the netlify.toml command field, but thats really up to personal preference. bottom line is, you’re responsible for writing the bash commands to install any dependencies that aren’t at the root.

i’m well aware this isn’t ideal. this may change in future iterations of the build bot that we are currently prototyping. but its what we have for now.

Also - I tried adding netlify-lambda to my project. In my root package.json, I then tried netlify-lambda build functions '. This is documented here: https://www.netlify.com/docs/functions/#tools-for-building-javascript-functions. Doing so gives me a new error:

this is almost a different topic altogether. to use netlify-lambda, be sure to read the docs: “At a high level, netlify-lambda takes a source folder (e.g. src/lambda , specified in your command) and outputs it to a built folder, (e.g. built-lambda , specified in your netlify.toml file).”

given this it would evidently be a bad idea to have your source and dist folders be the same folder. this is what the error is warning you about. you’re coming from a netlify dev workflow, which has no build step, so you don’t have a source and dist folder. For an easy fix, change your functions key inside netlify.toml to something else.

netlify-lambda serves slightly different and overlapping needs than netlify dev. I did my best to explain in the docs: https://github.com/netlify/netlify-lambda#netlify-lambda “Netlify Dev vs Netlify-Lambda”

This will be updated with a future version of our build system.

In the meantime you can use this script that will automatically loop through your functions directories and ensure that the dependancies are installed.

and run this in the prebuild scrip of your package.json like so:

  "scripts": {
    "prebuild": "node ./scripts/installer.js",
    "build": "react-scripts build"
  },

Thank you both. I was in a meeting the last 40 minutes or so and was able to quickly test.

So first - yeah, netlify-lamda was a mistake on my part. I was just throwing crap at the wall to see what would stick. :wink: I’m going to read your link, but it seems that for how I use Netlify Functions, I can probably avoid it.

Running npm i on the subdirectory worked great, and David, your script to do this dynamically is cool too.

I’m going to write this up in a blog post to specifically cover the use case of:

  • using netlify dev locally to test
  • using the CLI to scaffold an app
  • using code that requires node modules (which will be most functions)
  • and using Git to deploy where you are not committing node_modules

It sounds like yall may have a way to make this easier in the future too, which is good. Honestly this feels like something I’d have trouble documenting, but until you do get the fix working, maybe a quick reminder at the end of https://www.netlify.com/docs/functions/#javascript-lambda-functions? Before you get into “Tools for Building JavaScript Functions”. And actually, isn’t that section a bit out of date now with the way the netlify function can scaffold an application and netlify dev can run it for you?

Finally - thank you all!

Hi @DavidWells, is this installer.js necessary, or is it now included in the build system?

It’s not available yet, and won’t be for you for at least a little while. We’ll post more widely about the new build system as things develop :slight_smile:

I just wanted to share here that the new and improved Netlify docs include a couple improvements relevant to this discussion.