Netlify.toml context env variables do not apply to functions

answered
#1

Hopefully this saves someone some time (might be good to add the official documentation because I fell for this once in January and again today - although I realized faster why things weren’t working).

Question:

I overwrite some of my environment variables in my netlify.toml file, for my staging environment deploys. However, I’ve noticed that these overwrites only apply to the site, they don’t apply to my staging branch lambda functions. Is this a bug?

Support Answer (see update below, couldn’t get this to work):

It’s an open feature request to support environment variables from the toml file in functions.

In the meantime you may be able to leverage different build commands for deploy contexts per the TOML file to achieve your goal:

  1. set both variables in our UI, say “API_BACKEND_STAGE = https://stage.com” and “API_BACKEND_PROD = https://prod.com
  2. then create conditional build commands for each branch like this:
[context.production] 
    command = "export API_BACKEND=$API_BACKEND_PROD && npm run build"
[context.stage]
    command = "export API_BACKEND=$API_BACKEND_STAGE && npm run build"

and use only plain $API_BACKEND in your build

Update: I tried the above, where API_BACKEND_STAGE and API_BACKEND_PROD are both set in the Netlify UI, but API_BACKEND is not available as an environment variable when my function executes. So it does not work. Will be looking for alternative solutions.

This is related to the discussion here: https://github.com/netlify/netlify-lambda/issues/59#issuecomment-477383840 They’ve created an example to reproduce the behavior.

Update: I tried eval instead of export in the example above, and API_BACKEND is still not available in process.env when my function executes. I also tried both export and eval API_BACKEND=Test just to see if I could hard code something, but that also failed, when the staging function executes it does not have API_BACKEND in process.env

2 Likes
#3

I tried adding env variables in the build command, it didn’t work for me either.

In the same github issue mentioned above, I added a temporary workaround by just creating another netlify app that points to the same github repo but to the branch “staging” (or something), you can see it here: https://github.com/netlify/netlify-lambda/issues/59#issuecomment-482454231.

This is far from perfect and it might still need some clever solutions, like adding the functions endpoint to the process.env as well (e.g. process.env.ENDPOINT) so if your are hosting your website in the same netlify app, you can easily know which is your target domain.

1 Like
#4

Hey folks!

It is a known shortcoming that the netlify.toml file environment variables are not usable in functions. You must set them in our UI in the Build & Deploy settings page, and then to use them, you must ALSO change your function in some way to redeploy it. We snapshot your function by checksum and won’t redeploy it in case you only change environment variables…which means those new variables won’t be used since they weren’t there in the old snapshot - you have to change its checksum in some way to trigger this.

We’re working on an article about some of those limitations of functions so you’ll be able to find advice like that here in the near future!

#7

thanks!

It is a known shortcoming that the netlify.toml file environment variables are not usable in functions. You must set them in our UI in the Build & Deploy settings page, and then to use them, you must ALSO change your function in some way to redeploy it.

Just to be clear, this means that there is no way to have specific Env Var values for your branch deploy functions. Because all functions (regardless of what branch they are associated with) will use the Env Vars that were set in the UI. There is no workaround that solves this issue (aside from hosting a separate site and using it purely as a staging environment as suggested by @pgarciacamou)

1 Like
#8

There is a workaround :slight_smile: It’s to have a more intelligent build script that selects the appropriate variables to use during build. One can use contexts to choose a different build command, so something like this (untested) is what I might use:

  • in the UI, set $STAGING_ENDPOINT and $PRODUCTION_ENDPOINT to different values
  • in netlify.toml use build commands like this to incorporate them:
[context.staging]
  command = "./my-production-build-script.sh"
[context.production]
  command = "./my-staging-build-script.sh"

and then have those two scripts use separate environment variables. Another pattern using a makefile as shown here would work too: https://www.netlify.com/blog/2017/04/07/selective-password-protection/

#9

It’s to have a more intelligent build script that selects the appropriate variables to use during build.

So both me and @pgarciacamou have tried the intelligent build script approach (which we mentioned above) and haven’t gotten it to work. It seems that setting environment variables within a custom build command does not actually pass those variables to the env where the Function executes.

For example, if you try

[context.production]
     command = "export API_BACKEND=TEST && gatsby build"

and run console.log(process.env.API_BACKEND) within your lambda function, API_BACKEND won’t exist.

Have also tried eval instead of export.

1 Like
#10

Hey Chris (alias @fool),

Here goes nothing.

I tried again without eval/export, by directly assigning the env variable to the command, see: https://github.com/pgarciacamou/netlify-context-in-previews/pull/3 and same issue.

[context.deploy-preview]
  command = "WHO_AM_I=DeployPreview npm run build"

The Deploy Preview is generated/built correctly but the env variable is the value from the envs set in the UI:

https://deploy-preview-3–netlify-context-in-previews.netlify.com/.netlify/functions/hello

Basically, it should read “Hello DeployPreview” but instead returns “Hello Build”.

2 Likes
#11

I’m seeing exactly the same issue as what @pgarciacamou mentioned. I’ve just tried the approach @fool suggests but got Permission denied.

8:29:23 PM: Executing user command: ./setenv.sh; npm run generate; netlify-lambda build lambda
8:29:23 PM: /usr/local/bin/build: line 32: ./setenv.sh: Permission denied
#12

Hi @teakwood and @pgarciacamou,

The issue you’re seeing is that exporting an env variable while executing a process makes that variable available to that process. However, when your function is deployed, the environment variables we pass into it are sent by our API, which is why you are limited to just those in the UI at the moment (until we get that bug fixed).

The only way to get environment variables from another source in to your functions without adding them via our UI is to embed them in to the file, which you can do manually or using a plugin like https://www.npmjs.com/package/webpack-plugin-replace and then you can do something like:

[context.production]
 command = "export API_BACKEND='This is a test' && node replace.js"

As was mentioned, we do have an open feature request for this. In the meantime I recommend specifying the environment variables in the UI, which has the added benefit of not exposing them to anyone that has access to your repo.

@nerimplo I don’t know if your issue is related. You would have to provide more details, like what’s in that script that you’re running.

3 Likes