When deploying from a Git repo, is there a way to upload a private key file that's not in your repo

[briansteeleca]

My demo site is built with Svelte/Sapper and uses the npm google-spreadsheet to connect to Google Sheets. https://gbps.netlify.app/

The Google Sheets API provides a credentials.json file for authentication. I’ve created a Netlify function that references the credentials JSON file like so:

await doc.useServiceAccountAuth(require('./credentials.json'));

I’m deploying from my Git repo, but I don’t want my credentials.json file to be in the repo. Is it possible to upload my credentials file to my functions folder without putting it in my repo?

Some more info…

My preferred method was to put all of my credentials in Environment Variables through the Netlify UI…

…and reference them in my function like so:

const clientSecret = { "type": "service_account", "project_id": process.env.PROJECT_ID, "private_key_id": process.env.PRIVATE_KEY_ID, "private_key": process.env.PRIVATE_KEY, "client_email": process.env.CLIENT_EMAIL, "client_id": process.env.CLIENT_ID, "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": process.env.CLIENT_X509_CERT_URL }; await doc.useServiceAccountAuth(clientSecret);
But for some reason, that always fails on build. (Never the first time for some reason, but on all subsequent builds.)

When I make the credentials.json file external again, it builds without issues. (Again, never the first time, but on all subsequent builds.)

After days of trying everything I can think of to make it work with env vars, I’m (for now) resigned to leaving my credentials.json file external.

I’d greatly appreciate any suggestions on how to use my credentials file without having to keep it in my repo, or even suggestions on what I may be doing wrong with environment variables (I’m happy to provide more code).

Thanks!

[AaronP]

I would think the env variables route is the way to go. Have you been able to dig deeper into why the build fails? Just checking that the build doesnt rely on the function since the function will run in a different environment and the build wont access it. Only at run time.

I would also note that when you’re using env variables theyre always strings, so when you’ve got “—private key here —” in your env variable it will actually be “”—private key here"" which means it would be incorrect for your config.

Also, im not sure how long the env variables can be but maybe the whole json can go in and you’d parse it in the function?

[briansteeleca]

Thanks Aaron, you’ve definitely given me some things to think about. I thought that the Netlify function was being called on build because when I export from Sapper, I can log the data retrieved Sapper’s preload function. I’m still trying to wrap my head around this though! Either way, I’d better check that it fails gracefully.

Good to know about quotes on the env variables. I tried with and without quotes on the private key, but knowing they shouldn’t be there will rule out one potential issue. I’ll use your suggestions to give the it another shot once I recover from the first couple troubleshooting sessions.

[perry]

hi there, here is a great guide for using environment variables:

[Support Guide] Using environment variables on Netlify correctly

Last reviewed: August 2023 The preferred way to use environment variables in Netlify is to set them in our environment - be that in the Environment Variables settings UI (on the site’s “Build & Deploy settings” page), or via netlify.toml. You’ll want to choose the right pattern to meet your needs. Setting variables in our UI may be considered a bit more secure, since only people with access to the Netlify admin UI for your site can see them, rather than “anyone who can clone your repo!”. Thes…

[fool]

Also, here’s an explicit example of how to put a private key into variables, to use in the build! I prefer separate variables to trying to handle the \n in the quoted string(s) since that is so hard to handle in the shell. 28 year shell scripting professional here…so I have a ton of baggage

[Support Guide] Using an SSH key via environment variable during build

Last checked by Netlify Support Team: December 2022 Some workflows require use of an SSH private key during build - for instance, logging in to a server to restart your backend, or using git clone on a private repo during build. There are better ways to accomplish this goal for most use cases, but in case you choose to do that, below is an approach that can work. First, create an ssh key and add the public key (shorter one) to your service (e.g. on your GitHub repo settings, or in ~/.ssh/auth…

[briansteeleca]

Thanks so much @fool for letting me know why it’s failing!

Maybe there’s a reason I shouldn’t do this, but I’m now replacing _ with \n right in my Netlify function. I hope it makes sense to do it that way – since I made the change, my build hasn’t failed.

[LeaVerou]

Unfortunately env variables for something like this is not very portable, and doesn’t scale well.

Some ideas that would address this and potentially a host of other use cases:

• What about allowing private files to be gitignored and deployed via the Netlify CLI?
• What about being able to deploy from a public and a private repo, and have their file structures merged on the deployed site?

[hrishikesh]

I don’t anticipate any of those things to happen any time soon. What makes you think variables are not scalable?