[Common Issue] How do I access private repositories in the build environment?

answered
#1

The link from your repository to our service only authenticates us against the “main” repo that you first configured, and won’t automatically allow access to submodules even if they also belong to you and you’ve granted Netlify access to those repositories.

In case your submodule is public , you can change the URI schema for it to start with https:// and it will should work without further adjustment (so, in .gitmodules , change git@github.com… to https://github.com/… )

If you have a private submodule, you’ll nee to grant permissions for our build system to access your private submodule. This also needs some assistance from our Support team. You’ll need to do the following:

  1. Assign a deploy key (a site specific SSH key that is generated by our Support Team) as a read-only deploy key in your repository settings at GitHub, which can be found here: https://github.com/<your github account>/<your repository name>/settings/keys.

    NOTE: It will only work on the site the key is generated for; we have to generate separate ones for other sites, so feel free to ask if you need that.

  2. Once it’s in place, when we check out the first repo to build, we’ll fetch the submodule with this deploy key and this will work!

  3. You can use this workflow with multiple private submodules attached to the same repository, but that does take some extra work since GitHub does not allow Deploy Keys to be used in multiple places. You’ll have to instead add them to a user (perhaps you? or maybe a robot-account that you create on GitHub) that has access to all repos.

If you want to access private repositories not configured as a submodule, it can be a bit of a challenge in the build environment! We do have git and ssh in the build-image, so the tools are there. What’s missing is any authentication credentials at all - we don’t even have permission to fetch the same repo we are building from within the build environment. So, you’ll need to supply whatever you need to access that repository somehow.

There are three ways we know of to do it (and probably a thousand that we do not, feel free to tell us about different ways to do this). The best way to do this requires support of your package manager and Git provider, so it may not be an option depending on your toolchain. All of these options include storing access tokens in your repository or accessing them via secrets stored there.

If you think that could work, there are a couple of ways to get that info into our build environment:

  1. If your dependency manager (this works with npm at least) supports it, and your package.json is securely stored, you can add that dependency using a GitHub or GitLab access token like this: git+https://<github_token>:x-oauth-basic@github.com//.git (GitLab also documents this). If you use BitBucket, you can to use an app password in the same way.
  2. Simplest, but not recommended, is to have the ssh private key as part of your repository. Then you can just use it natively (eg if you have /.ssh/id_dsa and it is password-less, ssh will just use it automatically - when used with or without git). This isn’t recommended, because you may have a lot of people with access to your repository that can now access the private dependency!
  3. Instead, you can add some access token to the “Build Environment” section of the settings for your site. This restricts access to only those who can see your Netlify account settings, rather than all who can see your repository/source code. This could be the contents of the SSH key (remember, the ssh private key is multi-line text and formatting-sensitive). It might be easier to pull down the key from a remote server using curl:

curl -dump https://username:${PASSWORD}@hostname/sshkey > ~/.ssh/id_dsa

To use this last trick successfully, you should realize that your build command can be anything you want that can run on a Linux host. See this community post about debugging your site and this blog post for more info about how to tell what is available and test things out locally.

If you’ve found another way to access private repositories, please leave a comment below.

[Common Issue] Using private NPM modules on Netlify
#2

Hi @Dennis,

How do I get in touch to get a “deploy key (a site specific SSH key that is generated by our Support Team)”?

I guess this might solve the issue I have open on the community topic I’ve created below:
Build Ok on local, images missing on Netlify (they are on a separate private repo and I have a git submodule))

Thanks
Pat

1 Like
[solved] Build Ok on local, images missing on Netlify (they are on a separate private repo and I have a git submodule))
#3

Hi @pamora, you can write in to support@netlify.com which looks like you already did.

1 Like