Git submodule pull problem

I am having a problem updating git submodules during build phase. For some reason the build setup is using an older checkout, Manually updating a submodule via git submodule update --recursive --remote before building results it in a host key verification fail. The key is generated in deploy settings and added to gitlab ssh keys.

The site name is angry-easley-46326d

The build log file is

1:33:22 PM: Build ready to start

1:33:24 PM: build-image version: 6dfe19d15f524c85d6f9bf7df9fb30b0a9f0a61a

1:33:24 PM: build-image tag: v3.3.10

1:33:24 PM: buildbot version: 90cbc3789500c2dd7143d55a5b3bf1480401aa91

1:33:24 PM: Fetching cached dependencies

1:33:24 PM: Starting to download cache of 192.4MB

1:33:25 PM: Finished downloading cache in 1.027869974s

1:33:25 PM: Starting to extract cache

1:33:32 PM: Finished extracting cache in 6.887459177s

1:33:32 PM: Finished fetching cache in 7.958734118s

1:33:32 PM: Starting to prepare the repo for build

1:33:32 PM: Preparing Git Reference refs/heads/master

1:33:33 PM: Starting build script

1:33:33 PM: Installing dependencies

1:33:33 PM: Python version set to 3.7

1:33:34 PM: Started restoring cached node version

1:33:37 PM: Finished restoring cached node version

1:33:37 PM: v10.20.1 is already installed.

1:33:38 PM: Now using node v10.20.1 (npm v6.14.4)

1:33:38 PM: Attempting ruby version 2.6.2, read from environment

1:33:39 PM: Using ruby version 2.6.2

1:33:39 PM: Using PHP version 5.6

1:33:40 PM: 5.2 is already installed.

1:33:40 PM: Using Swift version 5.2

1:33:40 PM: Started restoring cached node modules

1:33:40 PM: Finished restoring cached node modules

1:33:40 PM: Started restoring cached yarn cache

1:33:40 PM: Finished restoring cached yarn cache

1:33:40 PM: Installing NPM modules using Yarn version 1.17.0

1:33:41 PM: yarn install v1.17.0

1:33:41 PM: warning package.json: No license field

1:33:41 PM: warning polulle@1.0.0: No license field

1:33:41 PM: [1/4] Resolving packages...

1:33:41 PM: [2/4] Fetching packages...

1:33:50 PM: info fsevents@1.2.12: The platform "linux" is incompatible with this module.

1:33:50 PM: info "fsevents@1.2.12" is an optional dependency and failed compatibility check. Excluding it from installation.

1:33:50 PM: [3/4] Linking dependencies...

1:33:54 PM: [4/4] Building fresh packages...

1:33:55 PM: success Saved lockfile.

1:33:55 PM: Done in 14.47s.

1:33:55 PM: NPM modules installed using Yarn

1:33:55 PM: warning package.json: No license field

1:33:55 PM: Started restoring cached go cache

1:33:55 PM: Finished restoring cached go cache

1:33:55 PM: go version go1.12 linux/amd64

1:33:55 PM: go version go1.12 linux/amd64

1:33:55 PM: Installing missing commands

1:33:55 PM: Verify run directory

1:33:55 PM: Executing user command: npm run build

1:33:56 PM: > polulle@1.0.0 build /opt/build/repo

1:33:56 PM: > npm run submodule; npm run static && parcel build src/frontend/index.html src/frontend/index.ru.html src/frontend/index.en.html -d dist --no-source-maps

1:33:56 PM: > polulle@1.0.0 submodule /opt/build/repo

1:33:56 PM: > git submodule update --recursive --remote && cd src/shared && git log -n 2 && cd .. && cd ..

1:33:56 PM: Host key verification failed.

1:33:56 PM: fatal: Could not read from remote repository.

1:33:56 PM: Please make sure you have the correct access rights

1:33:56 PM: and the repository exists.

1:33:56 PM: Unable to fetch in submodule path 'src/shared'

1:33:56 PM: npm

1:33:56 PM: ERR!

1:33:56 PM: code ELIFECYCLE

1:33:56 PM: npm

1:33:56 PM: ERR! errno 1

1:33:56 PM: npm

1:33:56 PM: ERR! polulle@1.0.0 submodule: `git submodule update --recursive --remote && cd src/shared && git log -n 2 && cd .. && cd ..`

1:33:56 PM: npm

1:33:56 PM: ERR! Exit status 1

1:33:56 PM: npm

1:33:56 PM: ERR!

1:33:56 PM: npm ERR!

1:33:56 PM: Failed at the polulle@1.0.0 submodule script.

1:33:56 PM: npm

1:33:56 PM: ERR! This is probably not a problem with npm. There is likely additional logging output above.

1:33:56 PM: npm ERR! A complete log of this run can be found in:

1:33:56 PM: npm

1:33:56 PM: ERR! /opt/buildhome/.npm/_logs/2020-05-20T10_33_56_732Z-debug.log

1:33:57 PM: > polulle@1.0.0 static /opt/build/repo

Hey @r0x0r,
We’ve seen a few of these git submodule things recently. Was this working for you before and it broke, or were you trying to get it working for the first time?

It worked once for me as in pulling the submodule for the first time after the initial configuration. After that it has failed to update the submodules.
I have tried to set up a manual pull (via setting ssh agent and configuring ssh keys), but eventually gave up and got rid of submodules for now. Hopefully this will get resolved soon.

Hey @r0x0r,
Thanks for your patience on this. I’m not sure how you’re loading your submodule or whether it’s public, but if it is public, I believe this is the fix:

In case your submodule is public , you can change the URI schema for it to start with https:// and it will should work without further adjustment (so, in .gitmodules , change git@github.com… to https://github.com/… )

If it’s private, things get a bit more complicated. We have an in-depth support guide with some suggestions:

Can you take a look and let us know if any of these fix things for you?

It is a private repo.
Netlify is able to access the submodule repo, but fails to do subsequent updates. Git head fro the submodule is stuck at the first pull.
I am building from Gitlab, generated SSH keys should be good for all the projects.

Hi @r0x0r. It looks like the submodule clone happens later in the build process and I don’t believe the build system will still have access to the deploy keys at that point. The submodule needs to be a Git submodule of the original repo to have our build system automatically clone the submodule.

Would you please confirm if the other repo is a Git submodule of the site’s repo or not?