I’ve lost my sanity trying to deal with LFS in Netlify and need help. Frequently encountered problems during builds #8 told me to reach out if I was still having issues. I think I might be in that 1%.
I’m building a Gatsby site from GitLab using their LFS implementation for images and fonts. When developing locally I’m able to work normally. My files are pulled from LFS and the file pointers are replaced with the real files. My local production builds work just fine.
When I started using Netlify to deploy my site this week I noticed that webpack was inlining my fonts (as base64 url blobs) into the site, but when decoded those blobs only contained the content of the LFS file pointers. After a lot of troubleshooting and trial and error I discovered that Netlify handles LFS strangely.
GIT_LFS_ENABLED=true is required to fix build cache errors (which I experienced), but that just uses
git lfs clone instead of
git lfs clone is deprecated and LFS is handled internally by
❯ git lfs clone email@example.com:johnrichter/personal-website test-lfs-clone WARNING: 'git lfs clone' is deprecated and will not be updated with new flags from 'git clone' 'git clone' has been updated in upstream Git to have comparable speeds to 'git lfs clone'.
My local git versions are
git version 2.24.0
git-lfs/2.8.0 (GitHub; darwin amd64; go 1.12.7)
I was able to figure out that only the font files (woff2, woff, eot, and ttf) would not clone successfully with LFS.
$ git lfs fetch --all; git lfs checkout; cat /opt/build/repo/.git/lfs/logs/* fetch: 100 object(s) found, done fetch: Fetching all references... batch request: missing protocol: "" error: failed to fetch some objects from '' ... Skipped checkout for "src/assets/fonts/Exo_300.eot", content not local. Use fetch to download. Skipped checkout for "src/assets/fonts/Exo_300.ttf", content not local. Use fetch to download. Skipped checkout for "src/assets/fonts/Exo_300.woff", content not local. Use fetch to download. Skipped checkout for "src/assets/fonts/Exo_300.woff2", content not local. Use fetch to download.
The first issue I’ve found is with Git.
batch request: missing protocol: "" and
error: failed to fetch some objects from '' resolve to issues with git lfs not having SSH support and the dependence that LFS has on the remote being
origin. I was able to add
origin by running
git remote add origin https://gitlab.com/johnrichter/personal-website.git before
gatsby build, but that alone didn’t fix issues since it happened after the initial clone. After adding
git lfs fetch --all && git lfs checkout I was able to finally able to get LFS to load all files correctly.
My final build command was
git remote add origin https://gitlab.com/johnrichter/personal-website.git && git lfs fetch --all && git lfs checkout && gatsby build.
I thought I was home free, but now I’m running into issues with subsequent builds that are saying that
remote origin already exists. This does not happen if I manually build without the cache.
Git LFS enabled Preparing Git Reference merge-requests/15/head Error adding remote: fatal: remote origin already exists. Failing build: Failed to prepare repo
This support question hinted at writing a script that adds origin if it doesn’t exist before the build and then removes
origin after the build to avoid the remote being configured for the next build from the Netlify cache, but its not working. I’ve tried to run this script directly as the build command and as a yarn
preinstall command, but nothing works. Using
preinstall did show me that my build script environment needs to be set carefully. When running the script as a normal build script I tried using
set -ex bash ..., but that didn’t work. Perhaps I need to do that within the script.
At this point I’m contemplating on doing my builds and deployments within GitLab CI via Netlify CLI because I think it’d be easier. I’d rather use Netlify CD to do this bercause of the ease of automatic deploy previews though!
The repo is here. My Netlify site is
jrichter-io.netlify.com. Links to the builds for troubleshooting are in my follow up comment because I’m limited to 6 links for new topics in this forum.
Thanks for reading to the end. I’d really really appreciate help. I’m not sure what to do. Maybe I’ll pull down the build image, read its source, and try to rapidly test solutions locally in Docker.