[Common Issue] Troubleshooting your Netlify Large Media Configuration

Hi, @tyirenkyi. We have sent you an email and we’ll communicate with you through that email to continue resolving this issue.

Also, please let us know with a reply here if you don’t receive a message from us.

Hi, @tyirenkyi, I also see the PM from you with the git lfs output. From that logging it appears the upload is succeeding to me.

I only see a single MP4 file tracked by LFS but this file isn’t in the deploy anywhere. Is the LFS tracked file being copied to the publish directory (which is dist in this case)?

The file must be published for it to be accessible as part of the published deploy and also for it to appear in the Large Media tab under the site settings.

EDIT - Disregard the below - I fixed it by disabling asset optimisation. Images are now loading correctly. Turns out it does say this in the guide, but I missed it. Perhaps there should be a warning in the admin UI if you try and enable optimisation with Large Media enabled?

Regardless, it’s working now - so hopefully this will be a useful post to someone else!



Hi,

I’m having some issues with Large Media. I’ve successfully enabled it, and can see the images have been (apparently) uploaded to Netlify (they are all listed in the Large Media tab at least).

However, most of the images on the site now don’t load. Instead, the images attempt to load from a cloudfront URL, but that is returning malformed images (in fact, look at the contents of the file served by cloudfront is actually the Git LFS pointer - example: https://d33wubrfki0l68.cloudfront.net/a6fcc4e173833f3885a2622cec01356f8def70c3/a3e87/uploads/20170708_180324.jpg). Weirdly, some images are still loading from their original path, not cloudfront.

Output from netlify lm:info is all good

  ✔ Checking Git version [2.17.1]
  ✔ Checking Git LFS version [2.10.0]
  ✔ Checking Git LFS filters
  ✔ Checking Netlify's Git Credentials version [0.1.8]

There is a file called .lfsconfig, and shows a URL with the correct app id. My local repo contains the binary images as expected, and when I look at the repo on GitHub, I can see the Git LFS pointers. All the files I expect are listed in the Netlify Large Media page.

Doing a git lfs push --all origin doesn’t make a difference.

I can PM you more details (site URL, GitHub repo URL and output from the git LFS push with debug enabled) if needed.

Many thanks,
Stephen

Hi, @TeknoVenus. Welcome to the Netlify community site. :slight_smile:

Thank you for posting the solution you discovered here.

I also want to confirm the feature request for the warning. I think there could/should be two checks:

  • one check in the Netlify CLI to see if asset optimization is enabled when netlify lm:setup is run and to fail to install if it is enabled (forcing the user to disable asset optimization to proceed)
  • a check in the Netlify web UI (and possibly API) when trying to enable asset optimization to check for a Netlify Large Media (NLM) configuration for the site and refuse to enable if NLM is installed

Do you agree that both would be helpful?

For those you using BitBucket I got this reply from them while trying to debug my netlify large media set up:

“Bitbucket Cloud’s UI doesn’t have support for third-party LFS providers at this point, as we don’t really fetch the pointers from the third-party provider, the UI expects that the repository uses the Bitbucket LFS storage itself.”

tl;dr Netlify Large Media does not work with BitBucket at the time of this post.

1 Like

hey @jcasserly - thanks for looking into this and letting us know. That’s frustrating, but hopefully at some point BitBucket will allow LFS to work with their systems.

maybe a warning could be added to the documentation as well? If I remember correctly there are explicit instructions/mentions for GitLab and GitHub.

Hi, @jcasserly, I wanted to clarify what is and is not working. The following are my results when testing:

  • Large Media does work with a repo hosted at Bitbucket
  • the Bitbucket UI does not work with Large Media (Git LFS hosting) at Netlify

So, you can use Large Media with a Bitbucket repo, but you won’t be able to see or edit those files in the Bitbucket web UI. However, if you clone the Bitbucket repo, the clone will work correctly as will Large Media - only the Bitbucket web UI won’t work and only for the specific files tracked by Git LFS.

Are you seeing the same thing? If not, we’d be happy to troubleshoot.

I follow up until, “the clone will work correctly as will Large Media.”

So there is way to have Netlify Large Media work with BitBucket?

Currently, BitBucket recognizes my git lfs files but cannot load the pointers (spinning wheel of death), and Netlify Large Media shows ‘No assets found.’

I agree and I’m saying the same thing (I’m pretty sure) but I’m likely not being clear.

I do think we agree that the Bitbucket web user interface doesn’t show the Git LFS files.

Bitbucket does host the repo correctly though. If you clone the Bitbucket repo when the Git LFS service is Large Media and not Bitbucket, that clone works correctly. Just as it would if Bitbucket was the Git LFS service and not Large Media.

I’m just fleshing out the details that the thing not working is the web UI for those files in Bitbucket. The rest of the infrastructure is working. It might be helpful to ask Bitbucket for a feature request to support third-party Git LFS services in their UI.

Hi,

Sorry for the late reply. Yes, I agree both of those items would be useful, as they’d prevent the user from ending up in a situation where they have NLM enabled with asset optimisation enabled, as I did.

Thanks.

Thanks for the input on the feature request, @TeknoVenus. There are now three feature requests in place for various aspects of this:

  • a public feature request for the Netlify CLI to check before enabling here: https://github.com/netlify/cli/issues/759
  • an internal only feature request for our web UI to prevent enabling asset optimization for Large Media sites (this is not a new feature request)
  • an internal only feature request for our API to check both for asset optimization when someone attempts to enable Large Media and vice versa (a check for Large Media when enabling asset optimization)

If/when one or more of the feature requests becomes a reality, we’ll let you know by posting an update about it here. Thanks again for your suggestions.

I believe that I followed all of the steps correctly for one of my projects, but there are still “No Assets” listed in the Large Media tab on Netlify.

I’ve verified the following:

  1. My .lfsconfig points to netlify
  2. git lfs track contains the correct files
  3. git lfs ls-files lists the correct files and their pointer hashes
  4. The files are not in my .gitignore
  5. My .gitattributes is correct
  6. The pointers are in github

One file is a large video (~220 MBs) but I also uploaded a test file that’s only a few bytes. Any advice?

Hi, @sklarsa, there two possible issues that are most common once you get to this point in troubleshooting:

  • the deployed files are pointers
  • the deployed files are not listed in Large Media

If the files are pointers, the solution is to force the upload of the binary files to Large Media using the following:

git lfs push --all origin

However, I think the issue you are experiencing is that the files themselves are missing. If this is the case, then please double check that the files to be deployed are being copied to the publish directory for this site during the deploy.

It looks like the publish directory is dist. Are you sure that these files have been moved to that directory (or a subdirectory of it) before the deploy has been completed?

If not, please make certain your build puts a copy of those files there. (I don’t see any video file extensions in for any files in the deploy. So, I do think they are not being copied to the publish directory.) If they are being copied there, where are the missing files being copied/moved?

If there are questions about any of this, we’re happy to answer.

Thanks for the help! The files were uploaded to LFS but the directory was not in dist. Everything looks good now!

Correct - when I clone the repo to my local machine I see that Bitbucket has actually set up the git lfs pointer files.

However, I’m still not seeing these files in the Netlify Dashboard Large Media section.

@sklarsa, thank you for sharing the solution! It really does help for other people searching here who are experiencing a similar issue. We appreciate you taking the time to follow-up here and I’m sure they do as well.

@jcasserly, we’re reaching out to you via email to troubleshoot this further. I may need to get debugging git output to proceed. I’m hesitant to share that logging publicly (which is probably me being overcautious but I’m playing it safe to be sure).

If you don’t see an email from us about this issue, @jcasserly, please reply here to let us know.

This is the first place I found that told me I need to run these commands. In the Netlify documentation if I run the commands as requested here. I run into this error, search google and find this post. Probably would be good to mention that you have to first run these commands in that section of the documentation.

Hi, @haberdashpi, and welcome to the Netlify community site. :smiley:

Thank you for this input about changing documentation. You are not the first person to mention this and each additional mention is helpful as it makes the scope of impact more clear.

I’ll certainly discuss making this change with our docs team. Thanks again for bringing attention to this.