How to setup pip cache save/restore on a node project?

slorber · October 25, 2019, 11:09am

Hi,

I have a Gatsby/node project, for which I must run aws s3 sync (python/pip/aws cli)

My build script always have to start with “pip install awscli”, which take a few seconds to complete on every single build.

Is there a way to install that awscli once for all and keep it in the cache?

I’ve noticed some others, using Hugo, had pip cache save/restore in their logs:
5:44:09 PM: Started saving pip cache
5:44:09 PM: Finished saving pip cache
Like in this thread: Build failure with hugo - setting hugo version

Is it possible to enable such pip caching on my node site?

Currently my site logs that the following get cached: node version, node modules, yarn cache, go. I don’t need go caching, but I need pip caching and have no idea how to enable this caching so that next builds are faster.

slorber · October 25, 2019, 11:38am

Ok I’ve found out I could put a requirements.txt file at the root of the site and it’s used to install pip deps

fool · October 28, 2019, 9:34pm

Heya @slorber sorry to be slow to see this; our office is pretty much closed over the weekend.

I’m not 100% certain that our python package caching works (I can see we’re trying:

github.com

netlify/build-image/blob/xenial/run-build-functions.sh#L591


      
                echo "Leiningen dependencies installed"
              else
                echo "Error during Leiningen install"
                exit 1
              fi
              echo "$(shasum project.clj)-$JAVA_VERSION" > $NETLIFY_CACHE_DIR/project-clj-sha
            else
              echo "Leiningen dependencies found in cache"
            fi
          fi
          
          
# Boot
          if [ -f build.boot ]
          then
            restore_home_cache ".m2" "maven dependencies"
            restore_home_cache ".boot" "boot dependencies"
            if install_deps build.boot $JAVA_VERSION $NETLIFY_CACHE_DIR/project-boot-sha
            then
              echo "Installing Boot dependencies"
              if boot pom jar install
              then

but have never tested myself)…but hopefully you find that it did ?

slorber · October 30, 2019, 9:23am

Yeah it did. It said “found cached version” or something in the logs in next builds. Actually reduced the pip dependencies install from 20s to 10s (as I need only 1) so it was not a game changer