Plugging into the Build pipeline with manual API deploys

Right up at the top of the https://docs.netlify.com/site-deploys/create-deploys/ page, I see:

When you create a deploy manually without continuous deployment, Netlify does not run a build command.

That’s spelled out plainly enough, but I’d like to know if there’s any way around that or if there’s a possibility that this is being changed in the future. And since I don’t know all of the inner workings of Netlify, could you tell me the reasoning behind that design choice?

From my limited, outside perspective, my server telling Netlify that something changed and that the site should be rebuilt using files that I provide seems very similar to Github telling Netlify that something changed and that the site should be rebuilt using use the files on Github.

Those are my main questions. If you want some more background on what I want to build and what I’ve tried so far, I’ve included it below. But you could also just stop here!

What I’m trying to build:

I’d love to build a drag and drop site builder where the site’s React markup would be represented as a JSON object - type, props, ids, etc. Ideally, this would be something I could use to build sites for other people and manage their sites for them.

Once I hit ‘Save’ on the builder, I would like to:

  1. Create a site via the Netlify API

  2. Upload some files to the site, including the JSON file of markup, a generated netlify.toml file with the appropriate build and plugin declarations, and a local plugin that I write that will:

  3. In the onPreBuild step, transform the JSON into React files and inject them into a gatsby/react-static/other static site generator template

  4. Run the appropriate build command as normal on Netlify to build and deploy the site

What I’ve tried:

  1. Just uploading the files via the API, hoping it would trigger a build. It did not, which is in line with the docs note that manual deploys do not run a build command

  2. Working around the ‘we don’t build on a manual deploy’ by trying to add a build hook and POSTing to the build hook URL after uploading the files for the deploy. The build hook was created for my site, but it does not show up in the Netlify UI nor does it run a build when I POST to it via Postman (with a 200 status)

Possible workarounds on my end:

  1. Create a git repository for each site with the JSON markup and add continuous deployment for each repository. This feels like an unnecessary step in between my server and Netlify. My server would mostly be wrangling Github and using Github as a database as a way to get code on Netlify instead of working with Netlify to get code on Netlify.

  2. Generate the full site dist folder on my server and then upload the entire thing to Netlify. This seems inefficient, since Netlify is full of super smart people who have put a ton of work into a build pipeline, and my build pipeline would be a piece of garbage in comparison. And if the entire thing needs to be built beforehand, I’m throwing a ton of bloat over the web when I could just send a small payload with all of the information required to generate the site.

  3. I’m open to other suggestions, if you’ve got any!


Thanks for reading this through! Again, my main questions:

  1. What is the reasoning behind not allowing manual deploys to run a build step?
  2. Is there any way for me to get manual deploys to use the build pipeline?

:v: Matthew

I’ll keep my answer short and sweet however I don’t think I’m missing a trick here?

The highlighted bit within our docs which you’ve pointed out is in reference (mainly) to drag and drop deploys where WYSIWYG.

Hey, @Pieparker - thanks for getting back to me.

I made a simple github repo that I hope demonstrates my problem: https://github.com/mknudsen01/netlify-build-from-api

I’ve tried using the createSiteDeploy API endpoint that you linked to. I’ve also tried using the convenience deploy method on the Netlify node client.

I’m able to get the deploy working. However, the build step does not seem to run when I deploy in this manner. It will deploy any files that I have that are already built, but it will not run my build commands or prebuild plugin handler.

On https://docs.netlify.com/configure-builds/stop-or-activate-builds/#stop-builds, I see a tip that states:

You can still update your site by running a build locally via the CLI and then creating a deploy manually via the CLI or the API.

Again, I’m trying to avoid doing the local build before deploying, as I would like to plug into Netlify’s build pipeline.

Also, on https://docs.netlify.com/configure-builds/get-started/#basic-build-settings, the build command settings are nested under Continuous Deployment: Settings > Build & deploy > Continuous Deployment > Build settings, so it seems as though continuous deployment needs to be set up to run any sort of build.

So, I’m more convinced now that it’s just something that isn’t supported unless I hook up with a git client for continuous deployment.

If I’m mistaken and manual deploys are possible without having to first completely build locally, could you please point me to a code example on how to get it set up?

Otherwise, I’d appreciate learning some of the reasoning as to why it’s not technically feasible/desirable on your end to allow for builds on manual deploys.

Thanks!

Hi @matchew and sorry to be slow to get back to you!

As you’ve found, most of our deploy-related endpoints aren’t “trigger a build” - they are instead “send files” or “create deploy to which you’ll send files”.

If you already have the repo linked (if not, this guide is what you need: [Support Guide] Linking a repository via API), then you should instead be able to trigger a deploy using this endpoint:

HTTP POST https://api.netlify.com/api/v1/sites/YOUR_SITE_ID/builds

This is documented here: https://open-api.netlify.com/#operation/createSiteBuild and you’ll probably be interested in this one too: https://open-api.netlify.com/#operation/listSiteBuilds

If you don’t want to link a repo to us, there is no way to use our CI. We didn’t design our system to take random inputs for builds, just to clone git repos. We figure if you have strong opinions about doing it a different way, you’ll use your own CI that you can configure to match those :slight_smile: