Prisma Client Binary Not Copied for Netlify Lambda Functions (For Graphql Server)

Bug description

I’m trying to deploy a GraphQL server using Netlify Functions; my app works in development, but doesn’t work in production because the necessary Prisma binary is not copied over post build.

This has been mentioned in another issue (here).

The binary apparently gets copied over using the netlify build command, but not with the specialized netlify-lambda build command.

The current Prisma docs offers an example that deploys the Prisma Client to Netlify Functions using REST API endpoints. For that particular use-case, the netlify functions:build command is entirely sufficient (see here). However, in order to build a GraphQL server, only the netlify-lambda build command will work. See here for a step-by-step instruction on how to set up a GraphQL server using Netlify Functions.

Using the netlify-lambda commands, the GraphQL server gets build on Netlify, but Prisma Client’s rhel-openssl-1.0.x binary never gets copied into the functions folder, producing the following log at /graphql.js:

Invalid `prisma.user.findOne()` invocation:

Query engine binary for current platform "rhel-openssl-1.0.x" could not be found.
This probably happens, because you built Prisma Client on a different platform.
(Prisma Client looked in "/query-engine-rhel-openssl-1.0.x")

You already added the platforms "native", "rhel-openssl-1.0.x" to the "generator" block
in the "schema.prisma" file as described in https://pris.ly/d/client-generator,
but something went wrong. That's suboptimal.

Attempts to manually fix it

Since I know (or think I know) what the issue is (i.e. a missing rhel-openssl-1.0.x binary file), I tried to copy it post-installation into the functions destination folder. At first I tried using a custom webpack plugin, which I was able to pass into the build by using netlify-lambda build functions -c ./webpack.config.js (documentation here):

webpack.config.js

const CopyPlugin = require('copy-webpack-plugin')

module.exports = {
	plugins: [
		new CopyPlugin([
			{
				from: './node_modules/@prisma/client/runtime/query-engine-rhel-openssl-1.0.x'
			}
		])
	]
}

However, the issue with that is that Webpack just packs the binary into the build-functions folder. However, then zip-it-and-ship-it just takes the graphql.js function out of the build-functions folder — and not the binary — and places it into a folder like this: /tmp/zisi-5e8687894a4e660007b0e842. Thus, the binary gets lost in production.

Next, I tried salvaging the binary by using a custom-built plugin.

prisma-client-plugin.js

const fs = require('fs')

module.exports = {
	onPostBuild: api => {
		console.log('Copying over `rhel-openssl-1.0.x` binary engine...')
		const src = `node_modules/@prisma/client/runtime/query-engine-rhel-openssl-1.0.x`
		const dest = `${api.constants.FUNCTIONS_DIST}/query-engine-rhel-openssl-1.0.x`
		fs.copyFile(src, dest, err => console.log(err))
	}
}

Sure enough, that got the binary actually copied into that temporary folder! Still, however, nothing happened. Binary was still missing in production.

So I wondered if the binary needs to be zipped in order for it to make it into the live site. So I did that:

const fs = require('fs')
const zip = new require('node-zip')()

module.exports = {
	onPostBuild: api => {
		console.log('Copying over `rhel-openssl-1.0.x` binary engine...')
		const src = `node_modules/@prisma/client/runtime/query-engine-rhel-openssl-1.0.x`
		const dest = `${api.constants.FUNCTIONS_DIST}/query-engine-rhel-openssl-1.0.x`
		fs.copyFile(src, dest, err => console.log(err))

		console.log('Zipping binary engine...')
		zip.file(`query-engine-rhel-openssl-1.0.x`, fs.readFileSync(dest))
		const data = zip.generate({ base64: false, compression: 'DEFLATE' })
		fs.writeFileSync(`${api.constants.FUNCTIONS_DIST}/query.zip`, data, 'binary')

		console.log('Removing unzipped binary engine...')
		fs.unlinkSync(dest)
	}
}

The zipping works, but now there’s a query.js function file :joy:

So as before, Prisma Client cannot access the rhel-openssl-1.0.x binary engine.

I have spent days trying, to no avail. Any help would be greatly appreciated!

netlify.toml

[build]
publish = "build/"
# Needed to take all netlify-lambda commands out of the package.json file and place them directly into the netlify.toml file for production, since during development Netlify attempts to run scripts that contain `netlify-lambda` functions! See: https://github.com/netlify/cli/issues/723#issuecomment-605511147
command = "cd functions && yarn && cd .. && yarn && npx prisma generate && npx netlify-lambda build functions && react-scripts build"
# During production, `functions` folder cannot be the same as destination when using `netlify-lambda`, so you need to create another destination.
functions = "build-functions/"

[build.environment]
# Fix to ensure the Prisma binary is packaged with the lambda function; [note from Martin: this seems unnecessary when using the netlify-lambda commands]
ZISI_VERSION = "0.4.0-9"

[[plugins]]
package = "./prisma-client-plugin"

[context.develop]
# `netlify dev` in local development won't work unless destination is the same as actual `functions` folder.
functions = "functions/"

How to reproduce

For reproduction, I created a dummy repo that contains all the crucial files. Feel free to clone and connect to your own Postgres db and deploy via Netlify Functions using this guide.


DEMO REPO


1 Like

I’m able to reproduce the problem.

It seems like a change in the build image caused the ZISI_VERSION to be ignored.

Here’s the build log that confirms this hypothesis:

12:16:47 PM: Installing Zip-it-and-ship-it 0.4.0-9
12:16:49 PM: zip-it-and-ship-it version: 0.3.1
12:16:49 PM: Started restoring cached go cache
12:16:49 PM: Finished restoring cached go cache
12:16:49 PM: unset GOOS;
12:16:49 PM: unset GOARCH;
12:16:49 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.12.linux.amd64';
12:16:49 PM: export PATH="/opt/buildhome/.gimme/versions/go1.12.linux.amd64/bin:${PATH}";
12:16:49 PM: go version >&2;
12:16:49 PM: export GIMME_ENV='/opt/buildhome/.gimme/env/go1.12.linux.amd64.env';
12:16:49 PM: go version go1.12 linux/amd64
12:16:49 PM: Installing missing commands
12:16:49 PM: Verify run directory
12:16:49 PM: Executing user command: zip-it-and-ship-it --version && npm run build
12:16:50 PM: 0.3.1
12:16:50 PM: > deployment-example-netlify@1.0.0 build /opt/build/repo
12:16:50 PM: > prisma generate
12:16:51 PM: ✔ Generated Prisma Client to ./node_modules/@prisma/client in 145ms
12:16:51 PM: You can now start using Prisma Client in your code:
12:16:51 PM: ```
12:16:51 PM: import { PrismaClient } from '@prisma/client'
12:16:51 PM: // or const { PrismaClient } = require('@prisma/client')
12:16:51 PM: const prisma = new PrismaClient()
12:16:51 PM: ```
12:16:51 PM: Explore the full API: http://pris.ly/d/client
12:16:51 PM: Function Dir: /opt/build/repo/functions
12:16:51 PM: TempDir: /tmp/zisi-5e8b017acdc08b0006942653
12:16:52 PM: Prepping functions with zip-it-and-ship-it 0.3.1
12:16:57 PM: [ { path: '/tmp/zisi-5e8b017acdc08b0006942653/createUser.zip',
12:16:57 PM:     runtime: 'js' },
12:16:57 PM:   { path: '/tmp/zisi-5e8b017acdc08b0006942653/posts.zip',
12:16:57 PM:     runtime: 'js' },
12:16:57 PM:   { path: '/tmp/zisi-5e8b017acdc08b0006942653/seed.zip',
12:16:57 PM:     runtime: 'js' },
12:16:57 PM:   { path: '/tmp/zisi-5e8b017acdc08b0006942653/status.zip',
12:16:57 PM:     runtime: 'js' },
12:16:57 PM:   { path: '/tmp/zisi-5e8b017acdc08b0006942653/users.zip',
12:16:57 PM:     runtime: 'js' } ]
12:16:57 PM: Prepping functions complete

Thanks for the detailed bug report! I’m working on reproducing the issue so that I can get it resolved. The wrong version of zip-it-and-ship-it being used definitely seems like a possible cause of the problem you’re seeing. I’ve filed a bug and I’ll update here once I have more information or get it resolved.

1 Like

Thanks,@benaiah. The issue however is that I’m using netlify-lambda due to the graphql.js endpoint being a GraphQL server. ZISI isn’t invoked for the netlify-lambda command, is my understanding.

So I’m in the process of creating a custom build plugin that uses ZISI. May I add you to the [secret] project for which I’m trying it? That might get you closer to solving it.

@heymartinadams ok, so you’re saying the error you were seeing is likely not the same thing as the incorrect ZISI_VERSION issue that @2color mentioned? Sorry that I misunderstood the post. I’ll try to track down an expert on netlify-lambda who can help us understand why you’re seeing that behavior when ZISI isn’t involved.

Thank you. After 2 weeks of working on this full-time, I have given up. Unless someone can come in and totally fix this, I will have to look elsewhere for deployment and hosting.

:wave: @heymartinadams - we definitely want to keep working on figuring this out, as I am sure you are not the only customer who is hitting this roadblock. In the middle of covid (we are supporting a lot of coronavirus sites) we have been a bit slower than some folks have been hoping to get things fixed.

@heymartinadams, the site we were troubleshooting no longer exists. As I mentioned in an email, I think it is possible that the binary is being copied to the function but there is a version mismatch.

This was the error mentioned:

Invalid `prisma.user.findOne()` invocation:

Query engine binary for current platform "rhel-openssl-1.0.x" could not be found.
This probably happens, because you built Prisma Client on a different platform.
(Prisma Client looked in "/query-engine-rhel-openssl-1.0.x")

You already added the platforms "native", "rhel-openssl-1.0.x" to the "generator" block
in the "schema.prisma" file as described in https://pris.ly/d/client-generator,
but something went wrong. That's suboptimal.

This might be because the build image is Ubuntu Linux and the Function is running in an AWS Lambda instance running Amazon Linux. The Amazon Linux Lambda probably requires the RHEL version of binary but the wrong version is copied.

We are not experts with Prisma itself and so providing technical support for it will be slower. This will likely require assistance from the Prisma team as well.

Do you want to keep troubleshooting this? If so, do you know how to get Prisma to install the “rhel-openssl-1.0.x” binary on a platform where that is not the correct version? The answer to that second question is likely going to be key in resolving this issue but this isn’t a question I know the answer to.

Hi @luke, @perry. Thank you kindly for your replies. I have, however, thrown in the towel; had been at this single issue for over two weeks, made custom build plugins, reached out to Netlify support several times, as well as to Prisma, and nobody has been able to fix the bug.

At this point, it’s financially unsustainable for me to spend more time trying to figure out a solution since I have other pain points to figure out to get the app to production. I’m sorry to say, I found another provider and was able to get a build working without a hitch; have a working GraphQL server up and running that uses Prisma.

Still happy to help out here and there to help you find a fix. To answer your question, @luke, you specify which version you wish Prisma to use in the schema.prisma file. https://github.com/prisma/specs/blob/master/binaries/Readme.md#how-to-fetch-binaries-1

Will keep replying to this thread to see if we can get this to work for others.

Hi, @heymartinadams, don’t see any proof of a bug here. All the data I’ve reviewed points to a configuration issue and not a bug in my opinion.

When you did get this working, how was it done? Did you create a serverless function or host this in AWS Lambda? Or is the new solution a stand-alone application instead of a serverless function?

Again, I think this is an issue bundling the requirements for serverless deployment (and not a bug) so figuring out how that was done (if it was done) will be key here.

As mentioned in the first post, I created a serverless function on Netlify. For the build instructions I used netlify-lambda.

I got everything to work, both locally and in production, except that the Prisma Client binary engine isn’t copied over for production.

Hi, @heymartinadams, you wrote this:

I’m sorry to say, I found another provider and was able to get a build working without a hitch; have a working GraphQL server up and running that uses Prisma.

I was asking about that solution above. How did you get the Prisma client installed in the AWS Lambda or serverless function? Or are you not using AWS Lambda (or a serverless function) for the Prisma client now?

I think the issue was getting the correct binary installed for the AWS Lambda instance. I suspect the wrong binary was being copied so, if you found a solution to do this with another serverless provider, information about how that was done may point to the solution here.

They have a “postinstall” script (“prisma generate”) that automatically copies the binary over during build. I believe I tried the postinstall script on Netlify with no success.

@heymartinadams, I’m glad you found a solution for this problem that works for you, even if that means you’re using a different provider. I’m sorry we weren’t able to be more helpful at this time. We’ll keep thinking about this and update here if we come up with a solution that might change things for you.

1 Like

Thanks, and no worries! Hope you’ll figure out a solution (if indeed the issue wasn’t with my own build instructions or code) since I’m sure building a GraphQL server using Prisma could become a popular approach in the years to come.

1 Like