[Functions] Issues parsing images from multipart/form-data

I opened an issue on the netlify-lambda repo here: https://github.com/netlify/netlify-lambda/issues/180

But the gist is that I’m trying to parse an image upload from a function/lambda event and it doesn’t appear to be working. I’ve tried using busboy and also aws-lambda-multipart-parser. I can get a resulting buffer from both, but it seems to be malformed each time. I’ve tried checking the mime type with file-type and also trying to load the parsed image buffer into Jimp and get the following error: Could not find MIME for buffer (null). So it seems like, at least from these experiences, that part of the form data is getting malformed/dropped along the way.

I can only put two links in a post so here are two follow up replies with the packages referenced in the OP.

Could you tell us more about your workflow? For instance, if you are using an event-triggered function you wouldn’t have any uploaded file data in the body - it would be a URL to the data. But perhaps you mean you wrote a function that accepts POSTs and you’re having trouble parsing the data directly off the wire?

Yup, it’s the latter! I’m posting directly to an endpoint. The errors I wrote about in the OP occur while running on Netlify/deployed to Lambda. Also when I use Netlify Dev the . parse.on('file', cb) never even occurs.

Here is where the post request occurs:

and the lambda function is here: https://github.com/dmmulroy/partyfy.party/blob/beta/.netlify/functions/partyfy.js

Hi,

I don’t actually think that will work right now. I’ve opened an issue regarding adding support for multi-part requests to Netlify functions, and added a link to this thread so we’ll let you know once we have more info. Sorry about that @dmmulroy!

1 Like

Ahhh, bummer! Thanks for letting me know though and linking back to this issue for future updates!

Man, I wish I had stumbled upon this discussion before spending hours.

Wrote a post in AWS subreddit asking for help before finding this thread, here it is: https://www.reddit.com/r/aws/comments/d23ff8/need_help_with_lambda_formdata_nodemailer/

In my case, PDFs and JPGs were malformed (slight filesize difference, and not openable).

According to the the library I’ve had the most success with (https://github.com/francismeynard/lambda-multipart-parser), a config has to be enabled:

Important Please make sure to enable the “Use Lambda Proxy integration” in API Gateway method Integration request.

If decided not to enable it for some reason, make sure to pass the required Lambda event parameters in Integration Request -> Mapping Templates section, such as body, headers and isBase64Encoded flag.

Sample Lambda and API Gateway implementation with Cloudformation can be found in here.

1 Like

I ended up switching to straight up using API Gateway and Lambda and got https://www.npmjs.com/package/lambda-multipart working. Although my lambda function takes >29 seconds on images >650kb so I’m getting time outs :sweat_smile: so I’m rewriting from Node -> Go

I imagined 650kb to be trivial for node to handle, didn’t think it would’ve required rewriting to Go. Well, I didn’t imagine sending email with attachment to be a hassle too. PS: I’m a newer developer, devs on Twitter always talk about how easy it was to set up Rails back in the day. Sorry for ranting :smiley:

It’s not so much the image itself it’s what I’m doing with it once I parse it :joy: . v. serious business

1 Like

I also spend some hours stuck on this :frowning:

Any updates?

I haven’t heard or seen any action from Netlify on their side for this - as mentioned above I’m using API Gateway + Lambda directly since their doesn’t seem to be support yet for it from netlify.

Edit: To the folks at Netlify: if the code for this open source, I’d be more than happy to help hack on this and try to get a PR up

We don’t have any immediate plans to change the way our functions are implemented around multipart submissions, if that’s what you’re asking?

We will follow up here if things change, but my suggestion would be don’t wait for a config change on our side but find a different flow to use:

  • if you want to use our functions implementation, work within today’s featureset.
  • if you need a different setup, you could use our proxying to your separate deploy of functions in your own AWS (or serverless.com, or whatever service) account.

If there is no plan to support images via multipart submissions straight off the request I’d love to at least see that added to the docs as I’m not the only one who has been caught off guard by this.

good point, @dmmulroy - i’ll chat with docs about this. thanks for pointing that out.

Hey! @dmmulroy! I am also making a netlify function to accept a form with two images, I have converted them into base64 format. They are working properly in my local environment.

But in production I am getting a base64 encoded body, with the event.isBase64Encoded being true. None of that was happeining in local environment. And worst part is the body after decoding is partiall of what is sent.

Can you please help in this.

Hi,

We don’t use the lambda API gateway at Netlify, we run our own proxy. That said, you should be able to BASE64 encode your data and then decode it from within the function. Do you have a simple example repo that shows how the files are being uploaded and how you’re decoding them? You mentioned them being cut off, are you sending them back to the frontend?