The problem
When I invoke my lambda function, I get this error:
Function invocation failed: Error: ENOENT: no such file or directory, open <file_name>
file_name is a 130mb .dat file that lives inside a node_modules dependency.
I’m using netlify-lambda to build the functions directory.
My understanding is that any dependencies in the root package.json file will be bundled with the lambda functions. This error is occurring both locally and when deployed. I thought perhaps the file_name exceeded some maximum size threshold, but if that were the case I’d expect it to fail on deploy.
Thanks to anyone who can point me in the right direction!
Yeah, if the file is too big, I would also expect it to fail at deploy time rather than runtime. I’d guess that the file is not actually bundled/shipped. Do you actually import it into your code, vs just listing it in package.json? I’m not sure how that would look and I’m not a javascript-er so not certain, but that is advice I’ve heard given to other folks in the past trying to include static content like that in functions…
@leveloneluke that may not work. a zipped function has to be smaller than 50MB. it’s possible that zipped the file will actually shrink significantly, but it may not, depending on the other modules bundled in to your function. Also note that you can only bundle text based files like js, html, css, svg, etc. You won’t be able to bundle a binary. You have to zip it in with your function. Is your repo public? Can you provide a link to it?
The 130mb dependency compresses down to <30mb when I compress it using the default Mac OS compressor. The lambda directory itself is negligible. Is there reason to think that the file size would be significantly greater via Netlify’s Zip&Ship tool?
Sorry, yes I compressed it using zip. I’m not super familiar with compression algorithms or archive formats, so I was uncertain about whether other zip tools might achieve different compressed file sizes.
Hi, zip it and ship it compresses your files including all dependencies that you import in your js file. how are you importing the custom dat file? just copying it in to the node_modules isn’t enough for zip it and ship it to bundle it in if it’s not being imported. Also have you tried deploying the zip archive you made yourself to see if that works?
The .dat file is part of this dependency, which lives inside the root level node_modules.
I’m importing this dependency at the top of the file where I define my lambda function:
import PokerEvaluator from 'poker-evaluator';
I’ve also tried importing it using the require syntax.
Looking back on your comments, it looks like this is the key line I overlooked:
Also note that you can only bundle text based files like js, html, css, svg, etc. You won’t be able to bundle a binary. You have to zip it in with your function.
Can you clarify for me what you mean when you suggest that I zip it in with my function?
Hi @leveloneluke, if you pre-zip your function and put that zip file in your function folder, our buildbot will deploy it as long as it has the correct file structure (as mentioned in the zip-it-ship-it repo). Let me know if that works for you.
I linked the repo only as a point of reference on the folder structure your zipped function should have. You shouldn’t need to install the zip-it-and-ship-it package as that is already built into our buildbot.
Just zip your function based what is mentioned in the repo (don’t try to install the package). Put the zip file in your functions folder (which you should have configured in your netlify.toml file or in your site’s configuration page in the functions tab). Our buildbot should find and deploy the zip file.
?