I have previously developed a node lambda function for generating pdfs using wkhtmltopdf. In order to achieve this feat, I put the latest Centos build of wkhtmltopdf binary into a tar archive then base64 encoded that, then put that string into a json file which is included with the function (so it got packaged up when built).
During execution, the function would decode the string from the json file, write the resulting tar file to the tmp directory, then untar it there. Then using a node wkhtmltopdf library I was able to generate a pdf using a handlebar.js html template and posted data.
It worked and I got pretty pdfs.
netlify site: divshares.netlify.com
function: prospect-report.js → divshares.netlify.com/.netlify/functions/prospect-report
NB: The function can only be called in POST. It requires a form encoded string consisting of a single field data whose value is a json string setup for reasons I can’t remember. Here is a sample submission:
data=%7B%22name%22%3A%22Person+Name%22%2C%22email%22%3A%22person%40example.com%22%2C%22prospects%22%3A%5B%5D%2C%22dataDate%22%3A%229th+March+2019%22%2C%22dummyData%22%3Atrue%2C%22values%22%3A%7B%22sort%22%3A%22company%22%2C%22PE_RATIO%22%3A%5B1%2C15%5D%2C%22PAYOUT_RATIO%22%3A%5B50%2C75%5D%2C%22NET_YIELD%22%3A%5B6%2C22%5D%2C%22FRANKING%22%3A%5Btrue%2Ctrue%2Ctrue%5D%2C%22EXCLUDE_SECTOR%22%3A%5B%5D%2C%22EXCLUDE_INDUSTRY%22%3A%5B%5D%2C%22EXCLUDE_COMPANY%22%3A%5B%22EBG%22%2C%22LYL%22%2C%22MTO%22%2C%22REX%22%2C%22RXP%22%2C%22SDG%22%2C%22SGF%22%2C%22SRG%22%2C%22TRS%22%2C%22VLW%22%2C%22VTG%22%2C%22CVF%22%2C%22CAA%22%2C%22ASH%22%2C%22AFG%22%5D%7D%7D
Recently, the function has stopped working. Initially it had 500 errors, when I investigated the logs it reported an ENOENT error no command found for tar. I wrote up a custom endpoint that reads the path variable and runs ls -1 on every directory it finds:
http://divshares.netlify.com/.netlify/functions/sys-probe
the current result is:
{"results":["node","npm","npx","","","[","alias","arch","awk","base64","basename","bash","bashbug","bashbug-64","bg","ca-legacy","captoinfo","cat","catchsegv","cd","chcon","chgrp","chmod","chown","cksum","clear","comm","command","cp","csplit","cut","date","dd","df","dgawk","dir","dircolors","dirname","du","echo","egrep","env","expand","expr","factor","false","fc","fg","fgrep","fmt","fold","gawk","gencat","getconf","getent","getopts","grep","groups","head","hostid","iconv","id","igawk","info","infocmp","infokey","infotocap","install","jobs","join","ldd","link","ln","locale","localedef","logname","ls","makedb","md5sum","mkdir","mkfifo","mknod","mktemp","mv","nice","nl","nohup","nproc","numfmt","od","p11-kit","paste","pathchk","pgawk","pinky","pldd","pr","printenv","printf","ptx","pwd","read","readlink","realpath","reset","rm","rmdir","rpcgen","runcon","sed","seq","sh","sha1sum","sha224sum","sha256sum","sha384sum","sha512sum","shred","shuf","sleep","sort","sotruss","split","sprof","stat","stdbuf","stty","sum","sync","tabs","tac","tail","tee","test","tic","timeout","toe","touch","tput","tr","true","truncate","trust","tset","tsort","tty","tzselect","umask","unalias","uname","unexpand","uniq","unlink","update-ca-trust","users","vdir","wait","wc","who","whoami","yes","","[","alias","arch","awk","base64","basename","bash","bashbug","bashbug-64","bg","ca-legacy","captoinfo","cat","catchsegv","cd","chcon","chgrp","chmod","chown","cksum","clear","comm","command","cp","csplit","cut","date","dd","df","dgawk","dir","dircolors","dirname","du","echo","egrep","env","expand","expr","factor","false","fc","fg","fgrep","fmt","fold","gawk","gencat","getconf","getent","getopts","grep","groups","head","hostid","iconv","id","igawk","info","infocmp","infokey","infotocap","install","jobs","join","ldd","link","ln","locale","localedef","logname","ls","makedb","md5sum","mkdir","mkfifo","mknod","mktemp","mv","nice","nl","nohup","nproc","numfmt","od","p11-kit","paste","pathchk","pgawk","pinky","pldd","pr","printenv","printf","ptx","pwd","read","readlink","realpath","reset","rm","rmdir","rpcgen","runcon","sed","seq","sh","sha1sum","sha224sum","sha256sum","sha384sum","sha512sum","shred","shuf","sleep","sort","sotruss","split","sprof","stat","stdbuf","stty","sum","sync","tabs","tac","tail","tee","test","tic","timeout","toe","touch","tput","tr","true","truncate","trust","tset","tsort","tty","tzselect","umask","unalias","uname","unexpand","uniq","unlink","update-ca-trust","users","vdir","wait","wc","who","whoami","yes","",""]}
Notably missing is tar, and also zip. I can imagine a reason for why it may have been removed, but I couldn’t find any announcements from either Netlify or Amazon that this had occurred. When I downloaded Amazon Linux 2, it had tar included.
I have tried deploying that tar binary in a similar way to the wkhtmltopdf binary, although it is small enough that I don’t need to compress it, I simply base64 encode it, then dump to decoded version also file in /tmp/. I can’t get any useful output from that either (I get an exit code of 2). Here is my latest log dump from the function:
3:03:49 PM: 2020-03-18T04:03:49.105Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO 1584504229105 -> Invoked
3:03:49 PM: 2020-03-18T04:03:49.105Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO {
name: 'Person Name',
email: 'person@example.com',
prospects: [],
dataDate: '9th March 2019',
dummyData: true,
values: {
sort: 'company',
PE_RATIO: [ 1, 15 ],
PAYOUT_RATIO: [ 50, 75 ],
NET_YIELD: [ 6, 22 ],
FRANKING: [ true, true, true ],
EXCLUDE_SECTOR: [],
EXCLUDE_INDUSTRY: [],
EXCLUDE_COMPANY: []
}
}
3:03:49 PM: 2020-03-18T04:03:49.172Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO /var/task
3:03:49 PM: 2020-03-18T04:03:49.184Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO Running command: chmod +x /tmp/tar
3:03:49 PM: 2020-03-18T04:03:49.204Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO
3:03:49 PM: 2020-03-18T04:03:49.204Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO Running command: tar --version
3:03:49 PM: 2020-03-18T04:03:49.208Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO TAR version tar (GNU tar) 1.26
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
3:03:49 PM: 2020-03-18T04:03:49.209Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO 1584504229208 -> Writing out executable
3:03:49 PM: 2020-03-18T04:03:49.370Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO 1584504229370 -> Expanding TAR
3:03:49 PM: 2020-03-18T04:03:49.371Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO Running command: tar --warning="all" xzvf /tmp/wkhtmltopdf.tar.gz -C /tmp/
3:03:49 PM: 2020-03-18T04:03:49.375Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO TAR output
3:03:49 PM: 2020-03-18T04:03:49.375Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO Running command: ls -1la /tmp/
3:03:49 PM: 2020-03-18T04:03:49.382Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO total 15328
drwx------ 2 sbx_user1051 495 4096 Mar 18 04:03 .
dr-xr-xr-x 17 root root 4096 Feb 26 15:40 ..
-rw-rw-r-- 1 sbx_user1051 495 2052 Mar 18 04:03 logo.svg
-rw-rw-r-- 1 sbx_user1051 495 7544 Mar 18 04:03 randomfile.html
-rwxrwxr-x 1 sbx_user1051 495 341984 Mar 18 04:03 tar
-rw-rw-r-- 1 sbx_user1051 495 15329991 Mar 18 04:03 wkhtmltopdf.tar.gz
3:03:49 PM: 2020-03-18T04:03:49.383Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO 1584504229383 -> Executable created and untarred
3:03:49 PM: 2020-03-18T04:03:49.383Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO [node-wkhtmltopdf] [debug] [command] /tmp/wkhtmltopdf --page-size "A4" --debug-javascript --orientation "Landscape" --footer-right "Page [page] of [toPage]" --footer-center "P/E: 1-15%, PR: 50-75, NY: 6-22%, F: f,p,n" --footer-left "Demo-data for evaluation only" - -
3:03:49 PM: 2020-03-18T04:03:49.386Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO [node-wkhtmltopdf] [debug] /bin/bash: /tmp/wkhtmltopdf: No such file or directory
3:03:49 PM: 2020-03-18T04:03:49.387Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO []
3:03:49 PM: 2020-03-18T04:03:49.387Z 2a3ced8c-d9bb-4f40-9d06-602c4c2eb531 INFO 1584504229387 -> PDF Generated
3:03:49 PM: Duration: 283.56 ms Memory Usage: 269 MB
It’s a bit of a mess, the ls command which lists the /tmp/ directory is performed after the untarring, so it should have the wkhtmltopdf binary, but doesn’t. The tar command also doesn’t output any text, despite using the warning flag. Using the same binary on Amazon Linux 2 with the same decoded wkhtmltopdf.tar.gz ejects the binary fine and the filename appears in the console. Rather than trying to figure out why uploading tar doesn’t work (I can confirm that the encoded version works when decoded again to a file) I would rather understand why tar has gone missing, but any help getting any tar working would be great.
Any help would be much appreciated.