Using Node.js functions from client-side Javascript

Hi everyone,

I have made some basic functions in Node.js that read and write to other files. I need to call these functions on command, such as when a button is pressed. Here is one example of the logic that I need:

<button onClick="readFileWithNodeJS()">Click me</button>

I have a file called test.js with two functions: readFile() and writeFile(param) and then a handler method. I need to call these functions from my client-side javascript.
test.js:
var fs = require(“fs”);

function readFile() {
    fs.readFile("src/textFiles/testFile.txt", function(err, buf) {
        if (buf) {
            console.log(buf.toString());
            return buf.toString();
        }
    });
}

function writeFile(data) {
    fs.writeFile("src/textFiles/testFile.txt", data, function(err) {
        if (err) {
            console.log(err);
        } else {
            console.log("Success");
        }
    });
}

exports.handler = function(event, context, callback) {
    callback(null, { statusCode: 200, body: "Hi" });
}

I have done extensive research over the past week and I’m seeing that I need to make an HTTP request. For me, since I am testing it locally on port 8888, the endpoint url for the functions would be http://localhost:8888/.netlify/functions/test. I am using $.post and sending a request to that url. This is the request I am making:
$.post("/.netlify/functions/test", {},
function(data, status) {
console.log(data);
}, “text”);

I have a few questions about how this all works:

  1. If I have a specific function that I want to use in node.js, does each function have to be in its own file with its own handler method? For example, I have the readFile() function I mentioned earlier. To use that method specifically, would I need to create another file called read.js and create another handler function that would complete the functionality, and then request would be directed to http://localhost:8888/.netlify/functions/read, or is there a way to do this in one file?

  2. If I want to actually send some data with the POST request, how would I access that at the endpoint in test.js? For example, if I am trying to use the writeFile(data) function and I pass {newData: “This should be the new contents of testFile.txt”} as the data in the post request, how would I access that as plain text so that I could just use fs.writeFile as shown in the function above.

I am just diving into this complex world for the first time and would greatly appreciate some assistance. I appreciate anyone who would be willing to help out.

Thanks.

Hello @RobotCrazy. When your lambda function is deployed to AWS, it won’t have access to your repository’s files or src folder. You’ll need to host the file you want to write to on another service to be able to accomplish what you are trying to do.

  1. I would recommend that you each of your lambda functions be separated by ‘task’ to keep things less complicated and able to complete in less than 10 seconds.
  2. You can access your payload you sent in your request within the event object. This is mentioned in our Functions doc.

That said, if you could provide more context on what you are trying to accomplish, we can maybe help point you in the right direction.

I greatly appreciate your response.

This explains the issue I have been having for about a day now. I actually figured out how to at least read and write locally by running netlify dev yesterday. When I actually pushed the files online, they were no longer able to read from the files. Is this difference in behavior expected?

As for hosting the files on a separate platform, where would you recommend I do this?

Are you able to elaborate on this? I’m not sure what you mean by separating things into tasks.

Thank you for this. I took a look at this again. Rereading that page helped me figure it out.
I did want to ask something about the POST data, however. When testing locally and sending data with the POST request, when I examined the data at the endpoint, the data was encoded in base64, so I had to decode the data using Buffers before using it. When I pushed the data online, all the data at the endpoint was in raw text. I did not do any encoding myself, so I’m assuming this is Netlify’s server doing this. Is this expected or could this be something with my machine?

Once again, thank you very much. I really do appreciate the assistance.

Hi @RobotCrazy, how are you testing locally? Are you using netlify dev? It would really depend on that. The behavior you see when deploying your function is the expected behavior.

When testing locally, I run netlify dev. The behavior of functions seemed to differ between when I ran netlify dev and when I pushed the code online to Netlify and ran the site online.

Hi @RobotCrazy can you share the request headers and payload that you are sending to the function? You should be able to take a screen shot in your browsers dev tools. Also what is the data that you are sending? is it a file? The more details you provide, the easier it will be to help you. If the site is live, providing us a complete reproduction case will be super helpful.

Sorry for the somewhat incomplete information I’ve provided about my question. Here are the full details of my question:

Objective: I’m trying to build a small scale content system in which the content for a website will be stored in JSON files. The website will pull its data from these files and there will be an interface built into the site to edit the content of the website. Based on the new content, the JSON files will be updated. (So far, I’ve only been working with text files to keep things simple. The end goal is for everything to be JSON).

I have two main JS files: test.js which has the lambda functions, and testScript.js, which has the client-side Javascript containing the code to send requests based on what the user selects. To help me grasp the concepts, I’ve made a test site at this url: https://hopeful-borg-f3c20e.netlify.com/
I can either select “read” or “write”. The read option allows me to specify the path to the file I want to read from. The write option allows me to specify the path to the file and a string that will be written in the file. Here is the code in each of the files:
test.js (Node.js lambda functions):
var fs = require(“fs”);

function parseRequestData(data, isEncoded) {
    let convertedData;
    if (isEncoded) {
        convertedData = new Buffer(data, "base64").toString("ascii");
    } else {
        convertedData = data;
    }
    return JSON.parse(convertedData);
}

function readFile(filePath) {
    return fs.readFileSync(filePath).toString("ascii");
}

function writeFile(filePath, data) {
    fs.writeFileSync(filePath, data);
}

exports.handler = function(event, context, callback) {
    let returnParams = { statusCode: 200 };
    let parsedRequest = parseRequestData(event.body, event.isBase64Encoded);

    if (parsedRequest.data) {
        writeFile(parsedRequest.file, parsedRequest.data);
        returnParams.body = "Wrote " + readFile(parsedRequest.file) + " to " + parsedRequest.file;
    } else {
        returnParams.body = readFile(parsedRequest.file);
    }
    callback(null, returnParams);
}

testScript.js (client-side Javascript):

function sendData() {
    let formObject = document.getElementById("dataCollector").elements;
    let params = {};
    params.file = formObject.namedItem("fileSrc").value;
    let dataObject = formObject.namedItem("data");
    if (dataObject) {
        params.data = dataObject.value;
    }

   //File Path: src/textFiles/testFile.txt
    console.log(JSON.stringify(params));

   $.post("/.netlify/functions/test",
        JSON.stringify(params),
        function(data, status) {
            console.log("This is the returned data:");
            console.log(data);
        }, "text");
}

To test this locally, I ran netlify dev. The code above was able to both read and write to the file located at src/textFiles/testFile.txt in my local site repository. I then pushed the code online to Github for which Netlify did an automatic build. Once the build was complete, I went to the site (https://hopeful-borg-f3c20e.netlify.com/) and did the exact same thing as what I did locally. I provided the same path to the file, but it said that the file cannot be found.

@Dennis then said that lambda functions deployed to AWS won’t have access to my repo’s files:

My question now is if it doesn’t have access, where should I put this file so that it does access so I can do what I need to?

As for the differences I mentioned above, I was referring to the fact that when I ran netlify dev on my machine and made the request, when I received the data at the endpoint, it was encoded in base 64. When I pushed the code online and tested it there, the data was in text format, so the decoding method I had in there caused more problems than solutions. I later realized that I can solve this problem by checking the isBase64Encoded variable to determine if I need to decode or not.

Hello @RobotCrazy, if you want to store a json file in your repo, that’s fine, but you’ll need to access that file directly from your repository via Github’s API (if you’re using that provider) and not the ‘fs’ module. Note that the deployed lambda function is ‘read-only’ and all the files in a given deploy is also ‘read-only’. Hope that clarifies that bit for you.

With regards to the base64 issue, I suspect it is due to the fact that you are trying to perform a direct filesystem manipulation which fails. Once you change your approach to accessing your json file directly from your repo, things should be more predictable.

Let us know how it goes.