Saving text from an element on a web page using javascript

Question

Let's say there's some text inside a particular element on a web page that I want to save. Using Javascript, how could I save that text/append it to a file "myfile.txt" on my hard drive? The element dynamically changes over time so whenever it updates i'd like it to append the new text to the file.

I've been doing some research on web scraping, and it just seems too over the top/complicated for this task.

• junwen-k: Are you using node.js ?
• SoDamnMetal: I don't know anything about node.js and yes i'm using chrome
• junwen-k: So your ideal solution is by simply providing a webpage url and an element ID, a program will extract the text inside the element and save it into a file right ?
• SoDamnMetal: yes, pretty much

Answer 1

I've written a Node.js program that fetches a webpage url every X seconds, and compare the previous and new value of a specific html element. It will only save changes to a specific output file.

Note that the previous value record will be deleted after each run of this program, meaning that the first time you run this program it will always save the extracted text ( Because there's nothing to compare to )

This program uses node-fetch and jsdom npm packages.

fs is a build in package for Node.js.

If you are new to Node.js, you can follow this to install in your computer.

const fetch = require('node-fetch');
const jsdom = require('jsdom');
const fs = require('fs');
// Local previous extracted text variable to compare and determine changes
let prevExtractedText;
// The webpage URL to fetch from
const url = 'https://en.wikipedia.org/wiki/Node.js';
// Setting your file's output path
const outputFilepath = 'myfile.txt';
// Setting timeout every 5 seconds
const timeout = 5000;
const handleOnError = (err) => {
 console.error(`! An error occurred: ${err.message}`);
 process.exit(1);
}
const handleFetchAndSaveFile = async () => {
 let html;
 try {
 console.log(`* Fetching ${url}...`);
 const resp = await fetch(url);
 console.log('* Converting response into html text...');
 html = await resp.text();
 } catch (err) {
 handleOnError(err);
 }
 // Convert into DOM in Node.js enviroment
 const dom = new jsdom.JSDOM(html);
 // Example with element of id "footer-places-privacy"
 const extractedText = dom.window.document.getElementById("footer-places-privacy").textContent;
 console.log(`* Comparing previous extracted text (${prevExtractedText}) and current extracted text (${extractedText})`);
 if (prevExtractedText !== extractedText) {
 // Update prevExtractedText 
 prevExtractedText = extractedText;
 console.log(`* Updating ${outputFilepath}...`);
 try {
 // Writing new extracted text into a file
 await fs.appendFileSync(outputFilepath, extractedText);
 console.log(`* ${outputFilepath} has been updated and saved.`);
 } catch (err) {
 handleOnError(err);
 }
 }
 console.log('--------------------------------------------------')
}
console.log(`* Polling ${url} every ${timeout}ms`);
setInterval(handleFetchAndSaveFile, timeout);

Working demo: https://codesandbox.io/s/nodejs-webpage-polling-jqf6v?fontsize=14&hidenavigation=1&theme=dark

• junwen-k: You can also research on this. MutationObserver can watch for DOM changes and could be helpful for your purposes.