WEB SCRAPING made simple with JAVASCRIPT tutorial
Why Stack Overflow recommends it
Stack Overflow users cite it as the tutorial they followed when their Puppeteer scrapers threw errors, such as unhandled promise rejections or XPath versus querySelector problems.
Recommended by 1 people across 1 different months, 2020 to 2021.
What it is
A short tutorial by Aaron Jack on web scraping with JavaScript, using Node.js and the Puppeteer library to launch a headless browser, load a page, select elements with XPath and read their properties. It walks through a simple scraper that pulls an image and text from a product page.
What people said
Recommends Sporego · 2020-02 · Stack Overflow · on “UnhandledPromiseRejectionWarning: TypeError: undefined is not a function”
I have been trying to get this scraper working by following the tutorial from [this video] I have tried scraping amazon in case yellow pages wasn't working. I went from node 12 to node 13. I tried using full xpath. Disabled prettier in case the format on save caused an error. Help I don't understand why this code isn't working. I swear to god I copy line by line from youtube. (node:1740) UnhandledPromiseRejectionWarning: TypeError: undefined is not a function at scrapeProduct (C:\Users\hello\coding\scrapy\server.js:8:16) at processTicksAndRejections (internal/process/task_queues.js:97:5) (node:1740) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by thr
Cites Walter Schrabmair · 2020-01 · Stack Overflow · on “How do I write a selector for DOM at Javascript vs. puppeteer $x eveluation selector?”
I have used puppeteer (More at is at: YT: Puppeteer Youtube Video [this video] WebSite [LINK] ) and could write program with the XPATH for the $x evaluation statement. const [WCPub] = await page.$x('//*[@id="details"]/div/table/tbody/tr[3]/td'); But now I would need for a querySelector also a selector string. The following transfer from XPATH does not work. Can anyone tell me the correct querySelector string base on the XPATH? WCPub = WCbookelement.querySelector('//*[@id="details"]/div/table/tbody/tr[3]/td'); I have tried that: WCbookJson.BOOKPUB = WCbookelement.querySelector('#"details" div > table > tbody > tr[3] >td').innerHTML; It is the Publication field from Worldcat: [LINK] But get an erro
Cites InspectorSchultz · 2021-01 · Stack Overflow · on “Node library puppeteer isn't working with vue.js”
I was watching a [this video] on how to use puppeteer in javascript and the page will not render if I require the library even if I use it or not. It is located right below where I imported my vue components and as such looks like this: <script> import Header from './components/Header.vue' import Item from './components/Item'; const puppeteer = require('puppeteer') Checking console showed this error: util.js?3022:602 Uncaught TypeError: The "original" argument must be of type Function at promisify (util.js?3022:602) at eval (index.js?6b55:11) at Object../node_modules/extract-zip/index.js (chunk-vendors.js:1642) at __webpack_require__ (app.js:849) at fn (app.js:151) at eval (BrowserFetcher.js?aafb
Cites hbk1414 · 2021-03 · Stack Overflow · on “(node:93364) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated”
im currently following a youtube video which is using node.js to do some web scraping. ( [this video] ) ive managed to follow the tutorial and it works so i tried doing my own example and for some reason its failing and im not sure why. i have attached the code below: const puppeteer = require('puppeteer') async function scrapeProduct(url){ const browser = await puppeteer.launch() const page = await browser.newPage() await page.goto(url) const [el] = await page.$x('//*[@id="s7viewer_swatches_listbox"]/div[1]/div/div') const src = await el.getProperty('src') const image = await src.jsonValue() console.log({image}) browser.close() } scrapeProduct(' [LINK] ')