I'm creating a chrome extension that scrapes photos from various websites. I have the URLs of each website and I want to fetch and parse those websites, and then grab specific img elements by class or ID.
I'm using the Javascript fetch() function together with DOMParser to parse the HTML and it returns the Document successfully, but any DOM query returns empty NodeLists / HTMLCollections.
fetch("www.example")
.then(res => {
const parser = new DOMParser();
const htmlDoc = parser.parseFromString(res, "text/html");
console.log(htmlDoc.querySelectorAll('h1'))
})
This gives me an empty NodeList. Why?? And how can I get the elements I'm looking for?
I'm creating a chrome extension that scrapes photos from various websites. I have the URLs of each website and I want to fetch and parse those websites, and then grab specific img elements by class or ID.
I'm using the Javascript fetch() function together with DOMParser to parse the HTML and it returns the Document successfully, but any DOM query returns empty NodeLists / HTMLCollections.
fetch("www.example.")
.then(res => {
const parser = new DOMParser();
const htmlDoc = parser.parseFromString(res, "text/html");
console.log(htmlDoc.querySelectorAll('h1'))
})
This gives me an empty NodeList. Why?? And how can I get the elements I'm looking for?
Share Improve this question asked May 31, 2019 at 19:30 maxmax 611 silver badge3 bronze badges 4-
10
I think you need to do
res.text()orres.body(). Check the Fetch Response documentation. – user47589 Commented May 31, 2019 at 19:33 - 1 The point Amy is making is that you are attempting to parse a DOM out of a Response object rather than the text returned from the server. – Randy Casburn Commented May 31, 2019 at 19:38
- @RandyCasburn Whats the difference? And what would the solution look like? – max Commented May 31, 2019 at 19:42
- @max - please see the answer – Randy Casburn Commented May 31, 2019 at 19:48
1 Answer
Reset to default 8You'll need to parse the response before you use it. Like this:
fetch("//randycasburn.")
.then(res => res.text())
.then(html => {
const parser = new DOMParser();
const htmlDoc = parser.parseFromString(html, "text/html");
console.log(htmlDoc.querySelector('span').textContent)
})Otherwise by passing res itself a DOM will be created and res will be casted to a string ([object Response]), which results in this DOM: (note no H1)
<html>
<head></head>
<body>[object Response]</body>
</html>