I'm trying to scrape table data on a webpage using R (package rvest). To do that, the data needs to be in the html source file (that's where rvest looks for it apparently), but in this case it isn't.

However, data elements are shown in the Inspect panel's Elements view:

Source file shows an empty table:

Why is the data shown on inspect element but not on the source file? How can I acces the table data on html format? If I can't access through html how do I change my web scraping strategy?

*The web page is .aspx?idMenu=IPC_VAR_MEN1_HIST&codCuadro=IPC_VAR_MEN1_HIST

Source file: view-source:.aspx?idMenu=IPC_VAR_MEN1_HIST&codCuadro=IPC_VAR_MEN1_HIST

EDIT: a solution using R is appreciated

I'm trying to scrape table data on a webpage using R (package rvest). To do that, the data needs to be in the html source file (that's where rvest looks for it apparently), but in this case it isn't.

However, data elements are shown in the Inspect panel's Elements view:

Source file shows an empty table:

Why is the data shown on inspect element but not on the source file? How can I acces the table data on html format? If I can't access through html how do I change my web scraping strategy?

*The web page is https://si3.bcentral.cl/siete/secure/cuadros/cuadro_dinamico.aspx?idMenu=IPC_VAR_MEN1_HIST&codCuadro=IPC_VAR_MEN1_HIST

Source file: view-source:https://si3.bcentral.cl/siete/secure/cuadros/cuadro_dinamico.aspx?idMenu=IPC_VAR_MEN1_HIST&codCuadro=IPC_VAR_MEN1_HIST

EDIT: a solution using R is appreciated

• javascript
• html
• r
• web-scraping
• rvest

• codementor.io/codementorteam/… How to Scrape an AJAX Website using Python – Progs Commented Dec 13, 2018 at 20:54
• Thanks, but I'm looking for an R tool – David Jorquera Commented Dec 13, 2018 at 20:57
• 1 The page URLs you posted do not work: La funcionalidad Excel dinámico será descontinuada a partir del 31 de Octubre de 2018 . Translation: "The dynamic Excel function will be discontinued October 31, 2018." – Old Pro Commented Dec 14, 2018 at 20:53
• @OldPro I don't know why it throws you away... though you can enter through: si3.bcentral.cl/siete/secure/cuadros/arboles.aspx and there select on the left menu "Información histórica" -> "Variación mensual". That's the table I want to get. – David Jorquera Commented Dec 15, 2018 at 12:57

6 Answers 6

I rly wish 'experts' would stop with the "you need Selenium/Headless Chrome" since it's almost never true and introduces a needless, heavyweight third-party dependency into data science workflows.

The site is an ASP.NET site so it makes heavy use of sessions and the programmers behind this particular one force that session to start at the home ("Hello, 2000 called and would like their session state preserving model back.")

Anyway, we need to start there and progress to your page. Here's what that looks like to your browser:

We can also see from