How can I get table 'td' values with CasperJS?

The HTML source looks like than this:

<table id="my_table">
  <tr id='header'>
    <th>sth_head_name</th>
    <th>ath_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
</table>

I'd want to get table values using CasperJS. Firstly, I need to select the rows of table; and then I want to get 'td' values. How can I solve this?

I tried a lot of ways, but those didn't work. My solution would look like something similar that you can see below. Its important, that firstly select 'table_rows'; and then select that's td value inside the for cycle.

var table_rows = casper.getElementsByXpath("//table[@id='my_table']/tr[not(@id='header')]");

for (var i = 0; i < table_rows.length; i++) {
  var firstRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=2]");
  var secondRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=4]");
  var firstRequiredCell = firstRequiredCell_query.text;
  var secondRequiredCell = secondRequiredCell_query.text;
}

How can I get table 'td' values with CasperJS?

The HTML source looks like than this:

<table id="my_table">
  <tr id='header'>
    <th>sth_head_name</th>
    <th>ath_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
</table>

I'd want to get table values using CasperJS. Firstly, I need to select the rows of table; and then I want to get 'td' values. How can I solve this?

I tried a lot of ways, but those didn't work. My solution would look like something similar that you can see below. Its important, that firstly select 'table_rows'; and then select that's td value inside the for cycle.

var table_rows = casper.getElementsByXpath("//table[@id='my_table']/tr[not(@id='header')]");

for (var i = 0; i < table_rows.length; i++) {
  var firstRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=2]");
  var secondRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=4]");
  var firstRequiredCell = firstRequiredCell_query.text;
  var secondRequiredCell = secondRequiredCell_query.text;
}

• javascript
• html
• web-scraping
• casperjs

1 Answer 1

CasperJS has two contexts. You can only access the DOM directly only from the page context which you get access to inside of casper.evaluate()¹. It is sandboxed and therefore variables defined outside are not available in evaluate().

__utils__.getElementsByXpath() and __utils__.getElementByXpath() are only available in the page context where casper is not available. Those two functions return DOM nodes directly, so those nodes itself don't have the getElementByXpath() function on them.

But you don't need that at all:

casper.then(function(){
    var info = this.evaluate(function(){
        var table_rows = __utils__.getElementsByXpath("//table[@id='my_table']/tr[not(@id='header')]");

        return table_rows.map(function(tr){
            return {
                a: tr.children[1].textContent,
                b: tr.children[3].textContent
            };
        });
    });
    this.echo(JSON.stringify(info, undefined, 4));
});

You can use all of the ways to traverse the DOM like children, querySelector() or document.evaluate().

¹ Please also read the PhantomJS documentation of the same function.