I want to get the html source of a webpage genereted by javascript using Curl(PHP)
I tried the curl but I get just a javascript code :(
Can I use ruby to resolve my problem ?!
I want to get the html source of a webpage genereted by javascript using Curl(PHP)
I tried the curl but I get just a javascript code :(
Can I use ruby to resolve my problem ?!
Share Improve this question edited Nov 19, 2019 at 15:50 Mosè Raguzzini 15.9k1 gold badge34 silver badges45 bronze badges asked Oct 29, 2012 at 13:27 AHmedRefAHmedRef 2,62114 gold badges44 silver badges76 bronze badges 04 Answers
Reset to default 3The javascript is executed by the browser to generate the HTML. If you make a request with CURL it will just show you the actual HTML content.You would need a Javascript engine to process the Javascript after receiving the response body.
just look at any web inspector tools (in chrome just ctrl+shift+i). here you can see the changes that the javascript has on the page reflected. I dont think curl or any curl-like-tool can do this.
This is a tough problem because the JavaScript has to run to get the right code. What I would say is download all the code locally and then add in an ajax call to the code, so it can ajax the source back to you after all the js has run. Then run the code in a browser.
If you need to do this a bunch of times you could queue these pages that needed to be loaded into a db and load all the pages using php. Then once the js has ajax'd the code back to the server it can refresh and pull the next page off the queue.
Let me know if you need me to clarify anything.
This can be done by headless browser activity like phantom js a great way to create your own logic whatever you want then get result array in console for php you can try activity here https://github./jonnnnyw/php-phantomjs & also https://github./ariya/phantomjs