I need to load 1000 urls on a tracking/reporting app (after authenticating via an HTML form) and trigger a "resubmit" javascript function. Unfortunately there is no bulk action to handle all at once so I'm left with automation. What are my options?

.php
.php
.php
...
.php

Each of the above pages has a resubmit() javascript function triggered by a href. How can I automate triggering these?

Example:

<form action="/resubmit" method="POST">
  <input type="hidden" name="security_token" value="SUPER-LONG-HASH">
  <input type="hidden" name="url" value=".html">
  <input type="hidden" name="redirect" value="long-string">
  <script type="text/javascript">
    window["resubmit"] = function () {
      document["resubmit"].submit();
      return false;
    }
  </script>
  <a href="javascript:resubmit()" class="resubmit-class">resubmit</a>
</form>

I'm on a Mac. Unix, Perl, Bash, PHP, Automator, FireFox iMarcos are all available.

I need to load 1000 urls on a tracking/reporting app (after authenticating via an HTML form) and trigger a "resubmit" javascript function. Unfortunately there is no bulk action to handle all at once so I'm left with automation. What are my options?

http://domain./0001.php
http://domain./0002.php
http://domain./0003.php
...
http://domain./1000.php

Each of the above pages has a resubmit() javascript function triggered by a href. How can I automate triggering these?

Example:

<form action="/resubmit" method="POST">
  <input type="hidden" name="security_token" value="SUPER-LONG-HASH">
  <input type="hidden" name="url" value="http://mysite./0001.html">
  <input type="hidden" name="redirect" value="long-string">
  <script type="text/javascript">
    window["resubmit"] = function () {
      document["resubmit"].submit();
      return false;
    }
  </script>
  <a href="javascript:resubmit()" class="resubmit-class">resubmit</a>
</form>

I'm on a Mac. Unix, Perl, Bash, PHP, Automator, FireFox iMarcos are all available.

• javascript
• browser
• automation
• browser-automation

• Have you thought about cURL? – Jared Farrish Commented Oct 27, 2011 at 23:27
• Can you just call the url on each page directly that the resubmit would call? – James Black Commented Oct 27, 2011 at 23:27
• Why can't each page run the resubmit() function onload? – lsl Commented Oct 27, 2011 at 23:55
• Updated question to provide a bit more context of the actual script that's run. – Ryan Commented Oct 28, 2011 at 0:00
• 1 Out of curiosity, why did I get a bunch of down votes for this question? – Ryan Commented Oct 31, 2011 at 20:22

4 Answers 4

You should check out PhantomJS, "a headless WebKit with JavaScript API". It allows you to run a WebKit browser instance from the mand line and execute Javascript.

You might be able to save some time using Pjscrape, a tool built on top of PhantomJS that can spider multiple pages or take a long list of URLs (disclaimer: this is my project). I haven't tried it with 1,000+ URLs, but I think you could do what you describe with the following 6 lines:

pjs.addSuite({
    urls: [...], // your very long list here
    scraper: function() {
        window.resubmit();
    }
});

I already voted up the other answers, but in the end I wound up using straight AppleScript. This was helpful because it used an existing session so I didn't have to deal with any authentication problems. Thanks for all of your help, everyone. I look forward to being familiar with the tools you shared.

set thePath to (path to desktop as Unicode text) & "list_of_urls.txt"
set theFile to (open for access file thePath)
set theContent to (read theFile)
close access theFile

set theURLs to every paragraph of theContent

tell application "Safari"
    repeat with theURL in theURLs
        make new document
        set URL of front document to theURL
        delay 5
        set theScript to "document.getElementsByClassName('resubmit-class')[0].click();"
        do JavaScript theScript in current tab of first window
        do JavaScript "window.resubmit()" in front document
        delay 5
        close front document
    end repeat
end tell

I would use Ruby+Watir for that. Example code (not tested):

require "watir-webdriver"
browser = Watir::Browser.new :firefox

urls = ["http://domain./0001.php", "http://domain./0002.php"] # add more URLs here
urls.each do |url|
  browser.goto url
  browser.a(:text => "resubmit").click
end

I don't know if this will help you, but you could try Fake. I think it would allow you to automate submitting the form and make a loop.