Alright, I managed to get it working how I want (although some/most might disagree with the method). I used Flask as recommended below. The part that might be considered "wrong" is the 404 check loop. It isn't proper design and results in a "stuck script" error in some browsers if it goes on for too long. However, the script I want to run doesn't last long enough for that to be a problem.
Thanks for the help and please let me know if you have any other suggestions.
Flask App:
import threading
import subprocess
import os
import sys
from flask import Flask
from flask import render_template, abort
app = Flask(__name__)
app.debug = True
def run_script():
theproc = subprocess.Popen([sys.executable, "run_me.py"])
theprocmunicate()
@app.route('/')
def index():
return render_template('index.html')
@app.route('/generate')
def generate():
threading.Thread(target=lambda: run_script()).start()
return render_template('processing.html')
@app.route('/is_done')
def is_done():
hfile = "templates\\itworked.html"
if os.path.isfile(hfile):
return render_template('itworked.html')
else:
abort(404)
if __name__ == "__main__":
app.run()
processing.html:
<!DOCTYPE html>
<html>
<head>
<script src="/static/jquery.min.js"></script>
</head>
<body>
<p>Processing...</p>
<script>
setInterval(function()
{
var http = new XMLHttpRequest();
http.open('HEAD', "is_done", false);
http.send();
if (http.status==200)
window.location.replace("is_done");
},2000);
</script>
</body>
</html>
Original question:
I am a beginner/intermediate Python programmer and a beginning web developer so please be gentle. What I want: A user clicks a hyperlink and is taken to an intermediate web page that says something like "Processing...". In the background (on the server) this link triggers a python script that processes a file and creates a new web page. On completion of the script, the user is redirected to the newly created page. What I have: I have a script that does the processing and spits out a new page. What I haven't figured out is how to trigger the python script and then trigger the redirect on script completion. I've looked at web frameworks like django, and cgi scripting. But I haven't found anything I feel fits the bill. It's very possible I'm just missing something completely obvious so I'd really appreciate any help. Thanks.
Alright, I managed to get it working how I want (although some/most might disagree with the method). I used Flask as recommended below. The part that might be considered "wrong" is the 404 check loop. It isn't proper design and results in a "stuck script" error in some browsers if it goes on for too long. However, the script I want to run doesn't last long enough for that to be a problem.
Thanks for the help and please let me know if you have any other suggestions.
Flask App:
import threading
import subprocess
import os
import sys
from flask import Flask
from flask import render_template, abort
app = Flask(__name__)
app.debug = True
def run_script():
theproc = subprocess.Popen([sys.executable, "run_me.py"])
theproc.communicate()
@app.route('/')
def index():
return render_template('index.html')
@app.route('/generate')
def generate():
threading.Thread(target=lambda: run_script()).start()
return render_template('processing.html')
@app.route('/is_done')
def is_done():
hfile = "templates\\itworked.html"
if os.path.isfile(hfile):
return render_template('itworked.html')
else:
abort(404)
if __name__ == "__main__":
app.run()
processing.html:
<!DOCTYPE html>
<html>
<head>
<script src="/static/jquery.min.js"></script>
</head>
<body>
<p>Processing...</p>
<script>
setInterval(function()
{
var http = new XMLHttpRequest();
http.open('HEAD', "is_done", false);
http.send();
if (http.status==200)
window.location.replace("is_done");
},2000);
</script>
</body>
</html>
Original question:
I am a beginner/intermediate Python programmer and a beginning web developer so please be gentle. What I want: A user clicks a hyperlink and is taken to an intermediate web page that says something like "Processing...". In the background (on the server) this link triggers a python script that processes a file and creates a new web page. On completion of the script, the user is redirected to the newly created page. What I have: I have a script that does the processing and spits out a new page. What I haven't figured out is how to trigger the python script and then trigger the redirect on script completion. I've looked at web frameworks like django, and cgi scripting. But I haven't found anything I feel fits the bill. It's very possible I'm just missing something completely obvious so I'd really appreciate any help. Thanks.
Share Improve this question edited Jun 7, 2012 at 18:06 bem3ry asked Jun 5, 2012 at 19:26 bem3rybem3ry 3452 gold badges3 silver badges8 bronze badges 4 |4 Answers
Reset to default 8A simple way to tackle this problem would be to use a simple web framework like Flask to build the web part uf your system. In the request handler for your magic link, you would need to spawn your script and keep track of it. A simple way to see if your script is done and relay that to your user is to periodically send off an ajax request to check for completion.
So for example the Flask website could look like:
import threading
import subprocess
import uuid
from flask import Flask
from flask import render_template, url_for, abort, jsonify, request
app = Flask(__name__)
background_scripts = {}
def run_script(id):
subprocess.call(["/path/to/yourscript.py", "argument1", "argument2"])
background_scripts[id] = True
@app.route('/')
def index():
return render_template('index.html')
@app.route('/generate')
def generate():
id = str(uuid.uuid4())
background_scripts[id] = False
threading.Thread(target=lambda: run_script(id)).start()
return render_template('processing.html', id=id)
@app.route('/is_done')
def is_done():
id = request.args.get('id', None)
if id not in background_scripts:
abort(404)
return jsonify(done=background_scripts[id])
And the index.html:
<a href="{{ url_for('generate') }}">click me</a>
And processing.html:
<html>
<head>
<script src="/static/jquery.js"></script>
<script>
function ajaxCallback(data) {
if (data.done)
window.location.replace("http://YOUR_GENERATED_PAGE_URL");
else
window.setTimeout(function() {
$.getJSON('{{ url_for('is_done') }}', {id: {{ id }} }, ajaxCallback);
}, 3000);
}
$(document).ready(function(){
ajaxCallback({done=false});
});
</script>
</head>
<body>
Processing...
</body></html>
This is all untested code at the moment, but I hope you get some idea on how to approach this problem. Also keep in mind that this will only work if you serve the page from one process, so if you set up Apache and mod_wsgi, make sure there is only one process in the process group.
If you need a more complex solution, you might want to look at message queues and the like.
This is probably too heavy-duty a solution for you, but assuming you cannot do away with the "creating a file on the server" part, this is considered a "best practice" solution in web development:
Use a web framework like Flask or Django to serve pages. When the user requests the job be started via a request (preferably a POST request because GET requests should not cause significant side-effects), the framework sends a message to a delayed task system like Celery. The user is then shown a page saying that the job has been submitted.
At this point you can use ajax to poll the server and see when the job is completed, but typically you don't want that to be your only method of seeing if the job is completed. If the connection is interrupted for any reason, for instance if the user mistakenly closes the browser or the tab (very common), the effort will be wasted and the user will have to start the process over again. You should have a backup way of communicating with the user if at all possible; this could be a flash message that appears elsewhere on the site, an automated "job finished" email, a box in the homepage that shows recently completed jobs, etc. How practical and important this is depends on whether the user is logged in and how long the job takes/whether it has side effects other than the creation of a file.
Finally you can allow the user to access the page through a url unique to that user's job results. Be sure to inform the user if that url will not be valid indefinitely, for instance if the file will be deleted on a timer.
Two simplish ways to do this:
1) Use ajax to poll the server. Check for completion every 10 seconds or whatever interval you deem appropriate. E.g. while the script is processing, it writes to a file, or database, whatever it might be to indicate the completion percentage as well as identify the process somehow (in case you have multiple scripts running concurrently you'll want to know which one is Bob's and which one is Suzy's).
2) Disable output buffering and stream the output to the client. This is simpler than the first option. Basically, run the script and as it's processing you can output javascript code to say, update a progress indicator. You need to disable output buffering, or manually flush the buffer otherwise your client might not receive the output immediately (the indicator update may only be visible to the client when it hits 100%). Google how to do it on whatever setup (e.g. PHP, Django) you're running.
What you want is possible, but it's really about 3 problems.
The first question is, how do you create a new file from PHP. I can't really answer this one. What you need to do, probably depends on what kind of file you're trying to create and why.
The second question is, how do you let the user know this is happening, and the third is how do you redirect to the new page.
Personally, I think the shortest route to victory, is probably to have your client-side javascript make an AJAX request to the server page or CGI module that creates your new content.
- On button click, make an ajax request to the PHP page that generates your new content page.
- Present your busy indicator.
- Have the requested page return the URL of the new page as its content.
- When the request completes redirect to the returned URL via javascript.
As someone new to web development, the easiest thing to do might be to look into jQuery and it's AJAX functionality. The wrappers there make it pretty easy to do the above. Of course you can do the same with any of the other major javascript frameworks or straight javascript as well.
For reference:
- http://api.jquery.com/jQuery.ajax/
- http://www.tizag.com/phpT/filecreate.php
setTimeoutinstead. Also, your version only works for generating one file and does not account for multiple people visiting your page at the same time, they might try to generate the file concurrently. Perhaps check if the file already exists before running the script. – mensi Commented Jun 7, 2012 at 8:44