I'm doing a project where my parser steals gets data about every video on the specific site and save it to my database. I have acplished everything except full link to the video which is hidden.
There is a player, which automaticaly starts on page load. I have found the JavaScript code which starts the player:

function getVidData(resolution, init) {
    << some code here >>
    jQuery.ajax({type: 'POST', url: '/ajaxdata.php', dataType: 'json', data: 'mod=videodata&vid=48902&res=' + resolution, success: function (response) {
        if (response.error != '' && response.error != undefined) {
        << error handling code here >>
        } else {
            StartPlayer(response.width, response.height, response.filename);
        }
    }  });
}

So after a call if no error found it starts a player using filename from response. That is what I need.
I rechecked a call in Live HTTP Headers:

http://<< SITE_URL >>/ajaxdata.php
POST /ajaxdata.php HTTP/1.1
Host: << SITE_URL >>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: << VIDEO_PAGE >>
Content-Length: 31
Cookie: << COOKIE VALUES >>
DNT: 1
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
    mod=videodata&vid=48901&res=640

HTTP/1.1 200 OK
Server: nginx/1.5.9
Date: Tue, 22 Apr 2014 16:30:06 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Expires: Tue, 22 Apr 2014 16:30:05 GMT
Cache-Control: no-cache
Pragma: no-cache
Content-Encoding: gzip

So it calls ajaxdata.php with specific params and in response i should find the filename.
However this Python code returns absolutely nothing to me (neither content nor errors)

import requests

url = "http://LALLALAA/ajaxdata.php"
data_video = {"mod": "videodata", "vid": "48901", 'res': '640'}

s = requests.Session()
s.post(login_url, data=login_data) # Authentication

content = s.post(url, data=data_video)
print content.content

Variable content prints only "Response [200]"
Now I'm pletely stuck and would be grateful if anyone could point to errors I done or solutions i could try.

Thanks

I'm doing a project where my parser steals gets data about every video on the specific site and save it to my database. I have acplished everything except full link to the video which is hidden.
There is a player, which automaticaly starts on page load. I have found the JavaScript code which starts the player:

function getVidData(resolution, init) {
    << some code here >>
    jQuery.ajax({type: 'POST', url: '/ajaxdata.php', dataType: 'json', data: 'mod=videodata&vid=48902&res=' + resolution, success: function (response) {
        if (response.error != '' && response.error != undefined) {
        << error handling code here >>
        } else {
            StartPlayer(response.width, response.height, response.filename);
        }
    }  });
}

So after a call if no error found it starts a player using filename from response. That is what I need.
I rechecked a call in Live HTTP Headers:

http://<< SITE_URL >>/ajaxdata.php
POST /ajaxdata.php HTTP/1.1
Host: << SITE_URL >>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: << VIDEO_PAGE >>
Content-Length: 31
Cookie: << COOKIE VALUES >>
DNT: 1
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
    mod=videodata&vid=48901&res=640

HTTP/1.1 200 OK
Server: nginx/1.5.9
Date: Tue, 22 Apr 2014 16:30:06 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Expires: Tue, 22 Apr 2014 16:30:05 GMT
Cache-Control: no-cache
Pragma: no-cache
Content-Encoding: gzip

So it calls ajaxdata.php with specific params and in response i should find the filename.
However this Python code returns absolutely nothing to me (neither content nor errors)

import requests

url = "http://LALLALAA/ajaxdata.php"
data_video = {"mod": "videodata", "vid": "48901", 'res': '640'}

s = requests.Session()
s.post(login_url, data=login_data) # Authentication

content = s.post(url, data=data_video)
print content.content

Variable content prints only "Response [200]"
Now I'm pletely stuck and would be grateful if anyone could point to errors I done or solutions i could try.

Thanks

• javascript
• php
• python
• ajax

• 3 Better start experimenting with the headers then; s.post(url, data=data_video, headers={'X-Requested-With', 'XMLHttpRequest'}), etc. – Martijn Pieters Commented Apr 22, 2014 at 17:18

1 Answer 1

As Martijn Pieters suggested, I tried headers one by one and found that this bination is working now:

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0',
    'Accept': 'application/json, text/javascript, */*; q=0.01',
    'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
    'X-Requested-With': 'XMLHttpRequest'
}

s = requests.Session()
s.post(login_url, data=login_data)

content = s.post(url, data=data_video, headers=headers)

I thank everyone and especially Martijn Pieters.