I'm crawling tweets and, if available, store information to external resources (e.g., Instagram images). Now poeple start using Periscope where users tweet links to a Periscope live streaming broadcast (also then available for the next 24h). Here is an example link, but it might be invalid/obsolete soon.
Is there any way I can fetch that broadcast, i.e., store it locally as a video file?
When I look at the source code of a Periscope broadcast, I get something like that:
<html>
<head>
...
<script src=".min.js"></script>
<script src=".a8cd99eb89a488ec6ea465f57d3ad41bcd832eff.js"></script>
</head>
<body>
<div id="periscope-app" class="u-fullHeight"></div>
<script src=".min.2a8176c317fb715fd0e6c2728cb04318c5c53941.js"></script>
</body>
So everything is essentially scripted. I've tried to look into the Javascript file to see if I can somehow at least reconstruct the stream URL or something. But I wasn't successful. I also tried with Google Chrome to inspect the requests made from the side. But this also didn't bring me any further.
EDIT: Here's my current Phantom.js script:
var page = require('webpage').create(),
system = require('system'),
t, address;
if (system.args.length === 1) {
console.log('Usage: periscope.js <periscope-url>');
phantom.exit();
}
page.onResourceRequested = function(request) {
console.log('Request ' + JSON.stringify(request, undefined, 4));
};
page.onResourceReceived = function(response) {
console.log('Receive ' + JSON.stringify(response, undefined, 4));
};
address = system.args[1];
page.settings.userAgent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36';
page.open(address, function(status) {
if (status !== 'success') {
console.log('FAIL to load the address');
} else {
}
//phantom.exit();
});
When I use Google Chrome's Developer Tools I find the application/vnd.apple.mpegurl request, but it never pops up from the Phantom.js script.
I'm crawling tweets and, if available, store information to external resources (e.g., Instagram images). Now poeple start using Periscope where users tweet links to a Periscope live streaming broadcast (also then available for the next 24h). Here is an example link, but it might be invalid/obsolete soon.
Is there any way I can fetch that broadcast, i.e., store it locally as a video file?
When I look at the source code of a Periscope broadcast, I get something like that:
<html>
<head>
...
<script src="https://assets.periscope.tv/assets/bugsnag-2.min.js"></script>
<script src="https://assets.periscope.tv/js/vendor.a8cd99eb89a488ec6ea465f57d3ad41bcd832eff.js"></script>
</head>
<body>
<div id="periscope-app" class="u-fullHeight"></div>
<script src="https://assets.periscope.tv/js/application.min.2a8176c317fb715fd0e6c2728cb04318c5c53941.js"></script>
</body>
So everything is essentially scripted. I've tried to look into the Javascript file to see if I can somehow at least reconstruct the stream URL or something. But I wasn't successful. I also tried with Google Chrome to inspect the requests made from the side. But this also didn't bring me any further.
EDIT: Here's my current Phantom.js script:
var page = require('webpage').create(),
system = require('system'),
t, address;
if (system.args.length === 1) {
console.log('Usage: periscope.js <periscope-url>');
phantom.exit();
}
page.onResourceRequested = function(request) {
console.log('Request ' + JSON.stringify(request, undefined, 4));
};
page.onResourceReceived = function(response) {
console.log('Receive ' + JSON.stringify(response, undefined, 4));
};
address = system.args[1];
page.settings.userAgent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36';
page.open(address, function(status) {
if (status !== 'success') {
console.log('FAIL to load the address');
} else {
}
//phantom.exit();
});
When I use Google Chrome's Developer Tools I find the application/vnd.apple.mpegurl request, but it never pops up from the Phantom.js script.
2 Answers
Reset to default 3It uses Http Live Streaming (HLS) on Chrome Desktop so just proxy/sniff HTTP responses of type application/vnd.apple.mpegurl and get the request URL. I was able to watch your stream in VLC.
You can use Phantom.js.
the latest version of the fantastic livestreamer (1.12.2) can download periscope videos
livestreamer https://www.periscope.tv/w/aKzPpDIxNDYxN3w2OTg1MTQxNZnjGACxw2Nyl6RSvdnX8T3EaE7z5mSvxfK_-uqS5SQv best --output output.mp4
if you want to do it programmatically seeing as this is StackOverflow, the periscope plugin in python is here: https://github./chrippa/livestreamer/blob/develop/src/livestreamer/plugins/periscope.py