I have a large set of data that I would like to be split into sections loaded through infinite scroll in Meteor in order to prevent having to load a large dataset all at once; instead I would load chunks of the dataset when needed. How do I do so while preserving live page updates on each section loaded by infinite scroll?
I have a large set of data that I would like to be split into sections loaded through infinite scroll in Meteor in order to prevent having to load a large dataset all at once; instead I would load chunks of the dataset when needed. How do I do so while preserving live page updates on each section loaded by infinite scroll?
Share Improve this question edited Jan 5, 2013 at 5:27 HGandhi asked Jan 5, 2013 at 1:01 HGandhiHGandhi 1,5862 gold badges14 silver badges27 bronze badges4 Answers
Reset to default 8 +50Here is the code I am using for a movie database playground. I experimented with jQuery events, but finally decided that it is enough if the data is transferred continuously. My testing collection is 680 records, with 20 fields and one thumbnail field which is between 10-20kb per cover. Here the Coffeescript code:
Movies = new Meteor.Collection("movies")
if Meteor.isClient
# starting with 10 items that the user sees a page fast
Session.set("queryLimit", 10)
# getting the item count from the server collection
Meteor.call 'getMoviesCount', (err, result) =>
Session.set('itemsInQuery',result)
# subscribe to a subset of the collection, and change the Session var on completion
# automatically changing the limit of the publishing function with autorun
Meteor.autorun ->
Meteor.subscribe "MoviesList", Session.get("queryLimit"), onComplete = ->
if Session.get("queryLimit") < Session.get('itemsInQuery')
queryLimit = Session.get("queryLimit") + 30 #change iterator depending on result size
Session.set("queryLimit", queryLimit )
# Client helper to add more items to handlebars template
Template.app.helpers movies: ->
Movies.find({})
if Meteor.isServer
Meteor.publish "MoviesList", (limit) ->
# show whole collections once limit is reached, by setting limit to 0
limit = 0 if limit > Movies.find().count()
console.log new Date(), limit
Movies.find({}, { limit: limit, fields: {title:1 } } ) #use fields to test different result sizes
Meteor.methods
getMoviesCount: (query) ->
return Movies.find().count()
And the html:
<body>
{{> app}}
</body>
<template name="app">
{{#each movies}}
<p>{{title}}</p>
{{/each}}
</template>
I did some quick performance test and it turns out that for a few lines of text per record, the quickest way to send data to the client is a limit around 100. I tried it also with 10-20kb thumbnails which are embedded in the file. When using bigger assets Chrome became quite unresponsive when the limit was bigger than 30 records. Here a few stats done on localhost (run 3x each):
limit seconds till last record rendered.
0 79s
010 121s
020 64s
030 45s
050 34s
100 16s
200 16s
It's interesting to note that it took around 79 s when meteor server sends the whole page in one go (limit=0). I am not sure, how this can be possible, because a continious stream should be the fastest. So there is probably something funny with my stats. Any ideas?
My solution is similar to Andrej's. Because I got a lot of records in database, I don't want my server to send them all at once, so on server I got:
Meteor.publish("items", function(page) {
if(!page)
page = 1;
var limit = 30 * page;
return Items.find({}, {limit: limit};
}
Client:
Template.templateName.items = function () {
Meteor.subscribe("items", Session.get("page"));
return Items.find({});
}
And a jQuery function to observe if page got to the bottom:
$(window).scroll(function(){
if ($(window).scrollTop() == $(document).height()-$(window).height()){
Session.set("page", Session.get("page") + 1);
}
});
Also to set page initial page on template created callback:
Template.templateName.created = function() {
Session.setDefault("page", 1);
};
And in template I'm showing these items using {{#each}} Also, I have to check if there are no more records
Better solution would be to show 30 when template is created and after that to get 30 more on scroll, but I don't know how to to show them. I may have a solution, but I'm lazy to implement because I'm not sure if it's going work. I'm thinking about appending rendered html
You can set a session key that stores what page you're currently on. Something like:
Session.set("cur_page", 1);
And then you feed that into your query, eg:
Template.list.items = function() {
return Collection.find({}, {
skip: Session.get("cur_page") * page_size,
limit: page_size
});
}
Then you just update the value with Session.set("cur_page", 2) and your list will redraw with the items for page 2. Tadaa!
I didn't try this yet, but it should work. So, if you have your Collection:
var Posts = new Meteor.Collection("posts");
And your template:
<template name="posts">
{{#each posts}}
{{> post}}
{{/each}}
</template>
This could be your template-helper:
Template.posts.helpers({
posts: function() {
return Posts.find({}, {limit: Session.get("current_page")*PAGE_SIZE});
}
});
where PAGE_SIZE is the amount of posts per "page". Every time the user scrolls to the bottom of the page, you need to increment the "current_page" session variable. Then the template will be updated and your inifnite scroll (hopefully) works!