I need to load a couple thousand records of user data (user contacts in a contact-management system, to be precise) from a REST service and run a seach on them. Unfortunately, the REST service doesn't offer a search which meets my needs, so I'm reduced to just loading a bunch of data and searching through it myself. Loading the records is time-consuming, so I only want to do it once for each user.

Obviously this data needs to be cached. Unfortunately, server-side caching is not an option. My client runs apps on multiple servers, and there's no way to predict which server a given request will land on.

So, the next option is to cache this data on the browser side and run searches on it there. For a user with thousands of contacts, this could mean caching several megs of data. What problems might I run in to storing several megs of javascript data in browser memory?

I need to load a couple thousand records of user data (user contacts in a contact-management system, to be precise) from a REST service and run a seach on them. Unfortunately, the REST service doesn't offer a search which meets my needs, so I'm reduced to just loading a bunch of data and searching through it myself. Loading the records is time-consuming, so I only want to do it once for each user.

Obviously this data needs to be cached. Unfortunately, server-side caching is not an option. My client runs apps on multiple servers, and there's no way to predict which server a given request will land on.

So, the next option is to cache this data on the browser side and run searches on it there. For a user with thousands of contacts, this could mean caching several megs of data. What problems might I run in to storing several megs of javascript data in browser memory?

• javascript
• ajax
• caching

• I'm not sure what the limit would be, nor if there's a way to find what the limit is (as I'm sure it's system-dependent). But the worst case scenario is browser crash. This, I know from experience doing my own silly things with JS. (I don't mean that your project is silly; mine was.) – Brian S Commented Jul 19, 2010 at 16:28
• 6 "What happens at 9000?" -> I suspect this is "Over Nine Thousand!!!" meme, not useful information. – Brian S Commented Jul 19, 2010 at 16:29
• Sounds like you are opening a window with a hammer. It will work very slowly, but it is just going to create a bigger mess in the end. I love when people throw JavaScript code into the equation to fix flaws in bad engineering on the backend. You can use localstoage and web workers if the browser supports them, but do not expect a mobile browser to do any of this. – epascarello Commented Jul 19, 2010 at 16:45
• 2 @espascarello - Opening a window with a hammer would actually be pretty quick. Still messy, though. – JasCav Commented Jul 19, 2010 at 17:29

4 Answers 4

Storing several megs of Javascript data should cause no problems. Memory leaks will. Think about how much RAM modern puters have - a few megabytes is a molecule in the drop in the proverbial bucket.

Be careful when doing anything client side if you intend your users to use mobile devices. While desktops won't have an issue, Mobile Safari will stop working at (I believe) 10Mb of JavaScript data. (See this article for more info on Mobile Safari). Other mobile browsers are likely to have similar memory restrictions. Figure out the minimal set of info that you can return to allow the user to perform the search, and then lazy load richer records from the REST API as you need them.

As an alternative, proxy the REST Service in question, and create your own search on a server that you then control. You could do this with pretty quickly and easily with Python + Django + XML Models. No doubt there are equally simple ways to do this with whatever your preferred dev language is. (In re-reading, I see that you can't do server-side caching which may make this point moot).

You can manage tens of thousands of records safely in the browser. I'm running search & sorting benchmarks with jOrder (http://github./danstocker/jorder) on such datasets with no problem.

I would look at a distributed server side cache. If you keep the data in the browser, as system grows you will have to increase the browser cache lifetime to keep traffic down.