最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

Best way to store a huge list with hashes in Javascript - Stack Overflow

programmeradmin0浏览0评论

I have a list with 10.000 entrys.

for example

myList = {};
myList[hashjh5j4h5j4h5j4]
myList[hashs54s5d4s5d4sd]
myList[hash5as465d45ad4d]
....

I dont use an array (0,1,2,3) because i can check very fast
-> if this hash exist or not.

if(typeof myObject[hashjh5j4h5j4h5j4] == 'undefined')
{
  alert('it is new'); 
}
else
{
  alert('old stuff'); 
}

But i am not sure, is this a good solution?
Is it maybe a problem to handle an object with 10.000 entries?

EDIT:
I try to build an rss feed reader which shows only new feeds. So i calculate an hash from the link (every news has an uniqe link) and store it in the object (mongoDB). BTW: 10.000 entrys is not the normal case (but it is possible)

I have a list with 10.000 entrys.

for example

myList = {};
myList[hashjh5j4h5j4h5j4]
myList[hashs54s5d4s5d4sd]
myList[hash5as465d45ad4d]
....

I dont use an array (0,1,2,3) because i can check very fast
-> if this hash exist or not.

if(typeof myObject[hashjh5j4h5j4h5j4] == 'undefined')
{
  alert('it is new'); 
}
else
{
  alert('old stuff'); 
}

But i am not sure, is this a good solution?
Is it maybe a problem to handle an object with 10.000 entries?

EDIT:
I try to build an rss feed reader which shows only new feeds. So i calculate an hash from the link (every news has an uniqe link) and store it in the object (mongoDB). BTW: 10.000 entrys is not the normal case (but it is possible)

Share Improve this question edited Sep 23, 2011 at 8:28 Peter asked Sep 23, 2011 at 7:45 PeterPeter 11.9k31 gold badges101 silver badges155 bronze badges 1
  • Please tell us what you are trying to acplish, as it will shape the kind of suggestions you get. – Sky Kelsey Commented Sep 23, 2011 at 8:14
Add a ment  | 

4 Answers 4

Reset to default 3

My advice:

  1. Use as small of a hash as possible for the task at hand. If you are dealing with hundreds of hashable strings, pared to billions, then your hash length can be relatively small.
  2. Store the hash as an integer, not a string, to avoid making it take less room than needed.
  3. Don't store as objects, just store them in a simple binary tree log2(keySize) deep.

Further thoughts:

  1. Can you e at this with a hybrid approach? Use hashes for recent feeds less than a month old, and don't bother showing items more than a month old. Store the hash and date together, and clean out old hashes each day?

You can use the in operator:

if ('hashjh5j4h5j4h5j4' in myList) { .. }

However, this will also return true for members that are in the objects prototype chain:

Object.prototype.foo = function () {};
if ("foo" in myList) { /* will be true */ };

To fix this, you could use hasOwnProperty instead:

if (myList.hasOwnProperty('hashjh5j4h5j4h5j4')) { .. }

Whilst you yourself may not have added methods to Object.prototype, you cannot guarantee that other 3rd party libraries you use haven't; incidentally, extending Object.prototype is frowned upon, so you shouldn't really do it. Why?; because you shouldn't modify things you don't own.

10.000 is quite a lot. You may consider storing the hashes in a database and query it using ajax. It maybe takes a bit longer to query one hash but your page loads much faster.

It is not a problem in modern browser on modern puters in any way.

10k entries that take up 50 bytes each would still take up less than 500KB ram.

As long as the js is served gzipped then bandwidth is no problem - but do try to serve the data as late as possible so they don't block perceived pageload performance.

All in all, unless you wish to cater to cellphones then your solution is fine.

发布评论

评论列表(0)

  1. 暂无评论