In my Vue application, I have Vuex store modules with large arrays of resource objects in their state. To easily access individual resources in those arrays, I make Vuex getter functions that map resources or lists of resources to various keys (e.g. 'id' or 'tags'). This leads to sluggish performance and a huge memory memory footprint. How do I get the same functionality and reactivity without so much duplicated data?
Store Module Example
export default {
state: () => ({
all: [
{ id: 1, tags: ['tag1', 'tag2'] },
...
],
...
}),
...
getters: {
byId: (state) => {
return state.all.reduce((map, item) => {
map[item.id] = item
return map
}, {})
},
byTag: (state) => {
return state.all.reduce((map, item, index) => {
for (let i = 0; i < item.tags.length; i++) {
map[item.tags[i]] = map[item.tags[i]] || []
map[item.tags[i]].push(item)
}
return map
}, {})
},
}
}
Component Example
export default {
...,
data () {
return {
itemId: 1
}
},
puted: {
item () {
return this.$store.getters['path/to/byId'][this.itemId]
},
relatedItems () {
return this.item && this.item.tags.length
? this.$store.getters['path/to/byTag'][this.item.tags[0]]
: []
}
}
}
In my Vue application, I have Vuex store modules with large arrays of resource objects in their state. To easily access individual resources in those arrays, I make Vuex getter functions that map resources or lists of resources to various keys (e.g. 'id' or 'tags'). This leads to sluggish performance and a huge memory memory footprint. How do I get the same functionality and reactivity without so much duplicated data?
Store Module Example
export default {
state: () => ({
all: [
{ id: 1, tags: ['tag1', 'tag2'] },
...
],
...
}),
...
getters: {
byId: (state) => {
return state.all.reduce((map, item) => {
map[item.id] = item
return map
}, {})
},
byTag: (state) => {
return state.all.reduce((map, item, index) => {
for (let i = 0; i < item.tags.length; i++) {
map[item.tags[i]] = map[item.tags[i]] || []
map[item.tags[i]].push(item)
}
return map
}, {})
},
}
}
Component Example
export default {
...,
data () {
return {
itemId: 1
}
},
puted: {
item () {
return this.$store.getters['path/to/byId'][this.itemId]
},
relatedItems () {
return this.item && this.item.tags.length
? this.$store.getters['path/to/byTag'][this.item.tags[0]]
: []
}
}
}
3 Answers
Reset to default 5To fix this problem, look to an old, standard practice in programming: indexing. Instead of storing a map with the full item values duplicated in the getter, you can store a map to the index of the item in state.all. Then, you can create a new getter that returns a function to access a single item. In my experience, the indexing getter functions always run faster than the old getter functions, and their output takes up a lot less space in memory (on average 80% less in my app).
New Store Module Example
export default {
state: () => ({
all: [
{ id: 1, tags: ['tag1', 'tag2'] },
...
],
...
}),
...
getters: {
indexById: (state) => {
return state.all.reduce((map, item, index) => {
// Store the `index` instead of the `item`
map[item.id] = index
return map
}, {})
},
byId: (state, getters) => (id) => {
return state.all[getters.indexById[id]]
},
indexByTags: (state) => {
return state.all.reduce((map, item, index) => {
for (let i = 0; i < item.tags.length; i++) {
map[item.tags[i]] = map[item.tags[i]] || []
// Again, store the `index` not the `item`
map[item.tags[i]].push(index)
}
return map
}, {})
},
byTag: (state, getters) => (tag) => {
return (getters.indexByTags[tag] || []).map(index => state.all[index])
}
}
}
New Component Example
export default {
...,
data () {
return {
itemId: 1
}
},
puted: {
item () {
return this.$store.getters['path/to/byId'](this.itemId)
},
relatedItems () {
return this.item && this.item.tags.length
? this.$store.getters['path/to/byTag'](this.item.tags[0])
: []
}
}
}
The change seems small, but it makes a huge difference in terms of performance and memory efficiency. It is still fully reactive, just as before, but you're no longer duplicating all of the resource objects in memory. In my implementation, I abstracted out the various indexing methodologies and index expansion methodologies to make the code very maintainable.
You can check out a full proof of concept on github, here: https://github./aidangarza/vuex-indexed-getters
While I agree with @aidangarza, I think your biggest issue is the reactivity. Specifically the puted property. This adds a lot of bloated logic and slow code that listens for everything - something you don't need.
Finding the related items will always lead you to looping through the whole list - there's no easy way around it. BUT it will be much faster if you call this by yourself.
What I mean is that puted properties are about something that is going to be puted. You are actually filtering your results. Put a watcher on your variables, and then call the getters by yourself. Something along the lines (semi-code):
watch: {
itemId() {
this.item = this.$store.getters['path/to/byId'][this.itemId]
}
}
You can test with item first and if it works better (which I believe it will) - add watcher for the more plex tags.
Good luck!
While only storing select fields is a good intermediate option (per @aidangarza), it's still not viable when you end up with really huge sets of data. E.g. actively working with 2 million records of "just 2 fields" will still eat your memory and ruin browser performance.
In general, when working with large (or unpredictable) data sets in Vue (using VueX), simply skip the get and mit mechanisms altogether. Keep using VueX to centralize your CRUD operations (via Actions), but do not try to "cache" the results, rather let each ponent cache what they need for as long as they're using it (e.g. the current working page of the results, some projection thereof, etc.).
In my experience VueX caching is intended for reasonably bounded data, or bounded subsets of data in the current usage context (i.e. for a currently logged in user). When you have data where you have no idea about its scale, then keep its access on an "as needed" basis by your Vue ponents via Actions only; no getters or mutations for those potentially huge data sets.