This is the current JSON file:

[{
    "name": "Peter",
    "age": 30,
    "hair color": "brown"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}]

I want to remove the duplicate Steve individual from the list. How can I make a new JSON that checks if the object's name matches and remove any duplicates in JavaScript?

This is the current JSON file:

[{
    "name": "Peter",
    "age": 30,
    "hair color": "brown"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}]

I want to remove the duplicate Steve individual from the list. How can I make a new JSON that checks if the object's name matches and remove any duplicates in JavaScript?

• javascript
• json

• Does the order matter? – thefourtheye Commented Mar 24, 2014 at 15:56
• 1 So you only care about the name, and none of the other properties? – cookie monster Commented Mar 24, 2014 at 15:58
• Order does not matter. And we only care if 'name' matches. If there are two of the same, just have one remaining. – Andrew Kind Commented Mar 24, 2014 at 16:06
• @user2836857 And what if other details in the objects are different? Which one to keep? – thefourtheye Commented Mar 24, 2014 at 16:07
• Good question. I am assuming they are all exact but you bring a good point up. – Andrew Kind Commented Mar 24, 2014 at 16:14

6 Answers 6

You must load the JSON data in to the program and parse that with JSON.parse, like this

var array = JSON.parse(content.toString())

To filter out the repeated names from the array of Objects, we use Array.prototype.filter function. You can store the names in an object, and next time when the same name appears we simply filter it out from the result.

var seenNames = {};

array = array.filter(function(currentObject) {
    if (currentObject.name in seenNames) {
        return false;
    } else {
        seenNames[currentObject.name] = true;
        return true;
    }
});

console.log(array);
# [ { name: 'Peter', age: 30, 'hair color': 'brown' },
#   { name: 'Steve', age: 55, 'hair color': 'blonde' } ]

var data = [{
    "name": "Peter",
    "age": 30,
    "hair color": "brown"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}, {
    "name": "Steve",
    "age": 55,
    "hair color": "blonde"
}]
 data = this.data.filter((obj, pos, arr) => {
            return arr.map(mapObj =>
                  mapObj.name).indexOf(obj.name) == pos;
            });
          console.log(data);

Using Underscore.js and the uniq function:

_.uniq(array, false, function (item) { return item.name; })

Loop, check, splice, repeat:

var distinctValues = {};
for (var i = 0; i < data.length; i++) {
    if (distinctValues.hasOwnProperty(data[i].name]) {
        //already has it
        data.splice(i, 1);
        i--;
    } else {
        distinctValues[data[i].name] = true;
    }
}

This was the best solution I could find that makes you able to filter on multiple values in you json object. Solution without _ (module)

array.filter((thing, index, self) =>
    index === self.findIndex((t) => (
    t.place === thing.place && t.name === thing.name // you can add more arguments here to filter more
  ))
)

//Other example
array.filter((thing, index, self) =>
    index === self.findIndex((t) => (
    t.place === thing.place && t.name === thing.name && t.time === thing.time
  ))
)

Python one-liner from CLI:

cat file_with_duplicates.json | python2 -c 'import sys; import json; sys.stdout.write(json.dumps(reduce(lambda x, y: x + [y] if y not in x else x, json.loads(sys.stdin.read()), [])))' > unique_items.txt