Get Unique Values in JavaScript Array (No Duplicates)

In JavaScript, arrays are a versatile data structure that allows us to store multiple values in a single variable. However, sometimes, you may encounter situations where you need to extract only the unique elements from an array, eliminating any duplicate elements. Luckily, JavaScript provides us with different methods to achieve this.

Iterate and Eliminate

One common approach is to create a new array and iterate over the original array elements, checking if each value already exists in the new array. If it does not, we add it to the new array. This way, we can ensure that the new array contains only a collection of unique values. Removing duplicate values from an array can streamline our code and perform operations more efficiently. Whether we are working with large datasets or want to avoid redundancies in our code, eliminating duplicate values can significantly enhance the performance of our JavaScript applications. In the upcoming sections, we will explore different techniques to get unique values in a JavaScript array, allowing us to handle data more effectively.

Below is a code snippet showing a reusable function using the filter method:

function getUniqueArray(array) {
  return array.filter((value, index) => array.indexOf(value) === index);
}

The indexof() method used in the callback function returns the first index at which a given element can be found in the array. If the current value is the first occurrence (meaning its index is equal to the current index), you add it to the new array. Otherwise, it is a duplicated element, and you discard it.

Iterate and Eliminate Using an Index

You can further develop the above example by indexing the array using Set. An index is a tradeoff between CPU performance and memory usage. You use more memory to get much lower CPU usage. Below is the example using a Set to store values in memory.

function getUniqueArray(array) {
  const valueIndex = new Set();
  return array.filter((value) => {
    if (valueIndex.has(value)) return false;
    valueIndex.add(value);
    return true;
  });
}

Using a New Set

The easiest way (better?) is to use the spread operator or spread syntax, and this feature allows you to quickly and effortlessly create a Set object from an existing array. It is a single-line solution. By utilizing the set constructor, you can easily obtain a new unique array from the original array. The Set object, a built-in feature in JavaScript, ensures that each value within the array is unique. No duplicates are allowed, making it an ideal solution for filtering out repetitive elements. This function is handy when working with large datasets or when dealing with user inputs that may contain duplicate entries. The simplicity and efficiency of this feature allow developers to streamline their code and improve its overall performance.

const a = [1, 2, 3, 2, 5];
const b = Array.from(new Set(a));

JavaScript Sets are objects that let you store unique values. We used the "uniqueness" feature of the sets to get unique values of an array. I used Array.from() function to make the example self-descriptive. However, the spread operator makes the code more compact, as seen in the following example:

const a = [1, 2, 3, 2, 5];
const b = [...new Set(a)];

Array of Objects & Array of Arrays

In the array, objects are stored differently compared to primitive values (i.e., an array of strings or an array of numbers). They are stored as an object reference instead of a value. You cannot use the previous methods as it is to find a duplicated element. It would help if you found a way to compare objects to get unique values in the resulting array. The first thing that comes to mind is to write a deep comparison function and use it to compare the array elements. However, it would be very CPU-intensive to compare the same objects again and again in a recursive manner.

Using a hash to compare objects would be much more efficient. A hash is a unique value representing the object. If you want a deterministic hash free from key order (meaning the order of keys within the object doesn't affect the hash), you can follow a custom approach to hash the object. One way to achieve this is by sorting the keys alphabetically and then hashing the sorted key-value pairs. You can use the provided function in the following code as a start because it does not support nested objects. To support nested objects, you should write a recursive function, but for the sake of simplicity of the example, I'll leave it to you as an exercise.

The example below uses Node.js' built-in crypto module. You can choose any hashing function according to your needs. It creates a SHA 256 hash to filter duplicate objects. It also supports an array of arrays.

import crypto from "node:crypto";

// Test array with duplicated values, objects and primitive values.
const a = [{ a: 1, b: 2 }, { a: 3, b: 2 }, { a: 1, b: 2 }, [1, 2], [1, 3], [1, 2], 1, 2, 1];

function getObjectHash(input) {
  let serializedData;
  if (Array.isArray(input)) {
    // Join elements of the array
    serializedData = input.join(":");
  } else if (typeof input === "object" && input !== null) {
    // Sort the object keys alphabetically
    const sortedKeys = Object.keys(input).sort();
    // Create an array of key-value pairs as strings
    serializedData = sortedKeys.map((key) => `${key}:${input[key]}`).join(",");
  } else if (typeof input === "boolean" || typeof input === "number" || typeof input === "string") {
    // Use primitive values as it is.
    serializedData = input.toString();
  }
  return crypto.createHash("sha256").update(serializedData).digest("hex");
}
function getUniqueArray(array) {
  const valueIndex = new Set();
  return array.filter((value) => {
    const hash = getObjectHash(value);
    if (valueIndex.has(hash)) return false;​
    valueIndex.add(hash);
    return true;
  });
}

Conclusion

JavaScript is a rich ecosystem and allows many options for many tasks. Which technique do you prefer? Do you have any favorite approach? Or a better method? Write your thoughts below in the comments section.