Find Most Frequent Row Or Mode Of A Matrix Of Vectors - Python / Numpy

I have a numpy array of shape (?,n) that represents a vector of n-dimensional vectors. I want to find the most frequent row. So far it seems that the best way is to just iterate o

Solution 1:

Here's an approach using NumPy views, which should be pretty efficient -

def mode_rows(a):
    a = np.ascontiguousarray(a)
    void_dt = np.dtype((np.void, a.dtype.itemsize * np.prod(a.shape[1:])))
    _,ids, count = np.unique(a.view(void_dt).ravel(), \
                                return_index=1,return_counts=1)
    largest_count_id = ids[count.argmax()]
    most_frequent_row = a[largest_count_id]
    return most_frequent_row

Sample run -

In [45]: # Let's have a random arrayb with three rows(2,4,8) and two rows(1,7)
    ...: # being duplicated. Thus, the most freequent row must be 2 here.
    ...: a = np.random.randint(0,9,(9,5))
    ...: a[4] = a[8]
    ...: a[2] = a[4]
    ...: 
    ...: a[1] = a[7]
    ...: 

In [46]: a
Out[46]: 
array([[8, 8, 7, 0, 7],
       [7, 8, 2, 6, 1],
       [2, 2, 5, 7, 6],
       [6, 5, 8, 8, 5],
       [2, 2, 5, 7, 6],
       [5, 7, 3, 6, 3],
       [2, 8, 7, 2, 0],
       [7, 8, 2, 6, 1],
       [2, 2, 5, 7, 6]])

In [47]: mode_rows(a)
Out[47]: array([2, 2, 5, 7, 6])

Solution 2:

The numpy_indexed package (dsiclaimer: I am its author) has functionality that does exactly this, that works on any number of dimensions:

import numpy_indexed as npirow= npi.mode(arr)

Under the hood, it is like Divakar's solution in terms of algorithmics and complexity, with a few more bells and whistles; see the 'weights' and 'return_indices' kwargs.

Solution 3:

If you're able to use Pandas, here's one approach, which draws heavily from this answer:

import numpy as np
import pandas as pd

# generate sample data
ncol = 5
nrow = 20000
matrix = np.random.randint(0,ncol,ncol*nrow).reshape(nrow,ncol)
df = pd.DataFrame(matrix)

df.head()
   01234030444140020233200303433411333# count duplicated rows
(df.groupby(df.columns.tolist())
   .size()
   .sort_values(ascending=False))

Output:

0  1  2  3  4
4  2  2  1  1    17
2  2  4  2  3    16
3  2  1  2  2    15
   1  2  4  3    15
                 ..
4  1  3  0  1     1
1  2  3  0  4     1

The most frequent row is the top row of this output. The frequency count is the rightmost column.