● Covered by 1 source · 1 reportMedium impact1 neutral
E-means algorithm achieves 10x faster k-means clustering with lower loss
Key points
- E-means is 10-45x faster on GPU than Flash-KMeans.
- E-means is 8-15x faster on CPU than Faiss.
- Achieves lower test loss than Lloyd's algorithm in 1.2 to 3 epochs.
- Uses mini-batch updates with exponential moving averages and automatic hyperparameter tuning.
New K-Means Algorithm Introduced
A new k-means clustering algorithm, named e-means, has been developed to optimize the k-means problem. This algorithm aims to provide a modern optimization rule for finding k centroids that minimize the mean squared distance between each point and its nearest centroid within a static dataset.
Performance Improvements
E-means demonstrates substantial performance gains over current state-of-the-art implementations. It achieves a 10-45x reduction in wall-clock time on GPUs compared to Flash-KMeans and an 8-15x reduction on CPUs against Faiss. These speedups are observed while also reaching a lower test loss than Lloyd's algorithm, often in 1.2 to 3 epochs compared to Lloyd's 25 epochs.
Algorithmic Innovations
The core of e-means' efficiency comes from its use of mini-batch updates with exponential moving averages. Unlike previous mini-batch approaches, e-means automatically sets batch and step sizes using scaling laws, eliminating the need for hyperparameter tuning. Additional improvements include enhanced initialization, a dead-cluster revival strategy, a cosine-scheduled decay rate, and a finalization stage to further reduce training loss with minimal runtime impact. The algorithm is highly parallelizable and includes specialized kernels to translate its algorithmic advantages into practical speedups.
Comparison with Existing Methods
Traditional k-means algorithms, such as Lloyd's, update centroids by recomputing statistics from scratch using the full dataset, requiring a full pass over the data for each iteration. E-means, by contrast, accumulates statistics through mini-batch updates, which contributes to its speed and efficiency. The centroid update rule in e-means is a central component, differing from methods like Lloyd's and MacQueen's in how sums (Sj) and counts (Nj) of assigned points are updated.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
A new k-means clustering algorithm, e-means, demonstrates significantly faster performance and lower test loss compared to existing methods. E-means achieves a 10-45x wall-clock speedup on GPU and 8-15x on CPU, while also reaching a lower test loss in fewer epochs. This is achieved through mini-batch updates with exponential moving averages, automatic hyperparameter tuning, and improved initialization strategies.