LAION Releases LAION-BVD, a 1.3 Billion URL Open Video Dataset for Multimodal AI Training

Introduction of LAION-BVD

LAION has introduced LAION-BVD (LAION — Big Video Dataset), an open-access video dataset for multimodal learning. The dataset comprises 1.3 billion platform-specific video URLs collected from CommonCrawl, from which 80 million videos, amounting to 10 million hours of content, have been downloaded and processed.

Multimodal Training Capabilities

LAION-BVD is structured to facilitate multimodal pre-training across video, audio, and image modalities. The dataset uses content-aware scene detection to extract clips, for which video and audio captions are synthetically generated. It also extracts scene-changing frames to serve as an alternative source for image-text data.

Performance and Benchmarks

Models trained using LAION-BVD have demonstrated competitive performance on standard video-text and audio-text benchmarks. For instance, ViCLIP models trained on LAION-BVD matched or exceeded InternVid-trained models by up to 2.1% on video-text benchmarks. CLAP models achieved competitive results against other large-scale uncurated audio datasets, and frame-based CLIP models showed strong image-text retrieval performance.

Impact on Research Community

The release of LAION-BVD aims to support open and reproducible multimodal research. This initiative addresses the concentration of large-scale video datasets and trained models within proprietary technology companies, providing an open resource for independent scientific investigation and reproducibility in the academic research community.