← All stories
● Covered by 3 sources · 4 reportsMedium impact4 neutral

Google Cloud's Lakehouse enables BigQuery to query Databricks Apache Iceberg tables directly

🔄 Updated 1d ago — new reporting from InfoQ
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • BigQuery queries Databricks Iceberg tables directly.
  • Zero-copy approach eliminates data duplication.
  • Supports GCS, Amazon S3, and Azure Blob Storage.
  • Metadata refreshes every 5 minutes.
  • Centralized security via delegated access.
  • Amazon S3 Tables now offer managed migration of Apache Hive tables to Apache Iceberg.
  • Migration works with Parquet tables in Amazon S3 registered in the AWS Glue Data Catalog.
  • Cloudflare's Basin is a serverless analytics platform.
  • Basin is built on Apache Iceberg and R2 Object Storage.
  • Basin integrates with DuckDB, Snowflake, and Apache Spark.
  • Basin offers no data egress fees.
  • Basin was introduced in preview as "Cloudflare Data Platform" over a year ago.
  • Apache Iceberg V4 redesigns table metadata for scalability and frequent writes.
  • Iceberg V4 introduces an Adaptive Metadata Tree and faster commits.
  • Iceberg V4 supports file and vector data types for AI and search workloads.
  • Daniel Weeks, Iceberg co-creator, presented on V4 at QCon San Francisco.

Direct Querying of Databricks Iceberg Tables

Google Cloud's borderless Lakehouse now allows BigQuery to directly query Apache Iceberg tables managed by Databricks. This integration eliminates the need to copy data between platforms, addressing historical challenges of bridging systems like Databricks and BigQuery for analytics workloads.

How the Integration Works

Databricks-managed Iceberg tables automatically generate Iceberg-compatible metadata alongside native Delta logs. Lakehouse reads this Iceberg metadata directly, applying BigQuery’s vectorized engine to open-format Parquet files stored in Google Cloud Storage (GCS). This zero-copy approach means BigQuery reads the same files Databricks writes, with changes appearing after Lakehouse's scheduled metadata refresh, which occurs as often as every 5 minutes.

Multi-Cloud and Cross-Cloud Capabilities

While GCS is a primary storage layer, Lakehouse extends this federation to external cloud environments such as Amazon S3 and Azure Blob Storage. By combining the open Apache Iceberg REST Catalog with cross-cloud connections, BigQuery can query data across multiple clouds without physically moving it. Each federated catalog is located in a user-chosen Google Cloud region, ideally close to the data, enabling multi-cloud analytics without replication pipelines. Cross-Cloud Interconnect (for AWS) and Lakehouse caching can help reduce remote-cloud egress charges.

Read/Write Interoperability and Security

The Lakehouse framework, built on Apache Iceberg, establishes read and write (R/W) interoperability across different engines, allowing data teams to execute multi-statement SQL commands directly within BigQuery. For security, Lakehouse centralizes access controls through delegated access. Administrators can configure a BigQuery Cloud Resource Connection linked to a service account, which manages IAM rights for GCS buckets, addressing vulnerabilities that arise from managing access across disparate systems.

Updates

🕒 2026-11-20 · new reporting from InfoQ
  • Apache Iceberg V4 redesigns table metadata for scalability and frequent writes.
  • Iceberg V4 introduces an Adaptive Metadata Tree and faster commits.
  • Iceberg V4 supports file and vector data types for AI and search workloads.
  • Daniel Weeks, Iceberg co-creator, presented on V4 at QCon San Francisco.
🕒 2026-11-10 · new reporting from InfoQ
  • Cloudflare's Basin is a serverless analytics platform.
  • Basin is built on Apache Iceberg and R2 Object Storage.
  • Basin integrates with DuckDB, Snowflake, and Apache Spark.
  • Basin offers no data egress fees.
  • Basin was introduced in preview as "Cloudflare Data Platform" over a year ago.
🕒 2026-11-05 · new reporting from AWS News Blog
  • Amazon S3 Tables now offer managed migration of Apache Hive tables to Apache Iceberg.
  • Migration works with Parquet tables in Amazon S3 registered in the AWS Glue Data Catalog.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

How outlets covered it

Apache Iceberg V4 is proposing a significant redesign of its table metadata to improve scalability for very large datasets and support more frequent writes. The update introduces new features like an Adaptive Metadata Tree, faster commits, and support for file and vector data types, expanding its utility for AI and search workloads.

Cloudflare has announced the general availability of Basin, a serverless analytics platform built on Apache Iceberg and R2 Object Storage. Basin supports open data formats, integrates with tools like DuckDB and Snowflake, and offers no data egress fees. The platform combines data ingestion, SQL transformation, and distributed SQL queries, allowing developers to analyze large datasets without managing infrastructure.

Amazon S3 Tables now provide a managed service for migrating Apache Hive tables to Apache Iceberg, specifically for Parquet tables registered in the AWS Glue Data Catalog. This feature automates the migration process, including data validation and type conversions, allowing data lake modernization without manual pipeline construction or compute provisioning.

Google Cloud's borderless Lakehouse now allows BigQuery to directly query Databricks-managed Apache Iceberg tables without data duplication. This integration removes the need for complex data pipelines, reducing storage costs, latency, and fragmented security controls. It enables cross-cloud analytics by reading open-format files in Google Cloud Storage, Amazon S3, or Azure Blob Storage.