Amazon S3 Tables now support all Apache Iceberg V3 data types

Expanded Data Type Support

Amazon S3 Tables now support all data types defined in the Apache Iceberg V3 specification. This includes new types such as variant, nanosecond timestamps, unknown, geometry, and geography. This expansion allows for more precise and native handling of diverse data formats within Iceberg tables stored on Amazon S3.

Upgrading and New Table Creation

Users can create new Apache Iceberg V3 tables directly within Amazon S3 Tables. Additionally, existing V2 tables can be upgraded in place to V3, enabling them to leverage the new features and data types without requiring data migration. S3 Tables continue to provide managed services like automatic compaction and maintenance for these tables.

Addressing V2 Limitations with V3 Features

Apache Iceberg V3 introduces features designed to resolve common issues encountered with V2, particularly as data volumes grow. Deletion vectors replace V2's positional delete files, reducing the overhead and compaction time associated with row-level deletions. For example, deleting 50,000 records now generates a single deletion vector file instead of numerous small delete files.

V3 also adds row lineage, which automatically includes `_row_id` and `_last_updated_sequence_number` to each record. This allows downstream pipelines to efficiently identify changed rows, improving data governance and change data capture processes. Native support for semi-structured and geospatial data types in V3 eliminates the need for workarounds like encoding these as strings or integers, which previously added storage costs, query latency, and pipeline complexity.

Impact on Analytics Workloads

The adoption of Iceberg V3 data types and features in Amazon S3 Tables provides improved performance and cost-effectiveness for managing large analytics datasets. By natively supporting complex data types and optimizing row-level operations, the update reduces the need for custom code and mitigates performance bottlenecks, particularly for compliance-related data deletions and handling semi-structured or geospatial information.