Google Cloud announced the general availability of Managed Service for Apache Spark 4.0. This release provides a new runtime foundation for both serverless and managed cluster deployments, designed to support enterprise data pipelines and agentic workloads.
The 4.0 runtime standardizes on Apache Spark 4.2, Java 25, Python 3.13 with Pixi, and Apache Iceberg 1.12. These updates are available for both serverless and managed cluster environments within Google Cloud.
The Google Cloud console has been redesigned with a code-first approach, including an Overview page with samples and public datasets. A unified Jobs view manages all workloads, and an AI agent is included for automatic diagnosis of job failures.
Integration with third-party IDEs like VS Code is enabled through the Data Agent Kit, allowing direct building and running of Apache Spark pipelines. The service now runs on default Gen4 infrastructure, providing faster startups without additional cost, and includes Lightning Engine acceleration for improved price-performance.
Managed Service for Apache Spark operates within Google Cloud's borderless lakehouse, allowing Apache Spark and BigQuery to interact with the same tables. Centralized governance, consistent policies, lineage, and semantic discovery for AI agents are shared through Knowledge Catalog.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
Google Cloud has released Managed Service for Apache Spark 4.0, featuring a new runtime based on Apache Spark 4.2, Java 25, and Python 3.13. This update introduces a redesigned console, AI-powered troubleshooting, third-party IDE integration, and performance improvements for serverless and managed cluster deployments.