← All stories
● Covered by 1 source · 1 reportMedium impact1 neutral

Google Cloud Announces General Availability of Managed Service for Apache Spark 4.0

🔄 Updated 1d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Managed Service for Apache Spark 4.0 is generally available.
  • Includes Apache Spark 4.2, Java 25, Python 3.13, and Apache Iceberg 1.12.
  • Features a redesigned console with AI agent troubleshooting.
  • Integrates with third-party IDEs via Data Agent Kit.
  • Offers faster startups and Lightning Engine acceleration.

Managed Service for Apache Spark 4.0 Release

Google Cloud announced the general availability of Managed Service for Apache Spark 4.0. This release provides a new runtime foundation for both serverless and managed cluster deployments, designed to support enterprise data pipelines and agentic workloads.

Key Runtime Updates

The 4.0 runtime standardizes on Apache Spark 4.2, Java 25, Python 3.13 with Pixi, and Apache Iceberg 1.12. These updates are available for both serverless and managed cluster environments within Google Cloud.

Console and Troubleshooting Enhancements

The Google Cloud console has been redesigned with a code-first approach, including an Overview page with samples and public datasets. A unified Jobs view manages all workloads, and an AI agent is included for automatic diagnosis of job failures.

Developer Tooling and Performance

Integration with third-party IDEs like VS Code is enabled through the Data Agent Kit, allowing direct building and running of Apache Spark pipelines. The service now runs on default Gen4 infrastructure, providing faster startups without additional cost, and includes Lightning Engine acceleration for improved price-performance.

Context within Agentic Data Cloud

Managed Service for Apache Spark operates within Google Cloud's borderless lakehouse, allowing Apache Spark and BigQuery to interact with the same tables. Centralized governance, consistent policies, lineage, and semantic discovery for AI agents are shared through Knowledge Catalog.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

Google Cloud has released Managed Service for Apache Spark 4.0, featuring a new runtime based on Apache Spark 4.2, Java 25, and Python 3.13. This update introduces a redesigned console, AI-powered troubleshooting, third-party IDE integration, and performance improvements for serverless and managed cluster deployments.