feat: add idle/auto stop TTLs and master/worker instance flexibility …

[sravani-bobbala]

…policies to Dataproc cluster configuration

This pull request adds support for configuring Instance Flexibility Policy and Lifecycle Configuration (specifically idle stop TTL and auto stop time) in the Dataproc ClusterGenerator.

Description

• Instance Flexibility Policy: Added support for specifying instance flexibility policies for both master and worker nodes. This allows users to provide a list of machine types that Dataproc can choose from, improving resource availability and cost optimization.
• Lifecycle Configuration: Added support for idle_stop_ttl and auto_stop_time to automatically stop clusters based on inactivity or a specific schedule.
  These changes allow for more flexible and cost-effective Dataproc cluster management directly from Airflow.
  Unit tests have been added to verify that these configurations are correctly generated.

---

Was generative AI tooling used to co-author this PR?

• Yes — Antigravity
  Generated-by: Antigravity following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.