Allow configuring cascade deletion for pipelines

[ronaldz-db]

Changes

For pipeline deletion, we offer a cascade parameter that controls whether the pipeline deletion cascades to datasets as well. We are extending support to Terraform, which is a key customer ask.

Based on DABs core feedback, we will name this new Terraform field for pipelines as cascade_on_destroy

We are making a corresponding change in DABs as well: databricks/cli#5846

Tests

• make test run locally
• relevant change in docs/ folder
• covered with integration tests in internal/acceptance
• using Go SDK
• using TF Plugin Framework
• has entry in NEXT_CHANGELOG.md file

[alexott]

I have another PR: #5600 that doesn't expose cascade directly but instead explicitly preserves tables.

[ronaldz-db]

@alexott oh great! There was a bug in the Go SDK that was recently fixed: databricks/databricks-sdk-go@a00222d

We can now merge either PR, but imo we should keep the cascade naming for consistency. Any thoughts?

[alexott]

I'm not sure about cascade. We were bitten many times by using default values when default is changed on the backend...

[ronaldz-db]

Synced offline with Alex - unfortunately we do need to provide a default value here and cannot solely rely on the server default. That said, we do need to be careful here and in DABs for changes to defaults

[github-actions]

Unit tests

If this PR is from a fork, the tests check runs offline against a pre-warmed Go module cache because fork PRs cannot authenticate to the internal Go module proxy.

If this PR changes go.mod or go.sum, the tests check will fail until a maintainer warms the cache for it:

Actions -> Warm Go Cache -> Run workflow -> pr_number = 5860

Re-run the failed check once the cache warming completes.

Integration tests

Integration tests don't run automatically for external contributors; an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

• PR number: 5860
• Commit SHA: e506b62fc758d72e3f2828d7dfdfc05ae96be0ee

Checks will be approved automatically on success.