Skip to content

CICD tests - VPC delete failed#724

Open
oldgiova wants to merge 2 commits into
mendersoftware:masterfrom
oldgiova:MC-8739-vpc-delete-failed
Open

CICD tests - VPC delete failed#724
oldgiova wants to merge 2 commits into
mendersoftware:masterfrom
oldgiova:MC-8739-vpc-delete-failed

Conversation

@oldgiova

Copy link
Copy Markdown
Contributor

This PR addresses:

  • Frequent NIC orphaned causes VPC deletion failure and dangling VPCs are causing the next jobs to fail
  • properly retry DNS during LB bootstrap

Co-authored-with: Claude

oldgiova added 2 commits July 16, 2026 11:17
eksctl delete cluster deletes nodegroups then the VPC in one atomic
call. When node instances are killed, the VPC CNI can leak secondary
ENIs in "available" state (known aws-vpc-cni-k8s behavior, worse on
spot). These get discovered only when CloudFormation tries to delete
the VPC afterward, failing the whole cleanup job.

Split nodegroup and cluster deletion into separate eksctl calls and
sweep available ENIs in between, plus once more if the cluster delete
still fails. Extract the sweep into a shared anchor and resolve
VPC_ID once per job instead of re-querying it five times.

Ticket: MC-8739

Signed-off-by: Roberto Giovanardi <roberto.giovanardi@northern.tech>
dig +short leaks "communications error to X#53: timed out" onto
stdout when it can't reach the DNS server, even with +short. The
ingress test treated that string as a valid IP since it only checked
for non-empty output, then fed it straight into curl --resolve,
which failed to parse.

Filter dig's output to a dotted-quad IPv4 pattern before accepting
it, and cap dig to a single 2s attempt so one bad query doesn't stall
the outer retry loop.

Ticket: MC-8739

Signed-off-by: Roberto Giovanardi <roberto.giovanardi@northern.tech>
@oldgiova
oldgiova requested a review from a team as a code owner July 16, 2026 09:40
@oldgiova

Copy link
Copy Markdown
Contributor Author

@oldgiova oldgiova changed the title VPC delete failed CICD tests - VPC delete failed Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant