[Test] Make the retrieval of flexible instance types more resilient to prevent test failures.

[gmarciani]

Description of changes

Make the retrieval of flexible instance types more resilient to prevent test failures.

In particular:

1. Retry the retrieval to be robust against transient failures (networking glithces or throttling)
2. in case of consistent failure, emit a warning log and fall back to the original instance type.
3. sort the list of equivalent instance types so that multiple calls to the function always returns the same result.
4. Add log line in get_similar_instance_types to facilitate troubleshooting.

The definitive solution to reduce even more the risk of failures is to cache the result, which is wip in #7402

Tests

SUCCEEDED with cluster config using the expected list of flex instance types. Also checked that every time the test retrieves the list of flex instance type the list is always the same.

test-suites:
  update:
    test_update.py::test_dynamic_file_systems_update:
      dimensions:
      - instances:
        - c5.xlarge
        oss:
        - ubuntu2404
        regions:
        - eu-west-2
        schedulers:
        - slurm

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.