Skip to content

[FEATURE]: Allow Metrics To Directly Accept Frameworks & Datasets #25

Description

@nmichlo

Is your feature request related to a problem? Please describe.
Current metrics require that you provide a representation function. This is inconvenient and always repeated. Metrics also always require that the dataset be a DisentDataset.

dataset = DisentDataset(data)
# we always need to produce this same function
get_repr = lambda x: module.encode(x.to(module.device))
# to use with metrics
results = metric_mig(dataset, get_repr)

Describe the solution you'd like
Allow the metrics to directly accept the frameworks instead, and automatically wrap datasets with DisentDataset

# directly use the framework and raw data instead!
results = metric_mig(data, module)

Describe alternatives you've considered

  • Detect if isinstance(obj, DisentFramework) then handle everything automatically!
  • Or add a function to frameworks called get_representation
  • Or detect if an object has the property encode and use that instead!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationenhancementNew feature or requestgood first issueGood for newcomersrefactoringInternal changes to the code that do not change the outputs.

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions