reliable metrics for generative models

We have hosted the application reliable metrics for generative models in order to run this application in our online workstations with Wine or directly.

Quick description about reliable metrics for generative models:

Reliable Fidelity and Diversity Metrics for Generative Models (ICML 2020). Devising indicative evaluation metrics for the image generation task remains an open problem. The most widely used metric for measuring the similarity between real and generated images has been the Fr�chet Inception Distance (FID) score. Because it does not differentiate the fidelity and diversity aspects of the generated images, recent papers have introduced variants of precision and recall metrics to diagnose those properties separately. In this paper, we show that even the latest version of the precision and recall (Kynk��nniemi et al., 2019) metrics are not reliable yet. For example, they fail to detect the match between two identical distributions, they are not robust against outliers, and the evaluation hyperparameters are selected arbitrarily. We propose density and coverage metrics that solve the above issues.

Features:

• Precision and recall metrics
• Density and coverage metrics
• Test 10000 real and fake samples form the standard normal distribution N(0,I) in 1000-dimensional Euclidean space
• Generating many fake samples around the real outlier is enough to increase the precision measure
• Set the nearest neighbour k=5
• Precision, recall, density, and coverage estimates

Programming Language: Python.
Categories: