This paper discusses an extension of the V-measure (Rosenberg and Hirschberg, 2007), an entropy-based cluster evaluation metric. While the original work focused on evaluating hard clusterings, we introduce the Fuzzy V-measure which can be used on data that is inherently ambiguous. We perform multiple analyses varying the sizes and ambiguity rates and show that while entropy-based measures in general tend to suffer when ambiguity increases, a measure with desirable properties can be derived from these in a straightforward manner.
Jason Utt, Sylvia Springorum, Maximilian Köper, and Sabine Schulte im Walde. 2014. Fuzzy V-Measure - An Evaluation Method for Cluster Analyses of Ambiguous Data. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 581–587, Reykjavik, Iceland. European Language Resources Association (ELRA).
@inproceedings{utt-etal-2014-fuzzy,
title = "Fuzzy {V}-Measure - An Evaluation Method for Cluster Analyses of Ambiguous Data",
author = {Utt, Jason and
Springorum, Sylvia and
K{\"o}per, Maximilian and
Schulte im Walde, Sabine},
editor = "Calzolari, Nicoletta and
Choukri, Khalid and
Declerck, Thierry and
Loftsson, Hrafn and
Maegaard, Bente and
Mariani, Joseph and
Moreno, Asuncion and
Odijk, Jan and
Piperidis, Stelios",
booktitle = "Proceedings of the Ninth International Conference on Language Resources and Evaluation ({LREC}'14)",
month = may,
year = "2014",
address = "Reykjavik, Iceland",
publisher = "European Language Resources Association (ELRA)",
url = "https://aclanthology.org/L14-1639/",
pages = "581--587",
abstract = "This paper discusses an extension of the V-measure (Rosenberg and Hirschberg, 2007), an entropy-based cluster evaluation metric. While the original work focused on evaluating hard clusterings, we introduce the Fuzzy V-measure which can be used on data that is inherently ambiguous. We perform multiple analyses varying the sizes and ambiguity rates and show that while entropy-based measures in general tend to suffer when ambiguity increases, a measure with desirable properties can be derived from these in a straightforward manner."
}
%0 Conference Proceedings
%T Fuzzy V-Measure - An Evaluation Method for Cluster Analyses of Ambiguous Data
%A Utt, Jason
%A Springorum, Sylvia
%A Köper, Maximilian
%A Schulte im Walde, Sabine
%Y Calzolari, Nicoletta
%Y Choukri, Khalid
%Y Declerck, Thierry
%Y Loftsson, Hrafn
%Y Maegaard, Bente
%Y Mariani, Joseph
%Y Moreno, Asuncion
%Y Odijk, Jan
%Y Piperidis, Stelios
%S Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14)
%D 2014
%8 May
%I European Language Resources Association (ELRA)
%C Reykjavik, Iceland
%F utt-etal-2014-fuzzy
%X This paper discusses an extension of the V-measure (Rosenberg and Hirschberg, 2007), an entropy-based cluster evaluation metric. While the original work focused on evaluating hard clusterings, we introduce the Fuzzy V-measure which can be used on data that is inherently ambiguous. We perform multiple analyses varying the sizes and ambiguity rates and show that while entropy-based measures in general tend to suffer when ambiguity increases, a measure with desirable properties can be derived from these in a straightforward manner.
%U https://aclanthology.org/L14-1639/
%P 581-587
Jason Utt, Sylvia Springorum, Maximilian Köper, and Sabine Schulte im Walde. 2014. Fuzzy V-Measure - An Evaluation Method for Cluster Analyses of Ambiguous Data. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 581–587, Reykjavik, Iceland. European Language Resources Association (ELRA).