Acta academica karviniensia 2011, 11(2):119-128 | DOI: 10.25142/aak.2011.028

EVALUATION OF RANKING SIMILARITY IN ORDINAL RANKING PROBLEMS

Jiří Mazurek
Mgr. Jiří Mazurek, Ph.D., Odborný asistent, Katedra matematických metod v ekonomiii, Slezská univerzita v Opavě, Obchodně podnikatelská fakulta v Karviné, Univerzitní náměstí 1934/3, 733 40 Karviná, e-mail: mazurek@opf.slu.cz

In ordinal ranking problems objects, alternatives, products, services, etc. are ranked by several experts and the goal is to convert a set of (generally different) rankings into the final group consensus ranking. However, this goal depends on a degree of agreement among rankings. With random rankings one cannot expect to get meaningful consensus, but if rankings are "close" and represent agreement between experts, then the final group consensus has much more sense. The aim of this article is to present the evaluation of similarity among rankings, which is based on Kendall's τ and W, Spearman's ρ, Pearson's r and dot product of vectors. Cases without and with ties are discussed as well as a problem of similarity between incomplete rankings (top k lists). Explanations are based on examples.

Keywords: correlation coefficient, decision making, ordinal ranking problem, ranking, ranking similarity
JEL classification: C10, D70

Published: June 30, 2011  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Mazurek J. EVALUATION OF RANKING SIMILARITY IN ORDINAL RANKING PROBLEMS. Acta academica karviniensia. 2011;11(2):119-128. doi: 10.25142/aak.2011.028.
Share...
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. ABDI, H. The Kendall Rank Correlation Coefficient. In: Neil Salkind (Ed.). Encyclopedia of Measurment and Statistics, 2007.
    2. COOK, W. D. Distance-Based and Ad Hoc Consensus Models in Ordinal Preference Ranking. European Journal of Operational Research, Nr. 172 (2006), p. 369-385. Go to original source...
    3. EMOND, E. J., MASON, D. A New Rank Correlation Coefficient with Application to Consensus Ranking Problem. Journal of Multicriteria Decision Analysis, Vol. 11, No 1., 2002, p. 17-28. Go to original source...
    4. FAGIN, R., KUMAR, R., SIVAKUMAR, D. Comparing k-top lists. SIAM, Journal of Discrete Mathematics, Vol. 17, No. 1, p. 134-160.
    5. CHRÁSKA, M. Metody pedagogického výzkumu. Havlíčkův Brod: Grada, 2007. ISBN 978-80-247-1369-4.
    6. KOSMÁK, L., POTŮČEK, R. Metrické prostory. Academia. Praha: 2004. ISBN: 80-200-1202-8.
    7. MELOUN, M., MILITKÝ, J. Kompendium statistického zpracování dat, Metody a řešené úlohy včetně CD, Praha: Academia, 2002. ISBN
    8. STEMLER, S. E. A Comparison of Consensus, Consistency and Measurement Approaches to Estimating Interrater Reliability. Practical Assessment, Research and Evaluation, Vol. 9, Nr. 4, 2004. ISSN: 1531-7714.
    9. TAVANA, M., LoPINTO, F., SMITHER, J. W. A Hybrid Distance-Based Ideal-Seeking Consensus Ranking Model. Journal of Applied Mathematics and Decision Sciences, vol. 2007, 18 pages. Go to original source...

    Return to the content