Because discrete data with values restricted to integer sets like (1,2,3,4,5) cannot be normally distributed, students often believe that standard means-based methods like $t$-tests cannot be used with such data. This note revisits the use of means-based and rank-based methods for scores from Likert data in 2 and $k>2$ independent samples. The results show that both type methods have good statistical properties such as Type I and II errors, but means are easier to understand and lead to simple confidence intervals with proper coverage. Thus, there should be no statistical justification for avoiding these methods, although one may object on philosophical grounds to the conversion of ordinal Likert responses to integers, and these objections are also briefly addressed.
➤ Version 1 (2022-01-17) |
Dennis Boos and Judy Chen (2022). Analysis of Likert-Type Data Using Metric Methods. Researchers.One. https://researchers.one/articles/22.01.00002v1
Robert RyleyApril 30th, 2022 at 02:49 pm
I posted a rebuttal to this paper over at Frank Harrell's Data Methods forum, as there have been a number of related discussions on this important topic as it is related to patient reported outcomes in medicine and rehabilitation.
https://discourse.datamethods.org/t/preprint-analysis-of-likert-type-data-using-metric-methods/5536
Suffice it to say, from strict decision theoretic principles and the perspective of research synthesis and meta-analysis, this practice isn't justifiable. The fact that the observed sign can be changed by arbitrary scale transformations implies that no information is communicated by parametric models on ordinal data.
Ryan MartinJanuary 29th, 2022 at 11:37 pm
Thanks for the contribution! While I'm uncomfortable with using a means-based analysis on categorical data like this, for the reasons you mentioned, your point is well taken -- you can get a quick-and-dirty comparison of groups using the simple means-based analysis.
However, your example focus on justifying the claim "the simple methods (means and ranks) generally do fine" (in terms of size and power) even for categorical data. But I'm curious if proper methods designed for Likert-scale data would actually do better and, if so, by how much. It's not a criticism, just a curiosity -- related to the trade-off between statistical performance and method complexity.
© 2018–2025 Researchers.One