Paper Reading #20: The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures

The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures

Chi '11

By:
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, James J. Higgins.

• Jacob O. Wobbrock is currently an Associate and Adjunct Associate Professor at the Information School and the CSE departments respectively at the University of Washington.
• Leah Findlater is currently an Undergraduate Research Advisor, also at the University of Washington.
• Darren Gergle is currently an Associate Professor in the school of Communication Studies and EECS at Northwestern University.
• James J. Higgins is a professor of Statistics at Kansas State University.

Summary
Hypothesis
The author hypothesises that current procedures for analysing non-parametric data arising from HCI experiments are inadequate, and proposes an better, simpler and easier to understand procedure to do the job instead.

Methods
Findlater et al. conducted a study in 2009 involving adaptive menus as used by the authors of this paper, and is therefore referenced in this paper. That study had 24 participants who were asked to use the menu and then asked to grade it upon accuracy and interface.

Results
The Findlater study was unable to come to a conclusion because the Friedman Test was insufficient and inadequate for the the purposes of analysing the gathered data. Running the same data through ART yields a far superior result; a legitimate analysable set of data points which support Findlater's comments in the original study. ART was similarly used to analyse data from previous studies that could not at the time be studied. At all attempts ART strongly supported author opinions on the data.

Contents
The ART method used by the system created for this paper essentially "aligns" input data in such away to allow analysis of non-parametric data to be analysed using a simple ANOVA test. ANOVA is infinitely easier to use and understand the results of than the other options out there to analyse non-parametric data. ART has 5 key steps: 1] computing residuals 2] computing estimated effects for all effects 3] compute aligned response Y' 4] assign averaged ranks Y'' 5) perform a full factorial ANOVA on Y''. This allows for an extremely high degree of accuracy thus allowing for more accurate end analysis.

Discussion
While I see the clear use for this system and most certainly see why and how it ties in with HCI. However I'm not a statistician or a mathematician and frankly half the stuff they talked about in the paper flew well over my head. Yes it is fantastic, yes it is highly useful and yes future researchers will thank the authors and creators of this paper and ART for their contribution; however wouldn't this paper be better off being presented in a Math or Statistics journal?