Zachary Estes
Experimental Design and Analysis (ANOVA)
Zachary Estes (PhD in Psychology, Princeton University) is Associate Professor of Marketing at Bocconi University in Milan, Italy. He previously served as Assistant Professor of Psychology at the University of Georgia (USA), and as Associate Professor of Psychology at the University of Warwick (UK). He has served as Associate Editor of Cognitive Science, and on the editorial boards of several other journals. His research on cognition, emotion, and consumer behavior has been awarded research grants from the British Academy (UK) and from the Economic and Social Research Council (UK), and has been published in psychology and marketing journals including Cognitive Psychology, Journal of Business Research, Journal of Consumer Psychology, Journal of Experimental Psychology: General, and Psychological Science. He has taught courses and workshops on Experimental Methods for both academics and practitioners at the University of Warwick (UK), University of Birmingham (UK), University of Innsbruck (Austria), and Bocconi University (Italy).
Contents
Experimental research is becoming more prevalent and prominent across the social sciences. The general aim of this workshop therefore is to provide a sound and thorough understanding of experimental research, from the initial stage of experiment design, through the intermediate stages of conducting an experiment and analysing the data, to the final stage of academic publication. The workshop has two specific and complementary aims. First, participants will learn the fundamentals of designing and conducting experiments. This includes not only discussion of various experimental designs, but also serious consideration of how to improve the validity and reliability of possible conclusions from the experiment. Second, participants will learn statistical approaches for analysing data from experiments, both at a theoretical level and a practical level. The focus will be on a robust and flexible statistical technique that is used in a great variety of research: Analysis of Variance (ANOVA). A major component of the course will be learning how to compute ANOVA using SPSS (a statistics package). The five daily sessions are designed as a step-by-step, practical guide to experimental research, with an emphasis on behavioral experiments. Each session will consist of approximately half-time for general instruction and half-time for hands-on tutorials to discuss or develop workshop participants' own experimental research.
- Session 1: Experimental Design. This session focuses on the fundamentals of experimental methods (e.g., variables, controls, validity, and reliability). We will also discuss how to build a conceptual model (e.g., mediation, moderation) and formulate specific hypotheses to test it via experiment(s).
- Session 2: Conducting Experiments. This session focuses on resources and procedures for conducting experiments. We will consider software for the design and conduct of experiments (Qualtrics), participant sources (e.g., Mechanical Turk), and best practices in experimental procedures.
- Session 3: Statistical Analysis I. This session will introduce the basic principles of statistical analysis of data from experiments. We will discuss and conduct data cleaning (e.g., outliers, transformations), analysis for simple designs (t-test), effect size, and statistical power.
- Session 4: Statistical Analysis II. This session focuses on statistical analysis of more complex experimental designs. We will learn and conduct ANOVA, and discuss interpretation of ANOVA outputs from SPSS.
- Session 5: Reporting Experiments. This session will provide practical guidance on how to report experimental research for academic publication. We will also discuss the academic publication process more generally, including consideration of ethical issues in experimental research.
Bibliography
- Field, A. (2013). Discovering Statistics using IBM SPSS Statistics. Sage, London. [This is the most easily understood and entertaining guide to statistical analysis in SPSS.]
- Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis. Guilford: London. [This is an important text on building and testing conceptual models.]
- Lynch, J. G., et al. (2012). Knowledge creation in consumer research: Multiple routes, multiple criteria. Journal of Consumer Psychology, 22, 473-485.
- Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184-188.
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2001). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Cengage Learning. [This is the most famous experimental design textbook.]
- Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 359-1366.
Prerequisites
Participants should bring along their laptop.