You may never be sure whether you have an effective user experience until you have tested it with users. In this course, you’ll learn how to design user-centered experiments, how to run such experiments, and how to analyze data from these experiments in order to evaluate and validate user experiences. You will work through real-world examples of experiments from the fields of UX, IxD, and HCI, understanding issues in experiment design and analysis. You will analyze multiple data sets using recipes given to you in the R statistical programming language -- no prior programming experience is assumed or required, but you will be required to read, understand, and modify code snippets provided to you. By the end of the course, you will be able to knowledgeably design, run, and analyze your own experiments that give statistical weight to your designs.
This course is part of the Interaction Design Specialization
Designing, Running, and Analyzing Experiments
Offered By
About this Course
3.6
394 ratings
Skills you will gain
ExperimentExperimental DesignStatistical ModelR ProgrammingStatistics
Syllabus - What you will learn from this course
Week
11 hour to complete
Basic Experiment Design Concepts
In this module, you will learn basic concepts relevant to the design and analysis of experiments, including mean comparisons, variance, statistical significance, practical significance, sampling, inclusion and exclusion criteria, and informed consent. You’ll also learn to think of an experiment in terms of usability, its participants, apparatus, procedure, and design & analysis. This module covers lecture videos 1-2.
2 videos (Total 19 min), 1 reading, 1 quiz
1 reading
ALL COURSE MATERIALS10m
1 practice exercise
Understanding the Basics16m
Week
22 hours to complete
Tests of Proportions
In this module, you will learn how to analyze user preferences (or other tallies) using tests of proportions. You will also get up and running with R and RStudio. Topics covered include independent and dependent variables, variable types, exploratory data analysis, p-values, asymptotic tests, exact tests, one-sample tests, two-sample tests, Chi-Square test, G-test, Fisher’s exact test, binomial test, multinomial test, post hoc tests, and pairwise comparisons. This module covers lecture videos 3-9.
7 videos (Total 41 min), 2 quizzes
7 videos
04. Getting Started with R and RStudio2m
05. Exploring Data and a First Test of Proportions7m
06. Understanding and Reporting Your First Statistical Test5m
07. Exact Tests, Asymptotic Tests, and the Binomial Test2m
08. More One-Sample Tests of Proportions9m
09. Two-Sample Tests of Proportions11m
2 practice exercises
Understanding Tests of Proportions36m
Doing Tests of Proportions30m
Week
31 hour to complete
The T-Test
In this module, you will learn how to design and analyze a simple website A/B test. Topics include measurement error, independent variables as factors, factor levels, between-subjects factors, within-subjects factors, dependent variables as responses, response types, balanced designs, and how to report a t-test. You will perform your first analysis of variance in the form of an independent-samples t-test. This module covers lecture videos 10-11.
2 videos (Total 25 min), 2 quizzes
2 videos
11. Analyzing a Simple A/B Test with a T-Test10m
2 practice exercises
Understanding Experiment Designs20m
Doing Independent-Samples T-Tests30m
Week
42 hours to complete
Validity in Design and Analysis
In this module, you will learn about how to ensure that your data is valid through the design of experiments, and that your analyses are valid by understanding and testing for their assumptions. Topics include how to achieve experimental control, confounds, ecological validity, the three assumptions of ANOVA, data distributions, residuals, normality, homoscedasticity, parametric versus nonparametric tests, the Shapiro-Wilk test, the Kolmogorov-Smirnov test, Levene’s test, the Brown-Forsythe test, and the Mann-Whitney U test. This module covers lecture videos 12-15.
4 videos (Total 41 min), 2 quizzes
4 videos
13. Data Assumptions and Distributions14m
14. Testing for ANOVA Assumptions9m
15. Mann-Whitney, a Nonparametric T-Test2m
2 practice exercises
Understanding Validity24m
Doing Tests of Assumptions30m
Week
51 hour to complete
One-Factor Between-Subjects Experiments
In this module, you will learn about one-factor between-subjects experiments. The experiment examined will be a between-subjects study of task completion time with various programming tools. You will understand and analyze data from two-level factors and three-level factors using the independent-samples t-test, Mann-Whitney U test, one-way ANOVA, and Kruskal-Wallis test. You will learn how to report an F-test. You will also understand omnibus tests and how they relate to post hoc pairwise comparisons with adjustments for multiple comparisons. This module covers lecture videos 16-18.
3 videos (Total 14 min), 2 quizzes
3 videos
17. Analyzing and Reporting a Oneway ANOVA9m
18. Kruskal-Wallis, a Nonparametric Oneway ANOVA3m
2 practice exercises
Understanding Oneway Designs12m
Doing Oneway ANOVAs30m
Week
62 hours to complete
One-Factor Within-Subjects Experiments
In this module, you will learn about one-factor within-subjects experiments, also known as repeated measures designs. The experiment examined will be a within-subjects study of subjects searching for contacts in a smartphone contacts manager, including the analysis of times, errors, and effort Likert-type scale ratings. You will learn counterbalancing strategies to avoid carryover effects, including full counterbalancing, Latin Squares, and balanced Latin Squares. You will understand and analyze data from two-level factors and three-level factors using the paired-samples t-test, Wilcoxon signed-rank test, oneway repeated measures ANOVA, and Friedman test. This module covers lecture videos 19-23.
5 videos (Total 49 min), 2 quizzes
5 videos
20. Counterbalancing Repeated Measures Factors8m
21. Long-Format and Wide-Format Data Tables3m
22. The Paired T-Test and Wilcoxon Signed-Rank Test14m
23. Analyzing a Repeated Measures ANOVA and Friedman Test15m
2 practice exercises
Understanding Oneway Repeated Measures Designs20m
Doing Oneway Repeated Measures ANOVAs30m
Week
73 hours to complete
Factorial Experiment Designs
In this module, you will learn about experiments with multiple factors and factorial ANOVAs. The experiment examined will be text entry performance on different smartphone keyboards while sitting, standing, and walking. Topics include mixed factorial designs, interaction effects, factorial ANOVAs, and the Aligned Rank Transform as a nonparametric factorial ANOVA. This module covers lecture videos 24-27.
4 videos (Total 40 min), 2 quizzes
4 videos
25. Understanding Interaction Effects5m
26. Analyzing a Factorial ANOVA12m
27. The ART, a Nonparametric Factorial ANOVA14m
2 practice exercises
Understanding Factorial Designs20m
Doing Factorial ANOVAs30m
Week
81 hour to complete
Generalizing the Response
In this module, you will learn about analyses for non-normal or non-numeric responses for between-subjects experiments using Generalized Linear Models (GLM). We will revisit three previous experiments and analyze them using generalized models. Topics include a review of response distributions, nominal logistic regression, ordinal logistic regression, and Poisson regression. This module covers lecture videos 28-29.
2 videos (Total 25 min), 2 quizzes
2 videos
29. Analyzing Three Generalized Linear Models18m
2 practice exercises
Understanding Generalized Linear Models20m
Doing Generalized Linear Models30m
Week
92 hours to complete
The Power of Mixed Effects Models
In this module, you will learn about mixed effects models, specifically Linear Mixed Models (LMM) and Generalized Linear Mixed Models (GLMM). We will revisit our prior experiment on text entry performance on smartphones but this time, keeping every single measurement trial as part of the analysis. The full set of analyses covered in this course will also be reviewed. This module covers lecture videos 30-33.
4 videos (Total 35 min), 2 quizzes
4 videos
31. Analyzing a Linear Mixed Model11m
32. Analyzing a Generalized Linear Mixed Model10m
33. Course in Review2m
2 practice exercises
Understanding Mixed Effects Models24m
Doing Mixed Effects Models30m
3.6
145 Reviews24%
started a new career after completing these courses
21%
got a tangible career benefit from this course
Top Reviews
This was really useful. The course was well structured and provided excellent real-life examples that are easily transferrable to other scenarios. Keep it up!
Have to watch three times to keep up because it is really hard for people without former experiences... Anyway it is worth it
Instructors
Scott Klemmer
ProfessorCognitive Science & Computer Science
Jacob O. Wobbrock
ProfessorThe Information School
About University of California San Diego
UC San Diego is an academic powerhouse and economic engine, recognized as one of the top 10 public universities by U.S. News and World Report. Innovation is central to who we are and what we do. Here, students learn that knowledge isn't just acquired in the classroom—life is their laboratory.
About the Interaction Design Specialization
You will learn how to design technologies that bring people joy, rather than frustration. You'll learn how to generate design ideas, techniques for quickly prototyping them, and how to use prototypes to get feedback from other stakeholders like your teammates, clients, and users. You'll also learn principles of visual design, perception, and cognition that inform effective interaction design.
Frequently Asked Questions
When will I have access to the lectures and assignments?
Once you enroll for a Certificate, you’ll have access to all videos, quizzes, and programming assignments (if applicable). Peer review assignments can only be submitted and reviewed once your session has begun. If you choose to explore the course without purchasing, you may not be able to access certain assignments.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
What is the refund policy?
Is financial aid available?
More questions? Visit the Learner Help Center.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2019 Coursera Inc. All rights reserved.
