When you enroll in this course, you'll also be enrolled in this Specialization.
Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate
There are 6 modules in this course
In the course "Training AI with Humans", you'll delve into the intersection of machine learning and human collaboration, exploring how to enhance AI performance through effective data annotation and crowdsourcing. You’ll gain a comprehensive understanding of machine learning principles and performance metrics while developing practical skills in using platforms like Amazon Mechanical Turk (AMT) for crowdsourced tasks. This unique approach combines theoretical knowledge with hands-on experience, allowing you to implement Inter-Annotator Agreement (IAA) techniques to ensure high-quality annotated data.
By completing this course, you will be well-equipped to design and conduct impactful crowdsourcing studies, improving AI models in real-world applications such as healthcare and research. Whether you're looking to enhance your skills in machine learning, optimize data collection processes, or understand the ethical implications of crowdsourcing, this course offers invaluable insights and tools.
This course explores the intersection of machine learning (ML) and human input through various methodologies and tools. Spanning five modules, you will gain a comprehensive understanding of machine learning techniques, the role of human annotation in ML performance, and the principles and practices of crowdsourcing. The course covers key aspects of designing and implementing crowdsourced studies, calculating inter-annotator agreements, and leveraging crowdsourcing to enhance ML performance. Practical skills will be developed through hands-on activities using platforms like Amazon Mechanical Turk (AMT) and analyzing the data collected from such platforms.
What's included
1 reading1 plugin
Show info about module content
1 reading•Total 10 minutes
Course Overview•10 minutes
1 plugin•Total 4 minutes
Instructor Biography - Dr. Ian McCulloh •4 minutes
Machine Learning
Module 2•6 hours to complete
Module details
In this module, you will be introduced to the fundamentals of machine learning (ML). You will learn the definition and principles of ML, and gain practical skills in calculating and comparing ML performance metrics. You will get a chance to understand how to construct ML classifiers and analyze their effectiveness across different algorithms. This module prepares you to apply ML techniques effectively in various domains, enhancing your ability to solve complex problems using data-driven approaches.
What's included
5 videos2 readings3 assignments1 ungraded lab
Show info about module content
5 videos•Total 85 minutes
Machine Learning•16 minutes
Models•7 minutes
Operationalize Data•20 minutes
Data Normalization•18 minutes
Decision Tree•23 minutes
2 readings•Total 100 minutes
Reading References•50 minutes
Reading References•50 minutes
3 assignments•Total 90 minutes
Machine Learning•60 minutes
Introduction to Machine Learning•15 minutes
Evaluating and Constructing ML Classifiers•15 minutes
1 ungraded lab•Total 60 minutes
Practice Lab - Machine Learning Classifier to Predict in R•60 minutes
Inter-Annotator Agreement (IAA)
Module 3•4 hours to complete
Module details
In this module, you will explore the significance of IAA in Machine Learning (ML) performance. You will learn to calculate IAA manually and implement Krippendorf’s Alpha using the software. You will gain insights into how IAA impacts the reliability of annotated data and its implications for ML model training. This module equips you with essential skills to ensure consistency and reliability in data annotation processes, crucial for effective ML applications.
In this module, you will be introduced to the concept and practical applications of crowdsourcing. You will get a chance to learn how crowdsourcing enhances problem-solving through collective efforts and explore real-world use cases. You will be able to establish your first Amazon Mechanical Turk (AMT) account and understand the platform's capabilities for executing crowdsourced tasks. You will get a chance to delve into crowdsourcing design principles to optimize task efficiency and reliability. This module prepares you to leverage crowdsourcing effectively for diverse applications, from data annotation to research experiments.
What's included
4 videos1 reading3 assignments1 ungraded lab
Show info about module content
4 videos•Total 47 minutes
Crowdsourcing•15 minutes
Amazon Mechanical Turk•12 minutes
Experimentation•14 minutes
Tutorial on setting up your first AMT account•6 minutes
1 reading•Total 20 minutes
Reading References•20 minutes
3 assignments•Total 90 minutes
Crowdsourcing•60 minutes
Introduction to Crowdsourcing•15 minutes
Setting Up and Designing Crowdsourcing Tasks•15 minutes
1 ungraded lab•Total 60 minutes
Practice Lab: Impact of Payment & Complexity on Crowdsourcing Task Efficiency•60 minutes
Platforms
Module 5•5 hours to complete
Module details
This module focuses on leveraging Amazon Mechanical Turk (AMT) for crowdsourcing studies. You will learn to design effective experiments using AMT, ensuring optimal task design and participant engagement. You will be able to collect data through AMT and perform initial analyses to derive meaningful insights from crowdsourced data. You will also understand the implications of AMT addiction and ethical considerations in platform-based research. This module equips you with practical skills to conduct reliable and insightful crowdsourcing studies using AMT.
What's included
2 videos3 readings3 assignments1 ungraded lab
Show info about module content
2 videos•Total 41 minutes
Design of Experiments•28 minutes
AMT Addiction•13 minutes
3 readings•Total 80 minutes
Reading References•20 minutes
Reading References•20 minutes
Self-Reflective Reading: Personal Reflection on Platforms•40 minutes
3 assignments•Total 90 minutes
Platforms•60 minutes
Designing Crowdsourcing Studies with AMT•15 minutes
Collecting and Analyzing AMT Data•15 minutes
1 ungraded lab•Total 60 minutes
Practice Lab: Neuroscientific Explanation of Addiction - Analyzing Stigma, Dangerousness, & Social Distance•60 minutes
Crowdsourcing and Machine Learning
Module 6•5 hours to complete
Module details
This module explores the intersection of crowdsourcing and ML performance enhancement. You will be able to evaluate how Inter-Annotator Agreement (IAA) affects ML model reliability and accuracy. You will explore case studies such as COVID test kit distribution and organ transplant matching to understand real-world applications. You will learn to optimize ML performance through effective crowdsourcing design, ensuring data quality and reliability in machine learning applications.
What's included
4 videos3 readings3 assignments
Show info about module content
4 videos•Total 76 minutes
Data Myths and the R.O.A.D. Framework•25 minutes
Case Study: COVID Test Kit Mailing•12 minutes
Case Study: Organ Transplant•26 minutes
Case Study: COVID Case Count Estimation•13 minutes
3 readings•Total 120 minutes
Reading References•40 minutes
Reading References•40 minutes
Self-Reflective Reading: Crowdsourcing and Machine Learning•40 minutes
3 assignments•Total 90 minutes
Crowdsourcing and Machine Learning•60 minutes
Impact of Inter-Annotator Agreement on ML Performance•15 minutes
Designing Effective Crowdsourcing for ML Improvement•15 minutes
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
The mission of The Johns Hopkins University is to educate its students and cultivate their capacity for life-long learning, to foster independent and original research, and to bring the benefits of discovery to the world.
When will I have access to the lectures and assignments?
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.