The Structure of a Data Science Project

Loading...

From the course by Johns Hopkins University

A Crash Course in Data Science

4670 ratings

Johns Hopkins University

A Crash Course in Data Science

4670 ratings

Course 1 of 5 in the Specialization Executive Data Science

By now you have definitely heard about data science and big data. In this one-week class, we will provide a crash course in what these terms mean and how they play a role in successful organizations. This class is for anyone who wants to learn what all the data science action is about, including those who will eventually need to manage data scientists. The goal is to get you up to speed as quickly as possible on data science without all the fluff. We've designed this course to be as convenient as possible without sacrificing any of the essentials. This is a focused course designed to rapidly get you up to speed on the field of data science. Our goal was to make this as convenient as possible for you without sacrificing any essential content. We've left the technical information aside so that you can focus on managing your team and moving it forward. After completing this course you will know. 1. How to describe the role data science plays in various contexts 2. How statistics, machine learning, and software engineering play a role in data science 3. How to describe the structure of a data science project 4. Know the key terms and tools used by data scientists 5. How to identify a successful and an unsuccessful data science project 3. The role of a data science manager Course cover image by r2hox. Creative Commons BY-SA: https://flic.kr/p/gdMuhT

From the lesson

A Crash Course in Data Science

This one-module course constitutes the first "week" of the Executive Data Science Specialization. This is an intensive introduction to what you need to know about data science itself. You'll learn important terminology and how successful organizations use data science.

The Structure of a Data Science Project6:51

Meet the Instructors

Jeff Leek, PhD
Associate Professor, Biostatistics
Bloomberg School of Public Health
Brian Caffo, PhD
Professor, Biostatistics
Bloomberg School of Public Health
Roger D. Peng, PhD
Associate Professor, Biostatistics
Bloomberg School of Public Health

This lecture discusses the basic structure of a data science project.

A typical data science project will be structured in a few different phases

that I'll talk about in separately in this lecture.

So there's roughly five different phases that we can think about

in a data science project.

The first phase is the most important phase,

and that's the phase where you ask the question and

you specify what is it that you're interested in learning from data.

Now, specifying the question and kind of refining it over time is really

important because it will ultimately guide the data that you obtain and

the type of analysis that you do.

Part of specifying the question is also determining the type of question that

you are gonna be asking.

There are roughly six types of questions that you can

ask going from kind of descriptive, to exploratory, to inferential,

to causal, to prediction, predictive and mechanistic.

And so figuring out what type of question you're asking and

what exactly the question is, is really influential.

And so you should spend a lot of time thinking about this.

Once you've kind of figured out what your question is, but

typically you'll get some data.

Now, either you'll have the data or you'll have to go out and get it somewhere or

maybe someone will provide it to you, but the data will come to you.

And then the next phase will be exploratory data analysis.

So this is the second part, there are two main goals to exploratory data analysis.

The first is you wanna know if the data that you have is suitable for

answering the question that you have.

Then so this will depend on a variety of factors depending on very basic things

like is there enough data, are there too many missing values, things like that.

To more fundamental ones, like are you missing certain variables or

do you need to collect more data to get those variables, etc?

The second goal of exploratory data analysis is

to start to develop a sketch of the solution.

And so if the data are appropriate for

answering your question, you can start using it to kinda sketch

out what the answer might be to get a sense of kinda what it'll look like.

This can be done without any formal modeling or any kind of the statistical

testing of things like that just to get a good picture of what it might be.

The next stage, the third stage, is formal modeling.

So if you're sketch kind of works out,

you've got the right data and it seems appropriate to move on,

the formal modeling phase is the way to kind of specifically write

down what questions you're asking, what parameters you're trying to estimate.

And it also provides a framework for challenging your results.

So just because you've come up with an answer in the exploratory data analysis

phase doesn't mean that it's necessarily going to be the right answer and

you need to be able to challenge your results through a variety of

approaches where the sensitivity analysis are other types of analysis.

So challenging your model and just developing a formal framework is really

important to making sure that you can develop robust evidence for

answering your question.

The next phase is interpretation so once you've done your analysis your

formal modeling you wanna think about how to interpret your results and

there are a variety of things to think about in the interpretation phase

the data science project.

The first is kinda like think about how your results jive with kinda what

you expected to find when you where first asking the question.

And also you wanna think about the kind the totality of the evidence

that you've developed.

At this point, you've probably done many different analysis,

you probably fit in many different models.

And so you have many different bits of information to think about and

part of the interpretation phase is to kind of

assemble all that information to weigh the different pieces of evidence.

So that you know what kind or

which are more reliable, which are more important than others and to get a sense

of the totality of evidence with respect to kind of answering the question.

The last phase is the communication phase.

Any data science project that is successful will wanna

communicate its findings to some sort of audience.

Now that audience may be internal to your organization, it may be external,

it may be to a large audience or even just a few people.

But communicating your findings is an essential part of data science in it

because it informs the data analysis process and

a it translates your findings into action.

So that's the last part which is not a formal part of a data science project

necessarily, but often there will be some decision that needs to be made or

some action that needs to be taken.

And the data science project will have been conducted in support

of making a decision or taking an action.

So that last phase will depend on more than just the results of the data size or

the data analysis, but may require

inputs from many different parts of an organization or from other stakeholders.

So ultimately if the decision is made,

the data analysis that was done will inform that decision and will support and

the evidence that was collected will support that decision.

So these are roughly the five phases of a data science project.

There's the question, there's exploratory data analysis, there's formal modeling,

and there's interpretation, and there's communication.

Now, there is another approach that can be taken,

it's very often taken in data science project.

And that is to really start with the data and

to start with an exploratory data analysis.

So often there will be a data set available, But,

it won't be immediately clear kind of what the data set will be useful for.

So it can be useful to kind of do some exploratory data analysis, to look at

the data, to summarize it a little bit, make some plots, and see what's there.

And to generate some interesting questions based on the data.

So this is sometimes called hypothesis generating because it kind of produces

questions that were already there.

Once you've produced the questions that you wanna ask,

based on your initial kind of exploratory data analysis,

it may be useful to kind of get more data or other data

to kind of do an exploratory data analysis that's specific to your question now.

And then continue with the formal modeling, interpretation and

communication.

One thing that you have to be wary of is to do the exploratory data analysis in one

data set, develop the question, and then go back to the same data set.

And pretend like you hadn't done the exploratory data analysis before and

come at it with say a fresh question.

That goes on to the rest of the stages.

This could often be a recipe for kind of, for bias in your analysis.

Because the results were derived from the same data set.

So it's important to be careful about doing that and to try to obtain other data

when you're using the data to generate the questions in the first place.

So this is the secondary approach to data science that can be very useful and can

often result in many interesting questions that are generated from the data.

Data science projects have a variety of phases and it's important to kind of

understand which phase you're in so that you know kind of how to progress and

how to move forward with any data science project.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play