DSC365: Introduction to Data Science

class:inverse

<br><br><br>
## DSC365: Intro to Data Science
### Data Wrangling: `dplyr`
#### September 8, 2026

---
### Announcements

- Comments on Lab 1:
  + A *complete answer* consists of code, output, *and a written answer.* 
  + Part of your job is to communicate what the `R` output means

- Lab 2 due tonight
  + **make sure to upload html or pdf, *not* qmd!**

- Start Lab 3 in class on Thursday!
  + Due **next tuesday** 
  
- **Quiz #1**
  + Last 30 Minutes on Thursday
  + Content: Intro to R, Quarto, ggplot2, dplyr -- no coding!
  
- **Mini-Project 1**
  + Due **Thursday September 17, 2026 at 1:59 pm** (ie. *before* class starts)
  + Presentations during class that day
  + I will do an example presentation on Thursday

---
### What is Data Wrangling?

Most **datasets** are made up of *rows* and *columns*. They are  a collection of *values*: these can be *numbers* (quantitative) or character *strings* (qualitative)

**Data Wrangling** can be defined as the process of cleaning, organizing, and transforming  raw data into the desired format for analysts to use for prompt decision making.

[Data Wrangling Flowchart](https://r4ds.had.co.nz/wrangle-intro.html) by Hadley Wickham and Garrett Grolemund

---

### Why do you need this "Data Wrangling" Skill?

- Data wrangling helps to improve data usability as it converts data into a compatible format for the end system.

- It helps to quickly build data flows within an intuitive user interface and easily schedule and automate the data-flow process.

- Integrates various types of information and their sources (like databases, web services, files, etc.)

- Help users to process very large volumes of data easily and easily share data-flow techniques.

[Source](https://www.simplilearn.com/data-wrangling-article#:~:text=Data%20wrangling%20helps%20to%20improve,automate%20the%20data%2Dflow%20process)

---
### What is tidy data?

Real datasets can, and often do, violate the three principles of tidy data in almost every way imaginable!

---
### Data is usually in a spreadsheet format, but

There are different ways of encoding the same information

.pull-left[
#### Option 1

#### Option 2
<img src="images/untidy2.png" alt="Example 2 of a dataset in spreadsheet format. Have four columns for patient and each of the three names of the patients. The rows are for each treatment." width="80%" />
].pull-right[
#### Option 3
<img src="images/tidy-data.png" alt="Example 3 of a dataset in spreadsheet format. Have three columns for patient, treatment, and score. Has six rows, one for each person and treatment combination." width="80%" />
]

---
### dplyr

dplyr is a grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges.

Rules of `dyplr`:

- First argument is always a data frame
- Subsequent arguments say what to do with that data frame
- Always returns a data frame, so you can chain together operations with "pipes"

---
### dplyr uses the Pipe (%>%) Operator

Use the pipe operator to combine dplyr functions in chain, which allows us to perform more complicated data manipulations

In programming, a [pipe](https://en.wikipedia.org/wiki/Pipeline_(Unix)) is a technique for passing information from one process to another.

+ Syntax:  dataframe %>% dplyr_function()
+ read %>% as "then do"
+ f(x) %>% g(y) is equivalent to g(f(x),y)
+ Recently `R` has developed the native pipe operator: `|>` (that's two characters: `|` then `>`)

---
### dplyr Terminology

There are some of the primary dplyr verbs, representing distinct data analysis tasks:

`select()`: Select particular *columns* of a data frame

`filter()`: Select specified *rows* of a data frame, produce subsets

`arrange()`: *Reorder* the rows of a data frame

`mutate()`: *Add new* or *change existing* columns of the data frame (as functions of existing columns)

`summarise()`: Create collapsed summaries of a data frame

`group_by`: Introduce structure to a data frame

---
### `select()`

`select()`: Select particular columns of a data frame

.center[

<img id='imgselect' src='images/select.png' alt='Illustration of how the select function works (from MDSR book)' width='100%'>

<div id='modalselect' class='modal'>
  
  <img class='modal-content' id='imgmodalselect'>
  
  <div id='captionselect' class='modal-caption'></div>
</div>

]

---
### `filter()`

`filter()`: Select specified rows of a data frame, produce subsets

.pull-left[

<img id='imgfilter' src='./images/filter.png' alt='Cartoon of how the filter function works (by Allison Horst)' width='100%'>

<div id='modalfilter' class='modal'>
  
  <img class='modal-content' id='imgmodalfilter'>
  
  <div id='captionfilter' class='modal-caption'></div>
</div>

].pull-right[

<img id='imgfilter2' src='./images/filter2.png' alt='Illustration of how the filter function works with a dataset (from MDSR book)' width='100%'>

<div id='modalfilter2' class='modal'>
  
  <img class='modal-content' id='imgmodalfilter2'>
  
  <div id='captionfilter2' class='modal-caption'></div>
</div>

]

---
### `mutate()`

`mutate()`: Add new or change existing columns of the data frame (as functions of existing columns)

.pull-left[

<img id='imgmutate' src='./images/mutate.png' alt='Cartoon of how the mutate function works (by Allison Horst)' width='100%'>

<div id='modalmutate' class='modal'>
  
  <img class='modal-content' id='imgmodalmutate'>
  
  <div id='captionmutate' class='modal-caption'></div>
</div>

].pull-right[

<img id='imgmutate2' src='./images/mutate2.png' alt='Illustration of how the mutate function works with a dataset (from MDSR book)' width='100%'>

<div id='modalmutate2' class='modal'>
  
  <img class='modal-content' id='imgmodalmutate2'>
  
  <div id='captionmutate2' class='modal-caption'></div>
</div>

]

---
### `arrange()`

`arrange()`: Reorder the rows of a data frame

.center[

<img id='imgarrange' src='./images/arrange.png' alt='Illustration of how the arrange function works with a dataset (from MDSR book)' width='100%'>

<div id='modalarrange' class='modal'>
  
  <img class='modal-content' id='imgmodalarrange'>
  
  <div id='captionarrange' class='modal-caption'></div>
</div>

]

---
### `summarize()`

`summarise()`: Create collapsed summaries of a data frame

.center[

<img id='imgsummarize' src='./images/summarize.png' alt='Illustration of how the summarize function works with a dataset (from MDSR book)' width='100%'>

<div id='modalsummarize' class='modal'>
  
  <img class='modal-content' id='imgmodalsummarize'>
  
  <div id='captionsummarize' class='modal-caption'></div>
</div>

]