```{r , include=FALSE}
# This is a good place to put libraries required for using the ggplot function
knitr::opts_chunk$set(echo = TRUE)
options(scipen=999)
library(tidyverse)    #Loads the tidyverse suite of packages
library(nycflights13) #Contains Flight Data
library(knitr)        #kable() for HTML Tables
library(kableExtra)   #HTML Table Styling
```

# Introduction

The main purpose of this tutorial is to put together all 5 key functions from **[dplyr](https://dplyr.tidyverse.org/reference/index.html)** and use them to create summary tables and graphs in RMarkdown. The functions and their purposes are listed as follows:

- `filter()` *Selects Observations Based on Values*

- `arrange()` *Sorts Observations Based on Criteria*

- `select()` or `rename()` *Selects, Deselects, Renames, and Reorders Variables*

- `mutate()` or `transmute()` *Creates New Variables Which Were Originally Nonexistant*

- `summarize()` *Aggregates the Data Based on Helpful Summary Statistics*

We will continue to practice our skills using the dataset `flights` by loading the R package `nycflights13`. 

# Part 1: Summarizing the Data

Using the pipe `%>%` to chain functions from `dplyr`, modify the code below to create a dataset named `flight.summary` that contains the following ordered modfications:

1. Starts with the raw dataset `flights` in the package `nycflights13`.

2. Transform delay variables so that they are measured in minutes since midnight, and then create a metric to measure accuracy based on those transformed delay variables.

3. Remove observations with missing `accuracy`.

4. Group data based on combinations of "origin", "dest", and "carrier".

5. Summarize the data using the number of observations, the accuracy mean, the accuracy variance, and the mean distance for each combination of "origin", "dest", and "carrier". 

6. Filters summarized data to remove all combinations of "origin", "dest", and "carrier" with less than or equal to 10 flights throughout 2013.

7. Release the grouping constraint using the function `ungroup()`.

8. Create a variable called "proportion"" that represents the proportion of flights within each combination.

```{r}
flight.summary = 
  
  #1
  flights %>%
          
  #2
  mutate(
    dep_min       = (dep_time%/%100)*60+dep_time%%100,
    sched_dep_min = (sched_dep_time%/%100)*60+sched_dep_time%%100,
    arr_min       = (arr_time%/%100)*60+arr_time%%100,
    sched_arr_min = (sched_arr_time%/%100)*60+sched_arr_time%%100,
    dep_delay_min = dep_min-sched_dep_min,
    arr_delay_min = arr_min-sched_arr_min,
    accuracy      = abs(dep_delay_min) + abs(arr_delay_min)
  ) %>%
  
  #3
  filter(!is.na(accuracy)) %>%
  
  #4
  group_by(origin, dest, carrier) %>%

  #5
  summarize(
    count=n(),
    mean.acc=mean(accuracy, na.rm=T),
    var.acc=var(accuracy, na.rm=T),
    mean.dist=mean(distance, na.rm=T),
    .groups="keep"
  ) %>%
  
  #6
  filter(!(count<=10)) %>%
  
  #7
  ungroup() %>%
  
  #8
  mutate(proportion=count/sum(count))
```


```{r}
flight.summary
```


# Part 2: Building Charts From Summary Data

<center><img src="graph1.png"></center>

The purpose of creating this plot was to investigate if flight accuracy gets worse or better for longer flights. The chart above displays the approximated linear relationship between flight accuracy and flight distance for each the 3 NYC airports. This first image was created using a combination of `geom_point()` and `geom_smooth()`. Recreate this image using `ggplot()` and remember to modify the defaults of `geom_smooth()` to get linear trend lines without standard error regions. 

```{r}
ggplot(data=flight.summary) +
  geom_point(aes(x=mean.dist,y=mean.acc,color=origin))+
  geom_smooth(aes(x=mean.dist,y=mean.acc,color=origin), method='lm',se=F)
```

The first thing we notice is that not many flights are over 3000 miles in distance. Design a new plot similar to the one above that ignores the scenarios where the distance exceeds 3000. This will prevent rare occurrences from effecting our linear trend comparison and present these relationships in a zoomed-in window where the majority of data exists.

```{r}
ggplot(data=filter(flight.summary,mean.dist<3000)) +
  geom_point(aes(x=mean.dist,y=mean.acc,color=origin))+
  geom_smooth(aes(x=mean.dist,y=mean.acc,color=origin), method='lm',se=F)
```


```{r}
Slope1 <- flight.summary %>%
    group_by(origin) %>%
    do({
      mod = lm(mean.acc ~ mean.dist, data = .)
      data.frame(Intercept = coef(mod)[1],
                 Slope = coef(mod)[2])
    })
Slope1
```

```{r}
Slope2 <- filter(flight.summary,mean.dist<3000) %>%
    group_by(origin) %>%
    do({
      mod = lm(mean.acc ~ mean.dist, data = .)
      data.frame(Intercept = coef(mod)[1],
                 Slope = coef(mod)[2])
    })
Slope2
```

Based on your work, think of answers to the following 2 discussion questions: 

1. Which trend line was most effected by the removal of few occasions where the average distance exceeded 3000 miles?

2. Each trend line is doing different things. Which of the three originating airports is doing the best regarding the relationship between flight accuracy and flight distance? 


# Part 3: Build a Nice Table to Be Displayed in RMarkdown

The dataset we created, `flight.summary`, summarizes the raw data but still contains 370 observations. Below I have provided code, that produces a dataset called `flight.summary2` that only contains the top 5 and bottom 5 combinations of "origin", "dest", and "carrier" based on mean accuracy. Closely examine this code and ensure that you understand what is happening here. Set `eval=T` to create `flight.summary2`. 

```{r}
flight.summary2 = 
  flight.summary %>%
  mutate(rank=min_rank(mean.acc)) %>%
  filter(min_rank(mean.acc)<=5 | min_rank(desc(mean.acc))<=5) %>%
  arrange(rank)
```

The `kable()` function in the `knitr` package allows for creating HTML tables. Furthermore, the `kableExtra` package allows you to add beauty to those internet-ready tables. Click **[here](http://haozhu233.github.io/kableExtra/awesome_table_in_html.html)** for a helpful introduction. Once you understand the function of `kable()` and the modifications you can apply, start an R code chunk to place the information required to produce an HTML table. Click **[here](https://rmarkdown.rstudio.com/lesson-7.html)** for additional guidance on the options required to turn a tibble into HTML code using `kable()` and then HTML code into an integrated webpage table.

```{r}
flight.summary2 %>%
  kable() %>%
  kable_styling(bootstrap_options = c("striped", "hover"))
```

# Conclusion

In this class you will be forced to analyze data you probably care very little about. *This is the part in the class where you begin to act like you care.* An important rule in this class is to "love thy data as thyself." Consider the `flights` data from both the view of the consumer and the airline executive. Think of the different questions that may arise on both sides. A consumer, such as myself, might view the data as an opportunity to explain to American Airlines why they are embarassing all Americans. The airline executive might want to know how their carrier compares to other carriers on departure and arrival delays. Once you have the question you want to answer, the next step is using the data to answer your question. 


