Check Assumptions Before a Parametric Test

Clean data, flag outliers, assess normality, and transform skewed counts — all in your browser

Welcome back. Everything on this page runs real R in your browser — same setup as the other coding activities. Before you run a parametric test like a t-test, you need to ask whether your data meet the test’s assumptions — especially normality. Today you will walk through the assumption-checking pipeline from lecture: handle missing values, remove impossible entries, flag distributional outliers, assess normality with a histogram and Shapiro-Wilk test, and apply a square-root transformation if needed. You will not run the t-test itself.

The first time you click Run Code, your browser downloads R once (a few seconds). After that it is quick.

1. Load the tidyverse

Coding Assignment 3 starts the same way. Run library(tidyverse) to load dplyr, ggplot2, and friends. A conflicts message afterward is expected, not an error.

NoteHint

From ?library: library(package, ...) attaches a package so its functions become available. Replace package with the name of the collection you need.

Try ?library in R for the full help page.

TipSolution 
library(tidyverse)

On DataHub and in your graded assignment you will type exactly this.

2. Warm-up — what does “normal enough” look like?

Before working with messy counts, practice on a variable that already looks fairly symmetric. Filter to Chinstrap penguins, then make a histogram of body_mass_g with ggplot() and geom_histogram().

NoteHint

From ?filter: keep rows where a condition is TRUE. From ?ggplot: add a layer with +.

Example:

ggplot(mtcars, aes(x = mpg)) +
  geom_histogram()

Filter to one species, then map body mass to the x-axis and add geom_histogram().

TipSolution 
ggplot(chinstrap, aes(x = body_mass_g)) +
  geom_histogram()

The distribution looks roughly bell-shaped — a good starting point for a normality check.

3. Run Shapiro-Wilk on body mass

Run shapiro.test() on the non-missing Chinstrap body-mass values. Save the result as shapiro_mass and print it.

NoteReading Shapiro-Wilk output

The null hypothesis is normality. A small p-value (conventionally below 0.05) means the data look inconsistent with a normal distribution. A large p-value means you do not have evidence against normality — not proof that the data are perfectly normal.

NoteHint

From ?shapiro.test: pass a numeric vector directly. Drop missing values first:

shapiro.test(mtcars$mpg)

Use the body-mass column from chinstrap, skipping NAs.

TipSolution 
shapiro_mass <- shapiro.test(chinstrap$body_mass_g)
shapiro_mass

You should see p ≈ 0.56 — no evidence against normality for body mass.

4. Meet the tick data

Ecologists counted ticks on each Chinstrap penguin in our sample. Run the cell below to build tick_data, then glimpse() it.

The tick counts here are invented for practice and attached to real Chinstrap rows. On Coding Assignment 3 you will read RodentParasiteLoad.csv with read.csv() — same column shape (host_id, sex, count column), different biological story (intestinal worms in deer mice).

NoteHint

From ?glimpse: glimpse(x) prints rows, columns, and types.

glimpse(mtcars)

Apply it to the dataset you just built.

TipSolution 
glimpse(tick_data)

You should see 68 rows. tick_count is <dbl> — some values are missing, and two are negative sign errors.

5. Mean tick count with missing values

Calculate the mean tick_count across all penguins and save it as mean_raw. Use na.rm = TRUE. Print the result and compare it to a reference mean of 13 ticks per penguin.

NoteHint

From ?mean: mean(x, na.rm = FALSE) — set na.rm = TRUE to skip missing values.

mean(mtcars$mpg, na.rm = TRUE)

Apply to tick_data$tick_count.

TipSolution 
mean_raw <- mean(tick_data$tick_count, na.rm = TRUE)
mean_raw

You should see about 12.9 — close to the reference value of 13.

6. Trim impossible values

Step 1 — trim impossible values. Remove rows where tick_count is negative (a sign error). Keep rows where tick_count is missing. Save the result as tick_clean.

Note| means “or”

Inside filter(), the vertical bar | means “or”. The condition is.na(tick_count) | tick_count >= 0 keeps a row if either the count is missing or it is zero or positive.

Writing filter(tick_count >= 0) alone silently drops the missing rows — because NA >= 0 evaluates to NA, not TRUE.

NoteHint

From ?filter: keep rows where conditions are TRUE.

mtcars |>
  filter(is.na(mpg) | mpg >= 0)

Keep missing tick counts or non-negative counts.

TipSolution 
tick_clean <- tick_data |>
  filter(is.na(tick_count) | tick_count >= 0)

66 rows — two negative sign errors removed, three missing values kept.

7. Flag distributional outliers

Step 2 — flag outliers. Using only the non-missing tick counts in tick_clean, calculate the upper fence of the 1.5 × IQR rule: Q3 + 1.5 × IQR. Save it as upper_fence and print it. Do not remove any rows.

NoteHint

From ?quantile: quantile(x, probs, na.rm = FALSE). From ?IQR: IQR(x, na.rm = FALSE).

q3 <- quantile(mtcars$mpg, 0.75, na.rm = TRUE)
fence <- q3 + 1.5 * IQR(mtcars$mpg, na.rm = TRUE)

Use the 75th percentile and IQR of non-missing tick counts.

TipSolution 
upper_fence <- quantile(tick_clean$tick_count, 0.75, na.rm = TRUE) +
  1.5 * IQR(tick_clean$tick_count, na.rm = TRUE)
upper_fence

The upper fence is 27 — three penguins have tick counts above it.

8. Histogram on the raw scale

Step 3 — assess normality. Make a histogram of the non-missing tick counts in tick_clean using ggplot() and geom_histogram(). Save the plot as hist_raw and display it.

NoteHint

From ?ggplot: add a layer with +.

hist_raw <- ggplot(mtcars, aes(x = mpg)) +
  geom_histogram()
hist_raw

Map tick_count to the x-axis and assign the whole plot to hist_raw.

TipSolution 
hist_raw <- ggplot(tick_clean, aes(x = tick_count)) +
  geom_histogram()
hist_raw

The distribution is right-skewed — a long tail toward high tick counts.

9. Shapiro-Wilk on the raw scale

Run shapiro.test() on the non-missing tick counts in tick_clean. Save the result as shapiro_raw and print it.

NoteHint

Pass the tick-count vector directly — Shapiro-Wilk ignores NAs if you subset first, or you can pass the column (NAs are dropped automatically in recent R for this test on vectors with NA… actually shapiro.test doesn’t accept NA - need to drop them).

shapiro.test(mtcars$mpg)

Use non-missing values from tick_clean$tick_count.

TipSolution 
shapiro_raw <- shapiro.test(tick_clean$tick_count)
shapiro_raw

p ≈ 0.004 — strong evidence against normality on the raw scale.

10. Square-root transform

The raw counts are not normally distributed. Apply a square-root transformation: use mutate() to add a column called sqrt_ticks equal to the square root of tick_count. Save the result as tick_transformed and display the first few rows with head().

NoteHint

From ?mutate: mutate(.data, ...) adds or changes columns.

mtcars |>
  mutate(sqrt_mpg = sqrt(mpg))

Add sqrt_ticks = sqrt(tick_count) to tick_clean.

TipSolution 
tick_transformed <- tick_clean |>
  mutate(sqrt_ticks = sqrt(tick_count))
head(tick_transformed)

Missing tick counts stay missing; negative rows were already removed.

11. Histogram on the transformed scale

Make a histogram of sqrt_ticks from tick_transformed, using only non-missing values. Save the plot as hist_sqrt and display it.

NoteHint

Same pattern as the raw histogram — map the transformed column to x.

hist_sqrt <- ggplot(mtcars, aes(x = sqrt(mpg))) +
  geom_histogram()
hist_sqrt
TipSolution 
hist_sqrt <- ggplot(tick_transformed, aes(x = sqrt_ticks)) +
  geom_histogram()
hist_sqrt

The distribution looks much more symmetric after the square-root transform.

12. Shapiro-Wilk on the transformed scale

Run shapiro.test() on the non-missing sqrt_ticks values in tick_transformed. Save the result as shapiro_sqrt and print it.

NoteHint

Same function as before — pass the transformed column.

shapiro_sqrt <- shapiro.test(tick_transformed$sqrt_ticks)
shapiro_sqrt
TipSolution 
shapiro_sqrt <- shapiro.test(tick_transformed$sqrt_ticks)
shapiro_sqrt

p ≈ 0.76 — no evidence against normality after transformation.

13. Make the call

Based on your histogram and Shapiro-Wilk result for the transformed data, are the square-root tick counts normally distributed enough for a parametric test? Assign sqrt_is_normal to TRUE or FALSE (no quotes, all caps).

NoteHint

Look at shapiro_sqrt$p.value. Is it above or below 0.05?

sqrt_is_normal <- TRUE
TipSolution 
sqrt_is_normal <- TRUE

p ≈ 0.76 — no evidence against normality on the transformed scale.

14. Impossible vs. extreme

Three penguins had tick counts above the upper fence (30, 33, 30), yet we kept them in the dataset. Those values are extreme but biologically plausible — heavily parasitized hosts happen in nature. They are not data-entry errors like the negative counts we removed in step 1.

The square-root transformation compressed the right tail of the skewed raw counts, making the distribution much more symmetric and bringing the Shapiro-Wilk test in line with the normality assumption — without throwing away real observations.

On Coding Assignment 3 you will explain this same distinction in prose for worm counts in deer mice.

Try this with a partner. One of you explains why negative counts get removed but high counts stay; the other explains what the square-root transform did to the histogram shape. Swap roles.

15. Recap — verbs you practiced

Function Role
library(tidyverse) Loads dplyr, ggplot2, and related packages
glimpse() Quick look at rows, columns, and types
mean(..., na.rm = TRUE) Average of a numeric column, skipping missing values
filter() Keeps rows matching a condition (is.na(...) \| ... >= 0)
quantile(..., 0.75, na.rm = TRUE) 75th percentile for the IQR fence
IQR(..., na.rm = TRUE) Interquartile range for the 1.5×IQR rule
ggplot() + geom_histogram() Visual check of distribution shape
shapiro.test() Formal test of normality (H₀: data are normal)
mutate(sqrt_ticks = sqrt(...)) Square-root transformation of a skewed count

If you can explain each row in your own words, you have covered the core of Coding Assignment 3.

16. Transfer to Coding Assignment 3

On Canvas you will repeat this pipeline with deer-mouse parasite data from RodentParasiteLoad.csv:

  1. library(tidyverse) and read.csv("RodentParasiteLoad.csv") → parasite_data
  2. mean(..., na.rm = TRUE) → mean_raw, compared to a reference mean of 12 worms
  3. filter(is.na(worm_count) | worm_count >= 0) → parasite_clean
  4. Upper fence with quantile() and IQR() → upper_fence
  5. Raw histogram → hist_raw; raw Shapiro-Wilk → shapiro_raw
  6. mutate(sqrt_worms = sqrt(worm_count)) → parasite_transformed
  7. Transformed histogram → hist_sqrt; transformed Shapiro-Wilk → shapiro_sqrt
  8. Assign sqrt_is_normal and write a short prose answer about retaining outliers

Same functions, different biological story — intestinal nematodes instead of penguin ticks. You already practiced every step here.

Keep playing

Compare two variables side by side: Chinstrap flipper length (roughly normal) vs. bill depth (often skewed). Run shapiro.test() on each and see which would need a transform before a parametric test.

Flipper length usually passes; bill depth often fails. Predict which will have the smaller p-value before you run the second line.