---
title: "Problem Set 4 Answer Key"
output: html_document
---

```{r setup, results = 'hide', message = FALSE, include = FALSE}
options(digits = 4, scipen = 999)
knitr::opts_chunk$set(echo = TRUE, fig.width = 4, fig.height = 4)
library(tidyverse)
library(lmSupport)
library(psych)
library(car)
library(kableExtra)
library(pwr)
options(knitr.kable.NA = '')
```

# 1

```{r, results = 'hide', message = FALSE}
n1 <- read_csv("https://whlevine.hosted.uark.edu/psyc5143/prob.csv")

# a: generating codes
n1 <- n1 %>% 
	mutate(con1 = ifelse(Text == "S", -2/3, 1/3),
				 con2 = case_when(Text == "S" ~ 0,
				 								 Text == "HE" ~ 1/2,
				 								 Text == "LE" ~ -1/2))

model.n1a <- lm(Y ~ con1 + con2, n1)

# b, c: effect sizes & power analyses
n1b <- modelEffectSizes(model.n1a)
n1b.sr2 <- n1b$Effects[2:3,4] # the effect-size estimates are in column 4; the parameters in rows 2 & 3
unbiased.sr2 <- 1 - (1 - n1b.sr2)*(46/45) # n = 48; PA = 3; PC = 2
f2 <- unbiased.sr2/(1 - unbiased.sr2)
sample1 <- ceiling(pwr.f2.test(u = 1, v = NULL, f2 = f2[1], power = .9)$v) + 1
sample2 <- ceiling(pwr.f2.test(u = 1, v = NULL, f2 = f2[2], power = .9)$v) + 1
# the ceiling function rounds up to the nearest whole number

# d: SSE for the model above
SSEd <- n1b$SSE

# e: intercept-only model and its SSE
model.n1e.int.only <- lm(Y ~ 1, n1)
SSEe <- modelEffectSizes(model.n1e.int.only)$SSE

# f: PRE & F
PRE <- (SSEe - SSEd)/SSEe
Fstat <- (PRE/2)/((1 - PRE)/45)
```

a) See the code above. Using a Bonferroni-adjusted $\alpha = \frac{.05}{2} = .025$, neither contrast is significant. Although the two explanation conditions are scoring higher ($M = .47$) than the standard condition ($M = .40$) by $.07$ or so, this advantage is not significant, $t(45) = 1.36, p = .18$. The second constrast shows an advantage of roughly $.12$ for the HE condition over the SE condition, but this too is not significant, $t(45) = 2.02, p = .05$.

b) The $sr^2$ values are `r n1b.sr2`. Unbiased, these values are `r unbiased.sr2`. Converted to $f^2$, these values are `r f2`.

c) For the larger effect size, $n$ = `r sample2` is the sample size needed to get power = .9; for the smaller effect size, $n$ = `r sample1` is needed. To get both to power = .9, we'd need the larger of the two sample sizes.

d) $SSE$ = `r SSEd`

e) $SSE$ = `r SSEe`

f) $PRE$ = $\frac{(`r SSEe` - `r SSEd`)}{`r SSEe`} = `r PRE`$; $F = \frac{`r PRE`/2}{(1 - `r PRE`)/45} = `r Fstat`$

    **NOTE: The SSE displayed in the `modelEffectSizes` table is rounded, but if you store the value in a variable, as I have done above, it will not be rounded!**

```{r}
# g
summary(aov(Y ~ Text, n1))
```

Woohoo! But with 2 $df$ in its numerator, the ANOVA $F$-ratio is an agglomerated answer to two (unspecified) research questions, so it bespeaks muddy thinking. I said what I said.

# 2

```{r, results='hide', warning=FALSE}
n2 <- read.csv("https://whlevine.hosted.uark.edu/psyc5143/reward.csv")

pairwise.t.test(x = n2$errors, g = n2$condition, p.adjust.method = "none") # a: all pairwise comparisons, raw p-values
pairwise.t.test(x = n2$errors, g = n2$condition, p.adjust.method = "b")    # b: Bonferroni-corrected p
pairwise.t.test(x = n2$errors, g = n2$condition, p.adjust.method = "BH")   # c: BH-adjusted p
TukeyHSD(aov(errors ~ condition, n2))                                      # d: Tukey's HSD
```
a) I'll use letter-pairs to indicate which comparisons are significant: AI, AN, FI, FN

b) AI, AN, FI, FN

c) AI, AN, FI, FN

d) AI, AN, FI, FN

e) In this case, there is no difference across procedures for which comparisons are significant. Sometimes data are like that. But the *p*-value differ. From most-powerful (i.e., lowest *p*-values) to least-powerful, here are the rankings:

* Tukey's HSD
* Benjamini-Hochberg
* Bonferroni

Sort of. Because the BH procedure uses a different alpha for each comparison, the *p*-values for it are lower in some cases than the Tukey HSD and higher in other cases.

## BONUS?

Where the Bonferroni (and BH) procedure will potentially be more useful than the Tukey's HSD procedure is if there are non-pairwise comparisons in the set of those executed. Tukey's HSD was developed for only pairwise comparisons, which is a limiting factor in its usefulness. Another limiting factor in the usefulness of the HSD procedure is that it involves comparing *every possible pairwise comparison*, which may not be what is planned. When not all pairwise comparisons are of interest, the Bonferroni & BH procedures will give more power than Tukey's HSD.

One other drawback of the Tukey's HSD procedure is that it is - strictly speaking - a post-hoc procedure, one that can only be done if the full ANOVA is significant. It may not be treated that way, but that's its purpose in life.