An Alternative Approach to Power

2/7/2002

This material is a more direct approach to power than I took in the class dated 2/5/02. I suggest looking at this material and then going back to the approach taken in that class.

Power

There are several things about power that should be obvious to anyone who has the faintest idea what power is.

    1.  More is better

        Experiments with more power are generally better than experiments with less power. We want to maximize power (within reason) because that means that we have a better chance of finding a difference if one actually exists.

    2.  Bigger differences are easier to see (and thus have more power).

    It should be obvious that if the true effect that we are looking for is large, it will be easier to detect than if it is small.

    3.  Bigger is better

    The more observations we have, all other things equal, the more power we have. In other words, power varies as a function of sample size.

The 2nd and 3rd points above are important, because they tell us something about how to calculate power. The second is particularly important, because it suggests that if we are trying to calculate power, we have to find a way to take the size of the difference(s) into account.

For the calculations we will carry out here, we will blur the distinction between a statistic and a parameter. Our formulae will be written in terms of parameters (e.g. m). But we will take the statistics that some study (in our case, Spilich's) actually found because we want to calculate the power of finding a significant difference if the means that Spilich actually found represent the true parameters in the population(s). This can get confusing when you see me insert a sample mean when the formula calls for a parameter.

Calculation of Power for Spilich Data Given Sample Means as Parameters

I will use the example of Spilich's study of smoking and task as they influence performance. This is a standard 2-way, with independent cells and equal sample sizes.

The following table gives the cell means, the row and column means, and the treatment effects. (The treatment effects for each cell are in parentheses, and these are defined below.)

 

	Nonsmoker	Delayed Smoker	Active Smoker	Mean
Pattern Rec	9.40 (1.948)	9.60 (-0.563)	9.93 (-1.385)	9.64	-8.615
Recall	28.87 (-7.718)	39.93 (0.637)	47.53 (7.081)	38.78	20.518
Driving	9.93 (5.770)	6.80 (-0.074)	2.33 (-5.696)	6.36	-11.904
Mean	16.07	18.78	19.93	18.26
	-2.193	0.519	1.674

The summary table for this factorial design follows. We will want to have it available later, so I stuck it in here.

Because power is going to depend on how large a difference exists between groups, we have to calculate those "effect sizes." Here we define an effect as the difference between the row (or column) mean and the grand mean. So the effect of being in the pattern recognition task is 9.64 - 18.26 = -8.62. (Carrying it to 3 decimals, it is -8.615.) In other words, people in the pattern recognition group will have a mean that is 8.615 points below the overall average.

For the individual interaction effects, the effect for a given cell is the difference between what that cell mean is, and what that cell mean would be predicted to be if we took row and column means, but not interaction, into account. We would predict that the mean of cell₁₁ for example, would be the grand mean plus the effect of being in row₁ plus the effect of being in column₁. That means that we would predict that the mean of cell₁₁ would be .  Because the mean for that cell is actually m_ij the interaction effect is the difference between the actual and predicted mean, which is . That is where all of the numbers in parentheses come from in the above table. They are measures of the degree to which the interaction of row and column variables influences the cell means.

 

Power in a factorial is a direct extension of the way we calculated power with a one-way. In the one-way we calculated

Here we will simply extend that to rows, columns, and interactions.

In what follows m_i. refers to the mean of row_i, averaged across columns. Similarly, m_.j refers to the mean of column_j, averaged across rows.

Cohen uses the letter f where I use F', but that is just a difference in terminology. I have history on my side, whereas Cohen has Cohen's prestige his side.

We could now incorporate the sample sizes and then look that up in tables of the non-central F distrtibution. I cover this in the book. 

We would go to the tables of the non-central f distribution with these values of F.

Instead, I am doing things differently, using Cohen's tables. I will work directly with the F' values.

There is a problem when we come to specifying the sample size for Cohen's tables. We are going to have to enter the table with the relevant F' and the sample sizes. We have already calculated F'. For reasons I won't go into, Cohen defines the adjusted sample size as 

n' = df_error/(df_effect +1) + 1

For our main effects this becomes 126/3 + 1 = 43

and for the interaction this is 126/5 + 1 = 26.2

Using the tables From Cohen , with df_e = 30 and phi-prime rounded, or a program called G*Power, I calculate power as

Effect	n'	Phi-prime (f)	Power
Task	43	1.42	.99
SmkGrp	43	0.16	.32
Interaction	26	0.43	.98

Notice:  I have shown both phi-prime and f. The symbols are used interchangeably, depending on what book you are reading at the time. Cohen was the one who popularized f.

Cohen's Tables

Cohen (1988) Statistical Power Analysis for the Behavioral Sciences (2nd ed.) calculates a statistic called f. This is equivalent to f’.

Cohen's tables are very extensive, and I can not present all of them here. As an example, I have scanned in the one for a test at the .05 level with 2 df for treatments. That appears below. (In this table n is the sample size, or, in the case of factorials, it is the n' that we calculated above. ) These tables have been scanned directly from Cohen's book.

Table 8.3.13

Power of F test at a = .05, u = 2

f

n	F c	.05	.10	.15	.20	.25	.30	.35	.40	.50	.60	.70	.80
2	9.552	05	05	06	06	07	07	08	08	10	12	15	18
3	5.143	05	05	06	07	08	09	10	12	17	22	29	37
4	4.256	05	06	06	08	09	11	14	17	24	33	44	54
5	3.885	05	06	07	09	11	14	17	22	32	44	56	69
6	3.682	05	06	07	10	13	16	21	26	39	53	67	79
7	3.555	05	06	08	11	14	19	25	31	46	62	76	87
8	3.467	05	06	08	12	16	22	28	36	53	69	83	92
9	3.403	05	07	09	13	18	24	32	40	59	75	88	95
10	3.354	05	07	10	14	20	27	35	45	64	81	91	97
11	3.316	05	07	10	15	21	30	39	49	69	85	94	98
12	3.285	06	07	11	16	23	32	42	53	74	88	96	99
13	3.260	06	08	11	17	25	35	46	57	77	91	97	99
14	3.238	06	08	12	18	27	38	49	61	81	93	98
15	3.220	06	08	13	20	29	40	52	64	84	95	99
16	3.205	06	08	13	21	31	43	55	67	86	96	99
17	3.191	06	09	14	22	33	45	58	70	89	97	99
18	3.179	06	09	14	23	34	48	61	73	90	98
19	3.168	06	09	15	24	36	50	64	76	92	99
20	3.159	06	09	16	26	38	52	66	78	93	99
21	3.150	06	09	16	27	40	54	69	80	95	99
22	3.143	06	10	17	28	42	57	71	82	96	99
23	3.136	06	10	18	29	43	59	73	84	96
24	3.130	06	10	18	30	45	61	75	86	97
25	3.124	06	10	19	32	47	63	77	87	98
26	3.119	06	11	20	33	48	65	79	89	98
27	3.114	06	11	20	34	50	66	Bo	go	98
28	3.110	06	11	21	35	52	68	82	91	99
29	3.105	06	12	22	36	53	70	83	92	99
30	3.102	06	12	22	37	55	71	85	93	99
31	3.098	07	12	23	39	56	73	86	94	99
32	3.095	07	12	24	40	58	75	87	94	99
33	3.091	07	13	24	41	59	76	88	95
34	3.088	07	13	25	42	61	77	89	96
35	3.086	07	13	26	43	62	79	go	96
36	3.083	07	'13	26	44	63	so	91	97
37	3.081	07	14	27	45	65	81	92	97
38	3.078	07	14	28	46	66	82	92	97
39	3.076	07	14	28	47	67	83	93	98

 

Table 8.3.13 (continued)

f

n	FC	.05	.10	.15	.20	.25	.30	.35	.40	.50	.60	.70
40	3.074	07	15	29	48	68	84	94	98
42	3.070	07	15	30	51	71	86	95	98
44	3.066	07	16	32	53	73	88	96	99
46	3.063	07	16	33	55	75	89	96	99
48	3.060	08	17	34	57	77	90	97	99
50	3.058	08	18	36	58	79	92	98	99
52	3.055	08	18	37	60	80	93	98
54	3.053	08	19	38	62	82	94	98
56	3.051	08	19	40	64	83	94	99
58	3.049	08	20	41	65	85	95	99
60	3.047	08	21	42	67	86	96	99
64	3.044	08	22	45	70	88	97	99
68	3.041	09	23	47	73	90	98
72	3.039	09	24	49	75	92	98
76	3.036	09	25	52	78	93	99
80	3.034	09	27	54	80	94	99
84	3.032	10	28	56	82	95	99
88	3.031	10	29	58	84	96	99
92	3.029	10	30	60	85	97
96	3.028	10	31	62	87	97
100	3.026	11	32	64	88	98
120	3.021	12	38	73	94
140	3.018	14	44	79	97
160	3.015	15	49	85	98
180	3.013	16	54	89	99
200	3.011	18	59	92
250	3.008	22	69	97
300	3.006	25	78	99
350	3.004	29	84
400	3.003	33	89
450	3.002	36	92
500	3.002	40	95
600	3.001	47	98
700	3.000	53	99
800	3.000	59
900	2.999	65
1000	2.999	70

 

For our example we would look up the 2 main effects with F' = 1.42 and 0.16, respectively, and sample sizes of 45. For the interaction we would enter with F' = .43, but with the sample size only equal to 26.2 (see calculations above). We can't use this specific table for power of the interaction, because that is on 4 df, not 2 df. I will give that answer in a minute.

The yellow sections in the table show the cells that are involved in the interpolation of power for SmokeGrp. For the interaction, Cohen's table would have us interpolate between .96 (for f = .40) and 1.00 for f = .50). The resulting answer is .98 (approx).

Cohen has argued that if you really don’t have any idea what values of µ to expect, or cannot calculate f‘ for some other reason, then you can define

small effect f’ = .10

medium effect f’ = .25

large effect f’ = .50

These correspond to h ² = .01, .06. And .14, respectively.

G*Power

A great source for power is a program called G*Power. It is available at

http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/how_to_use_gpower.html

Even if you don't want the program itself, they have an excellent manual that covers lots of stuff about power.

Mark Gorman gave me another link that may be of interest. It is http://www.math.yorku.ca/SCS/Demos/power/ The only problem is that it is not super user-friendly.

I used G*Power in Tuesday's class, and you can see it in those notes. I will leave it out here, because I am trying to shorten the coverage.

Last revised: 02/06/02