Lab Exercise-Multiple Comparisons

11/29/01

This lab is intended to give you experience with running multiple comparison procedures using SPSS, and to contrast some of these procedures. It is also intended to illustrate the kinds of variability that we can expect in even fairly large studies, and the confusion that can result from trying to make sense of data.

Solomon, Secker-Walker, Skelly, and Flynn (1996) in the Journal of Behavioral Medicine studied smoking behavior in pregnant women. They looked at the women's determination to quit smoking while pregnant. They interviewed 349 women at their first pre-natal visit, all of whom were smokers when they became pregnant, and classified them into four groups.

They wanted to look at the subsequent smoking behavior of these subjects over the course of their pregnancy, but one important consideration is how much these women smoked when they became pregnant. If the groups differ on that variable, that might affect the interpretation of the results.

The means and standard deviations of these four groups, in terms of cigarettes/day when they became pregnant, are given below. What can we tentatively conclude about group differences in pre-pregnant smoking behavior?

Notice that this is not really a simple problem. Your sample sizes are grossly unequal, as you might expect in the real world, and you have problems with heterogeneity of variance. We are not going to worry about that at the moment, but it is still worth noting.

We will recreate this data set by randomly sampling from populations with these means and standard deviations. Each of you will do that 5 times, run the appropriate analysis of variance, and subsequent multiple comparison procedures. You will then report the results to me and I will collate them for the entire class.

The SPSS syntax program to do this is attached at the end of this document. You can cut and paste it into SPSS, so you don't have to do a lot of typing.

After you get the syntax written, run the entire program. It will perform 5 analyses, then it will automatically reduce the sample sizes by half, and then run five more analyses. For the first table below, just fill in the 5 F values that you found with the full and half Anovas. The differences that you see relate to the relative power of the two levels of sample size. Next, I want you to fill in the following tables, based on Fisher's LSD procedure (which is basically all possible t tests at a = .05), and then Tukey's test (which is one of the more commonly recommended procedures). In each cell of the table I want to know the number of times (out of 5) that each procedure, and each sample size, found a significant difference between corresponding pair of groups. 

LSD with full sample   1 2 3 4 1         2         3         4

Tukey with full sample   1 2 3 4 1         2         3         4

LSD with half sample   1 2 3 4 1         2         3         4

Tukey with half sample   1 2 3 4 1         2         3         4

Notice that this is a real data set, and this is the kind of problem that each of you can expect to face in the future. This isn't some trumped up example that doesn't apply to anything important. What can you conclude from these data?

Other Procedures

I have not covered other testing procedures in class, though I do discuss them in the book. I would like you to go through the rest of the choices for multiple comparison procedures and apply each of those to your data for r1, which is the first dependent variable. I just want you to get a sense of how to apply them. You do not have to cut and paste the results, because that would be too much paper, but I would like you to right a sentence of two indicating what, if anything, you noticed when you tried the other procedures.

Solomon, L. J., Secker-Walker, R. H., Skelly, J. M., & Flynn, B. S. (1996) Stages of change in smoking during pregnancy in low-income women. Journal of Behavioral Medicine, 19, 333-344.

Last revised: 11/28/2001

SPSS Syntax: