StudyRePYDinOSt_Tebes_2007 — File 25-0857

Original article Impact of a Positive Youth Development Program in Urban After-School Settings on the Prevention of Adolescent Substance Use Jacob Kraemer Tebes, Ph.D.a,*, Richard Feinn, Ph.D.a, Jeffrey J. Vanderploeg, Ph.D.a, Matthew J. Chinman, Ph.D.b, Jane Shepard, Psy.D.c, Tamika Brabham, M.B.A.c, Maegan Genovese, M.S.c, and Christian Connell, Ph.D.a aDepartment of Psychiatry, Yale University School of Medicine, New Haven, Connecticut bRand Corporation & West Los Angeles VA Healthcare Center, Santa Monica, California cDepartment of Psychiatry, Yale University School of Medicine, New Haven, Connecticut Manuscript received October 18, 2006; manuscript accepted February 1, 2007 See Editorial p. 219 Abstract Purpose: Positive youth development (PYD) emphasizes a strengths-based approach to the pro- motion of positive outcomes for adolescents. After-school programs provide a unique opportunity to implement PYD approaches and to address adolescent risk factors for negative outcomes, such as unsupervised out-of-school time. This study examines the effectiveness of an after-school program delivered in urban settings on the prevention of adolescent substance use. Methods: A total of 304 adolescents participated in the study: 149 in the intervention group and 155 in a control group. A comprehensive PYD intervention that included delivery of an 18-session curriculum previously found to be effective in preventing substance use in school settings was adapted for use in urban after-school settings. The intervention emphasizes adolescents’ use of effective decision-making skills to prevent drug use. Assessments of substance use attitudes and behaviors were conducted at program entry, program completion, and at the 1-year follow-up to program entry. Propensity scores were computed and entered in the analyses to control for any pretest differences between intervention and control groups. Hierarchical linear modeling (HLM) analyses were conducted to assess program effectiveness. Results: The results demonstrate that adolescents receiving the intervention were significantly more likely to view drugs as harmful at program exit, and exhibited significantly lower increases in alcohol, marijuana, other drug use, and any drug use 1 year after beginning the program. Conclusions: A PYD intervention developed for use in an urban after-school setting is effective in preventing adolescent substance use. © 2007 Society for Adolescent Medicine. All rights reserved. Keywords: Youth development; After school; Urban; Substance use; Resilience; Propensity scores; HLM Positive youth development (PYD) is an emerging area of practice and research that emphasizes a strengths-based approach to the promotion of positive outcomes for youth [1]. Such a perspective eschews a view of youth as trouble- some and in need of fixing but, instead, emphasizes their resilience and value to others and to their community [2]. Although the PYD perspective initially declined to focus on risk reduction and prevention because these approaches characterize adolescents as having problems, more recently there have been calls for the integration of PYD and pre- vention science perspectives [3,4]. This integration ac- knowledges that PYD programs should not only attempt to build competencies and promote resilience, but they should also emphasize the reduction of health-compromising be- *Address correspondence to: Jacob Kraemer Tebes, Ph.D., Division of Prevention & Community Research and The Consultation Center, Yale University School of Medicine, 389 Whitney Avenue, New Haven, CT 06511. E-mail address: [email removed] Journal of Adolescent Health 41 (2007) 239–247 1054-139X/07/$ – see front matter © 2007 Society for Adolescent Medicine. All rights reserved. doi:10.1016/j.jadohealth.2007.02.016

haviors [4]. The rapprochement between these two ap- proaches is also possible because of recent shifts in preven- tion science that acknowledge the value of balancing risk reduction with promotive approaches to prevention [5]. Endemic to the PYD perspective is (1) that settings provide essential contexts to promote or impede youth de- velopment; and (2) that parents, practitioners, policy mak- ers, and researchers should attempt to identify settings that promote PYD [2,6–8]. One type of setting that has been suggested as especially well suited for PYD interventions is the after-school program [1,7,9]. About 14 million children and adolescents regularly spend after-school time without adult supervision [10], with approximately 4 million of these being 13- and 14-year-olds [11]. In addition, in one study of adolescents in three cities, one third to two thirds of 15- to 19-year-olds reported being involved in some constructive activity after school [11]. Research has shown that unsupervised out-of-school time is associated with various negative youth outcomes [12,13], such as diminished academic and behavioral functioning [13–15] and involvement in risky behaviors, including criminal behavior and substance use [13,16–18]. Adoles- cent substance use, in particular, has been linked with un- supervised out-of-school time, especially among youth with low levels of parental monitoring [13,15–16,19]. Structured after-school programs for youth have been developed to address the potential risks posed by the lack of adult super- vision [9,20,21], particularly for urban youth [9,22,23]. Re- cent research has also suggested that some after-school programs may reduce substance use among adolescents [9,20,24–26]. However, it remains unclear whether after- school programs using a strengths-based, PYD approach to substance use prevention are also effective, particularly for urban minority adolescents. The present study examines one such program, the Pos- itive Youth Development Collaborative (PYDC), which specifically targets substance use attitudes and behaviors among urban minority adolescents. This program involves the implementation of an evidence-based curriculum em- bedded in a comprehensive after-school program based on PYD principles that is intended to prevent substance use. Methods Participants A total of 304 adolescents participated in the study: 149 in the intervention group and 155 in the control group. The final sample of 149 adolescents in the intervention group represented 91% of those eligible to participate in the in- tervention, and the final sample of 155 adolescents in the control group represented 88% of those eligible to partici- pate in the control group. In the intervention group, adoles- cents were enrolled in one of five after-school programs— two programs serving middle school youth, and three programs serving high school youth. For the comparison group, adolescents were enrolled in four programs serving middle and high school youth. All programs provided ser- vices during the school year (September through June). Programs in each condition took place in neighboring cities in the Northeast about 35 miles apart that were comparable in terms of racial and ethnic composition as well as house- hold per capita income. Characteristics of the overall sample were as follows: the mean age was 14.5 years (SD, 1.6 year) and 53% were male. In all, 75.7% of participants were African-American, 19.7% Hispanic, 3.9% Caucasian, and less than 1% American Indian and Asian, respectively. Table 1 shows the pretest characteristics of the interven- tion and control groups as well as any pretest differences by group. 2 tests were used to determine whether the inter- vention and control groups differed on pretest categorical variables (demographic characteristics, substance use be- havior), and independent-samples t tests were used to ex- amine pretest differences on any continuous measures (sub- stance use attitudes). The intervention group contained a significantly higher percentage of girls (57% vs. 37%); was older (14.9 vs. 14.2 years); had a higher grade level (9.4 vs. 8.5 years); had parents with more education (58.6% at- tended some college vs. 26.9%); were less likely to live with two parents (21.2% vs. 41.7%), and had a significantly higher percentage of adolescents who had ever tried ciga- rettes (84% vs. 60%) (Table 1). Intervention The Positive Youth Development Collaborative (PYDC) is a comprehensive program to promote well-being and prevent substance use among adolescents. The program teaches substance use prevention skills along with partici- pation in health education and cultural heritage activities. The core substance use prevention component of the pro- gram is an 18-session curriculum known as Adolescent Decision-Making for the Positive Youth Development Col- laborative (ADM-PYDC) [27]. This component is an adap- tation of two curricula previously shown to be effective in preventing adolescent substance use in school-based set- tings: the Yale Adolescent Decision-Making Program [28] and the Positive Youth Development Program [29]. The ADM-PYDC curriculum consists of 18 sessions that cover the following topics: (1) program introduction and overview (one session); (2) understanding and coping with stress, and learning stress-reduction strategies (three sessions); (3) learning the steps of effective decision-making, includ- ing: (a) defining the problem, (b) brainstorming alternatives, (c) identifying consequences and risks for each alternative, (d) understanding personal values related to the decision making process, (e) identifying social influences on deci- sion-making such as peer pressure and the media, and how to deal with these when making decisions, (f) learning how 240 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

to obtain additional information if needed to make effective decisions, and (g) making one’s best decision (seven ses- sions); (4) learning essential information about tobacco, alcohol, and other drug use (two sessions); (5) applying the decision-making process to one’s life through identifying positive personal attributes, dealing with job and school stressors, setting positive goals for healthy living, and en- hancing one’s social networks and resources (four sessions); and (6) program close and review (one session). More than one third of the material in the ADM and PYD curricula already overlapped before being combined for the present study (i.e., both included information about stress, tobacco, alcohol, and other drugs, as well as enhancement of social networks and resources). In addition, more than one half of the final curriculum examined was derived from the ADM curriculum that emphasized teaching adolescents effective decision-making skills, particularly in substance use situations. The remainder of the material was drawn from the PYD curriculum that emphasized identifying pos- itive personal attributes and setting goals for healthy living. Another adaptation of the final ADM-PYDC curriculum was its development for use in urban, after-school settings that serve predominantly African-American adolescents. As part of the curriculum redevelopment process, cultural her- itage materials tailored for African-American youth were included from the Aban Aya Youth Project [30], along with cultural materials relevant to Hispanic youth. Middle school and high school versions of the curriculum were also de- veloped to ensure that materials were developmentally ap- propriate for both early and late adolescents. The ADM-PYDC curriculum was part of an overall after-school program that included regular field trips to com- munity agencies, civic organizations, businesses, and schools to promote learning about community service and under- standing one’s cultural heritage. These trips exposed ado- lescents to a variety of after-school experiences and opened up opportunities for them to receive academic support, counseling services, and vocational support services, as well as to participate in intergenerational programming and to attend community theater. Consistent with youth develop- ment principles, adolescents were provided with numerous opportunities to carry out youth-led activities through the program, and to develop positive partnerships with and mentoring by adults who were part of collaborating agen- Table 1 Pretest characteristics by condition and any observed differences Intervention group n 149 Control group n 155 Test statistic p Value Age Mean (SD) 14.9 (1.5) 14.2 (1.6) t (302) 3.67 .001 Grade level Mean (SD) 9.4 (1.4) 8.5 (1.6) t (302) 5.09 .001 Gender Female 57.0% 37.4% 2 (1) 11.75 .001 Male 43.0 62.6 Race/ethnicity African American 76.5% 74.8% 2 (3) 4.57 .206 Asian Am./Pac. Islander .7 .6 Hispanic 16.8 22.6 Native American — — Caucasian 6.0 1.9 Parent education Less than high school 6.9% 19.5% 2 (3) 12.78 .005 High school 34.5 53.7 Some college 31.0 22.0 College degree 27.6 4.9 Living situation Both parents/step-parents 21.2% 41.7% 2 (5) 13.83 .017 Mother 57.6 48.3 Father 6.1 0.0 Grandparents 3.0 6.7 Other 12.1 3.3 Ever drank alcohol No 45.6% 52.9% 2 (1) 1.60 .205 Yes 54.4 47.1 Ever smoked cigarettes No 59.7% 83.9% 2 (1) 21.98 .001 Yes 40.3 16.1 Ever smoked marijuana No 76.5% 76.0% 2 (1) 0.01 .913 Yes 23.5 24.0 241 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

cies and organizations. These organizations formed the “collaborative” that is central to the project’s identity. Finally, all program activities were delivered by commu- nity group leaders who were provided with more than 12 hours of training in the curriculum and in youth manage- ment skills. Group leaders attended bi-weekly group meet- ings in which supervision was provided by members of the research team. Control group participants also participated in rich and varied after-school experiences delivered through after- school settings that provided such activities as academic support, counseling services, and recreational activities. These settings also sponsored occasional field trips to com- munity settings for recreational purposes, and held periodic drug education lectures and group discussions facilitated by after-school counselors and guest speakers. Procedures Procedures for this study were approved by the univer- sity institutional review board governing research concern- ing human subjects. All adolescents in after-school settings in both conditions were invited to participate in the study through informational and consent letters sent home to par- ents. Letters were supplemented with follow-up phone calls to parents to ensure that all consent procedures were under- stood and voluntary. Once parental consent was obtained, individual assent was sought and obtained from adolescents before their study enrollment. Adolescents completed a pretest interview shortly after entering the after-school program but before curriculum delivery (pretest; October/November), after completing the program (post-test; June/July), and 1 year after the initial interview (follow-up; October/November of the following year). Interviews were conducted in private at after-school sites, community settings, or participants’ homes. Each in- terview required about 30–45 minutes to complete, for which the adolescents received a $40 gift card to a local mall. Measures Measures included interview assessments of adolescents’ demographic characteristics, substance use attitudes, and substance use behavior. Demographic characteristics. Demographic characteristics assessed included adolescent gender, age, grade level, race/ ethnicity, living situation, family moves in the last year, and parent educational level using a measure developed through the Center for Substance Abuse and Prevention [31]. Substance use attitudes. Two types of substance use atti- tudes were assessed: 1) risk of harm [31], and 2) drug beliefs [31]. The Risk of Harm scale consisted of a five-item measure derived from the CSAP student survey that mea- sures adolescent perceptions of harm when using drugs. Each item uses a four-point Likert scale ranging from “no risk” to “great risk.” The scale used in the present study was a sum of scores for respondent ratings of the risk of harm for the use of alcohol, tobacco, and marijuana. The internal consistency reliability of this scale in the present study was .67, a lower figure than desired but still acceptable. Drug Beliefs is a four-item measure of negative attitudes toward substance use also taken from CSAP student survey [31]. Once again, each item uses a Likert scale with four choices ranging from “not wrong at all” to “very wrong” in response to various attitudes or beliefs about drug use. The overall drug beliefs scale consisted of summed scores of all items. Internal consistency of this summed scale in the present study was .77. Substance use behavior. Substance use was assessed by having adolescents indicate whether they had used various drugs within the past 30 days [31]. Drugs assessed included: alcohol, marijuana, cocaine/crack, heroin/other opiates, non- prescription methadone, hallucinogens, amphetamines, tran- quilizers, inhalants, and other drugs. This question was also asked for cigarettes, chewing tobacco, snuff, and pipe, which were collapsed into a “tobacco” variable for the analyses. Responses to amphetamines, cocaine, heroin, non- prescription methadone, hallucinogens, tranquilizers, or in- halants were collapsed into an “other drugs” category. Fi- nally, a variable called “any drug use” was created for an affirmative response to the use of any of the above drugs. Data analyses A multilevel regression model using hierarchical linear modeling (HLM) [32] was used because the structure of the data was longitudinal with respondents providing up to three possible observations of their use/nonuse of substances. A Bernouli-linked function was then used to model the propor- tion of substance use at each time point. This modeling ap- proach permits use of all data, even with some time points missing (unlike the usual repeated measures approaches) and properly models the correlated observations within each re- spondent (unlike regular regression). HLM models observa- tions by using separate regression equations for each level. In the present study, the repeated observations within each re- spondent are considered level 1 and the intervention received for each respondent is represented at level 2. Inspection of the outcome data revealed that the change from pretest to exit to follow-up did not fit a linear trend; therefore, indicator variables representing exit and follow-up were entered rather than one representing time. As noted earlier, examination of pretest demographics indicated that the intervention and control groups were significantly different on several variables related to out- come. This was not surprising, given that the study used a quasi-experimental, comparative-outcome design with pre- established groups. Because the above variables were likely to be related to drug use attitudes and behavior [33], pro- 242 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

pensity scores were calculated to control for these pretest differences [34]. For this study, propensity scores were calculated using logistic regression to predict group member- ship from the following variables: previous substance use, gender, race, age, parental education, and living situation [34]. These propensity scores were then entered in the HLM model to adjust for the bias between the nonequivalent groups. Results Attendance was tracked for all adolescents enrolled in the intervention. Adolescents who attended 50 percent or more sessions were identified as receiving a full interven- tion dose. Analyses were completed on this “intention to treat” sample as well as a reduced sample of 18 fewer adolescents who did not attend at least one-half of the sessions. The results revealed no significant differences in outcomes between these two intervention samples; thus, the results from the full sample are reported here. Finally, although attrition for the exit and follow-up assessments was slightly higher for the control group, 2 analyses indicated that the differences in attrition between the two conditions were not significant. Attrition at the exit interview was 22.1% for the intervention group and 30.3% for the control group; at the follow-up interview attrition was 37.6% for the intervention group and 42.3% for the control group. Finally, attrition analyses of the pretest scores of sub- stance attitudes and behaviors for those who stayed in or dropped out of the program that compared the two conditions revealed no significant differences by condition. Substance use attitudes Figure 1 displays the estimated risk of harm score, and Table 2 shows the estimated regression coefficients from the HLM analysis. There was a significant increase from pretest to exit in the perception of risk of harm for the intervention group compared with the control group (11 0.79, t(704) 2.79, p .006). Both groups showed a significant increase in that score at follow-up compared with pretest (20 0.70, t(704) 2.77, p .006), but this did not differ by group (21 0.16, t(704) 0.48, p .63). Figure 1 also displays the trajectory of the estimated drug beliefs score over the span of the study, and Table 2 gives the corresponding regression coefficients. Neither group Risk of Harm 17.0 17.0 16.3 16.4 16.2 17.0 15.0 15.5 16.0 16.5 17.0 17.5 Baseline Exit Follow-Up Time Point Score Control Intervention Drug Beliefs 5.6 5.3 5.8 5.5 5.8 5.5 5 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 Baseline Exit Follow-Up Time Point Score Control Intervention Figure 1. HLM estimated scores for substance use attitudes. Table 2 Results from HLM analysis for substance use attitudes Parameter Coefficient S.E.M. t Sig. Risk of harm Pretest (0) Intercept (00) 16.202 0.261 62.05 .000 Intervention (01) 0.134 0.304 0.44 .659 Propensity (02) 0.155 0.544 0.29 .775 Exit (1) Intercept (10) 0.087 0.203 0.43 .666 Intervention (11) 0.794 0.285 2.79 .006 Follow-up (2) Intercept (20) 0.695 0.251 2.77 .006 Intervention (21) 0.165 0.346 0.48 .634 Drug beliefs Pretest (0) Intercept (00) 4.959 0.229 21.66 .000 Intervention (01) 0.067 0.255 0.26 .795 Propensity (02) 1.187 0.518 2.29 .023 Exit (1) Intercept (10) 0.220 0.163 1.35 .177 Intervention (11) 0.273 0.212 1.29 .199 Follow-up (2) Intercept (20) 0.210 0.207 1.02 .311 Intervention (21) 0.06 0.276 0.22 .829 243 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

demonstrated a significant change in drug beliefs at exit or follow-up compared with pretest; however, as shown in Figure 1, the intervention group demonstrated a nonsignif- icant change of 0.5 units at exit (t(704) 1.29, p .19) relative to the control group. Substance use Figure 2 displays the predicted probabilities of use/non- use for various drugs within the past 30 days at each of the three time points assessed, and Table 3 lists the correspond- ing regression coefficients. The rates of use reported are comparable to those found in the most recent comprehen- sive national survey of adolescents [35]. Because the num- ber of participants who indicated that they used tobacco in the past month was extremely low across the three assess- ments, tobacco use was dropped from the analyses. Alcohol use is shown in the first panel, where there was no significant difference in use from pretest to exit between groups (11 0.16, t(704) 0.36, p .72). However, at follow-up, the change in alcohol use from pretest signifi- cantly differed between groups (21 1.01, t(704) 2.19, p .029). The odds of using alcohol from pretest to follow-up was 0.365 (95% CI 0.15–0.90) for the inter- vention group relative to the control group; that is, the odds of using alcohol was 63% (1 .365) less for the interven- tion group. As can be seen in the second panel of Figure 2, marijuana use significantly decreased at exit (10 1.40, OR 0.25 (0.11–0.55), t(704) 3.43, p .001) and increased at follow-up (20 2.12, OR 8.35 (4.24–16.47), t(704) 6.13, p .001) when compared with pretest. The change in use at exit did not differ by group, but the intervention group showed a decrease in use compared with the control group at follow-up (21 1.73, OR 0.18 (0.08–0.42), t(704) 3.96, p .001). Although the odds of marijuana use at follow-up increased by a factor of 1.45 (e 2.12 –1.73) for the intervention group, the odds of marijuana use increased by a factor of more than 8 (OR 8.353, or e 2.12) for the control group. The third panel of Figure 2 shows the results for using other types of drugs. Once again, there was a significant overall decrease in use at exit compared with pretest (10 0.86, OR 0.42 (0.21–0.85), t(704) 2.42, p .016), which did not differ by group. However, there was a sig- nificant increase in other drug use at follow-up for the control group (20 2.14, OR 8.49 (4.32–16.66), t(704) 6.21, p .001) only. Although the intervention group also increased in other drug use at follow-up, the increase was significantly smaller (21 1.67, OR 0.19 (0.08–0.44), t(704) 3.86, p .001). The last panel of Figure 2 shows the estimated probabil- ities for using any type of substance (alcohol, marijuana, or Alcohol Use Past 30 Days 5.8 7.2 4.0 5.7 12.2 8.4 0 2 4 6 8 10 12 14 Baseline Exit Follow-Up Time Point Percent Usage Control Intervention Other Drug Use Past 30 Days 16.8 12.8 4.8 7.8 8.7 10.8 0 3 6 9 12 15 18 21 Baseline Exit Follow-Up Time Point Percent Usage Control Intervention Any Drug Use Past 30 Days 10.3 19.9 10.4 18.4 19.3 16.1 0 5 10 15 20 25 Baseline Exit Follow-Up Time Point Percent Usage Control Intervention Marijuana Use Past 30 Days 17.4 12.1 3.7 7.9 7.6 10.6 0 4 8 12 16 20 Baseline Exit Follow-Up Time Point Percent Usage Control Intervention Figure 2. HLM estimated proportions for substance use. 244 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

other drugs). Neither group showed a significant change from pretest to exit, but the control group showed a signif- icant fivefold increase in use at follow-up (20 1.65, OR 5.19 (2.69–10.03), t(704) 4.91, p .001); no such increase was observed in the intervention group. Discussion The present study showed that a PYD intervention that included an evidence-based substance use prevention com- ponent adapted for an urban after-school setting was effec- tive in preventing adolescent substance use. Adolescents participating in the intervention were significantly more likely to view drugs as harmful at program exit (about 7 months after enrollment), and demonstrated a significantly reduced incidence of past-30-day use of alcohol, marijuana, or other drugs, as well as any drug use 1 year after program enrollment. Although substance use among program partic- ipants increased slightly over time, these increases were significantly less than those observed for the control group. Such reductions in the progression of substance use among adolescents have been found to protect against later in- creased or escalating use, and thus are an accepted indicator of prevention effectiveness [5,36–39]. This study has several implications for practice and research. First, the study involved a collaboration among Table 3 Results from HLM analysis for substance use Parameter Coefficient S.E.M. t Sig. OR 95% C.I. Alcohol Pretest (0) Intercept (00) 7.366 0.675 10.92 .000 0.001 0.00–0.00 Intervention (01) 0.324 0.588 0.55 .581 1.383 0.44–4.39 Propensity (02) 5.112 1.129 4.53 .581 166.1 18.1–1528 Exit (1) Intercept (10) 0.640 0.384 1.67 .096 1.896 0.89–4.02 Intervention (11) 0.164 0.453 0.36 .717 1.179 0.49–2.87 Follow-up (2) Intercept (20) 0.689 0.387 1.78 .075 1.992 0.93–4.25 Intervention (21) 1.007 0.461 2.19 .029 0.365 0.15–0.90 Marijuana Pretest (0) Intercept (00) 6.340 0.615 10.31 .000 0.002 0.00–0.01 Intervention (01) 0.207 0.597 0.35 .729 1.229 0.38–3.97 Propensity (02) 3.955 1.149 3.44 .001 52.19 5.45–499 Exit (1) Intercept (10) 1.399 0.408 3.43 .001 0.247 0.11–40.55 Intervention (11) 0.565 0.500 1.13 .259 1.759 0.66–4.68 Follow-up (2) Intercept (20) 2.123 0.347 6.13 .000 8.353 4.24–16.47 Intervention (21) 1.727 0.437 3.96 .000 0.178 0.08–0.42 Other drug Pretest (0) Intercept (00) 6.380 0.619 10.30 .000 0.002 0.00–0.01 Intervention (01) 0.251 0.605 0.42 .678 1.286 0.39–4.22 Propensity (02) 3.816 1.163 3.28 .002 45.43 4.62–447 Exit (1) Intercept (10) 0.865 0.357 2.42 .016 0.421 0.21–0.85 Intervention (11) 0.236 0.457 0.52 .605 1.266 0.52–3.10 Follow-up (2) Intercept (20) 2.139 0.344 6.21 .000 8.487 4.32–16.66 Intervention (21) 1.670 0.433 3.86 .000 0.188 0.08–0.44 Any type of drug Pretest (0) Intercept (00) 5.803 0.553 10.50 .000 0.003 0.00–0.01 Intervention (01) 0.306 0.529 0.58 .562 1.358 0.48–3.84 Propensity (02) 4.610 1.010 4.57 .000 100.5 13.8–731 Exit (1) Intercept (10) 0.010 0.335 0.03 .978 0.990 0.51–1.91 Intervention (11) 0.527 0.425 1.24 .216 1.694 0.74–3.90 Follow-up (2) Intercept (20) 1.648 0.336 4.91 .000 5.194 2.69–10.03 Intervention (21) 1.240 0.424 2.92 .004 0.289 0.13–0.67 245 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

several community agencies committed to promoting PYD. Such collaboratives are becoming increasingly common in the youth development field [40], and provide a basis for involving multiple community partners in meaningful roles and relationships with youth. Collaboratives have the po- tential to expand opportunities for PYD and, thus, community- based prevention practice. Although the present study was focused more narrowly on the prevention of adolescent substance use, the PYD framework provided a useful plat- form for delivery of a prevention and risk reduction inter- vention that complemented other after-school activities fo- cused on competence or resilience promotion. Future research should examine how such approaches not only impact problem behaviors, such as substance use, but also how they promote broader competencies and resilience among adolescents. A related implication of the present study is that both risk- and competency-based interventions can be blended to yield an effective PYD intervention. Different after-school experiences provide different developmental opportunities. For example, research has shown that sports activities pro- vide excellent settings for the development of initiative among youth; faith-based activities emphasize identity de- velopment, emotional regulation, and interpersonal devel- opment; and service activities are likely to foster teamwork, positive relationships, and social capital [7]. The processes associated with more comprehensive PYD interventions such as that used in this study remain unknown. Future research should assess youth experiences in such compre- hensive programs and examine whether they are related to intended outcomes. Finally, the present study supports the value of adapting interventions to after-school settings that have been previ- ously found to be effective in other contexts. Out-of-school time offers considerable opportunities for both positive developmental experiences [2,7] and problem behaviors [15,17,20]. Interventions for adolescents that have been rigorously evaluated in other contexts to reduce risky be- haviors such as substance use are appropriate for use in after-school contexts, as long as they are designed to com- ply with setting constraints and are tailored to the develop- mental needs and cultural characteristics of participants. In the present study, a structured, facilitator-led substance use prevention program that was originally developed and im- plemented in middle schools and high schools was delivered in after-school contexts with necessary developmental and cultural adaptations and was adjusted to conform to a more informal program schedule. Adaptations of similar pro- grams hold promise for use in other after-school settings. Study limitations The present study was limited by its quasi-experimental design and the use of self-report data from adolescents. The absence of randomization does not rule out the possibility of a number of internal validity threats, particularly selection, that may have influenced the results. To address this par- tially, however, pretest differences among groups were ac- counted for statistically through the computation of propen- sity scores using pretest demographic and substance use variables. This created a statistical equivalency between groups at pretest so that exit and follow-up differences could be examined with confidence [34]. The high number significant values yielded for the propensity scores in the HLM analyses illustrated the value of controlling for these pretest differences. Another study limitation was the inabil- ity to examine the impact of the program on tobacco use because of the extremely low number of adolescents who reported 30-day tobacco use. However, the positive findings demonstrated for reductions in the use of other substances is promising, and suggests that future research should examine the impact of the program on tobacco use. An additional limitation of the present study was the use of self-report measures from adolescents. Although this limitation is real, the data were collected using a semi-structured interview by research staff with considerable previous experience inter- viewing youth. Although reporting distortions were still possible, they were minimized using this approach. Another limitation of this study was that school- and town-level effects could not be examined in the analyses because they were not crossed in the study design. Future research should examine these issues to determine the differential effective- ness of this intervention by school and town. In conclusion, the results of this study support the find- ings of a recent extensive review of PYD programs that showed that such approaches can be effective in the pre- vention of problem behaviors among adolescents [41], even though the explicit aim of these programs is more often to promote positive behaviors. In the present study, strong and consistent effects were observed in the area of substance use prevention and related attitudes after adolescent participa- tion in a comprehensive after-school program. Acknowledgments This work was supported by grant KD1 SP09280 from the Center for Substance Abuse Prevention. The authors acknowledge Kenneth Darden, Susan Florio, Terry Free- man, Cindy Grabarek, Kaye Harvey, Martin Jackson, Jill Popp, Beverly Richardson, Deborah Stewart, Stephanie West, and the group facilitators and after-school program staff who contributed to this project. References [1] Larson RW. Toward a psychology of positive youth development. Am Psychol 2000;55:170–83. [2] Lerner RM, Almerigi JB, Theokas C, et al. Positive youth develop- ment: a view of the issues. J Early Adolesc 2005;25:10–6. 246 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247

[3] Catalano RF, Hawkins JD, Berglund ML, et al. Prevention science and positive youth development: competitive or cooperative frame- works? J Adolesc Health 2002;31:230–9. [4] Roth JL, Brooks-Gunn J. What exactly is a youth development program? Answers from research and practice. Appl Develop Science 2003;7:94–111. [5] Tebes JK, Kaufman JS, Chinman MJ. Teaching about prevention to mental health professionals. In: Glenwick D, Jason L, eds. Innovative Strategies for Promoting Health and Mental Health Across the Life Span. New York: Springer, 2002:37–60. [6] Eccles J, Gootman J, eds. Community Programs to Promote Youth Development. Washington, DC: National Academy Press, 2002. [7] Larson RW, Hansen DM, Moneta G. Differing profiles of develop- mental experiences across types of organized youth activities. De- velop Psychol 2006;42:849–63. [8] Weissberg RP, O’Brien MU. What works in school-based social and emotional learning programs for positive youth development. Ann Am Acad 2004;591:86–97. [9] Mahoney JL, Larson RW, Eccles JS, eds. Organized Activities as Contexts of Development: Extracurricular Activities, After School and Community Programs. Mahwah, NJ: Lawrence Erlbaum Asso- ciates; 2005. [10] After School Alliance: America after 3pm: a household survey on afterschool in America. Available at: http://www.afterschoolalliance. org. Accessed October 4, 2006. [11] National Institute on Out-of-School Time. Making the case: a fact sheet on children and youth in out-of-school time. Center for Re- search on Women, Wellesley College. Available at: http://www.niost. org. Accessed October 4, 2006. [12] Posner JK, Vandell DL. After-school activities and the development of low-income urban children: a longitudinal study. Develop Psychol 1999;35:868–79. [13] Richardson JL, Radziszewska B, Dent CW, et al. Relationship between after-school care of adolescents and substance use, risk taking, depressed mood, and academic achievement. Pediatrics 1993;92:32–8. [14] Mahoney JL, Cairns BD, Farmer TW. Promoting interpersonal com- petence and educational success through extracurricular activity par- ticipation. J Educ Psychol 2003;95:409–18. [15] Pettit GS, Bates JE, Dodge KA, et al. The impact of after-school peer contact on early adolescent externalizing problems is moderated by parental monitoring, perceived neighborhood safety, and prior adjust- ment. Child Develop 1999;70:768–78. [16] Flannery DJ, Williams LL, Vazsonyi AT. Who are they with and what are they doing? Delinquent behavior, substance use, and early adolescents’ after-school time. Am J Orthopsychiat 1999;69:247–53. [17] Gage JC, Overpeck MD, Nansel TR, et al. Peer activity in the evenings and participation in aggressive and problem behaviors. J Adolesc Health 2005;37:7–14. [18] Mahoney JL. School extracurricular activity participation as a mod- erator in the development of antisocial patterns. Child Develop 2000; 71:502–16. [19] Dwyer KM, Richardson JL, Danley KL, et al. Characteristics of eighth-grade students who initiate self-care in elementary and junior high school. Pediatrics 1990;86:448–54. [20] Gottfredson DC, Gerstenblish SA, Soule DA, et al. Do after-school programs reduce delinquency? Prev Science 2004;5:253–66. [21] Morrison G, Storino MH, Robertson L, et al. The protective function of after-school programming and parent education and support for students at-risk of substance abuse. Eval Progr Plang 2000;23:365–71. [22] Halpern R. The role of after-school programs in the lives of inner-city children: a study of the Urban Youth Network. Child Welfare 1992; 121:215–30. [23] Shann MH. Students’ use of time outside of school: a case for after-school programs for urban middle school youth. Urban Rev 2001;33:339–56. [24] Amuedo-Dorantes C, Mach T, Clapp JD. The impact of schools on juvenile substance initiation and use. Prev Science 2004;5:91–9. [25] LoSciuto L, Freeman MA, Harrington E, et al. An outcome evalua- tion of the Woodrock Youth Development Project. J Adolesc Res 1997;11:116–29. [26] VanderWaal CJ, Powell LM, Terry-McElrath YM, et al. Community and school drug prevention strategy prevalence: differential effects by setting and substance. J Primary Prev 2005;26:299–320. [27] Shepard J, Jackson D, Jones R, et al. Adolescent Decision-Making Curriculum for the Positive Youth Development Collaborative. New Haven, CT: Yale University School of Medicine, 2003. [28] Snow DL, Tebes JK, Ayers TS. Impact of two social cognitive interventions to prevent adolescent substance use: test of an amena- bility to treatment model. J Drug Educ 1997;27:1–17. [29] Caplan M, Weissberg RP, Grober JS, et al. Social competence pro- motion with inner-city and suburban young adolescents: effects on social adjustment and alcohol use. J Consult Clin Psychol 1992;60: 56–63. [30] Flay BR, Graumlich S, Segawa E, et al. Effects of 2 prevention programs on high-risk behaviors among African-American youth. Arch Pediatr Adolesc Med 2004;158:377–84. [31] Center for Substance Abuse Prevention: Student survey. Substance Abuse and Mental Health Services Administration. Rockville, MD: U.S. Government Printing Office, 2001. [32] Raudenbush SW, Bryk AS, Cheong YF, et al. HLM 6: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International, 2004. [33] Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: implications for substance abuse prevention. Psychol Bull 1992;112: 64–105. [34] Luellen JK, Shadish WR, Clark MH. Propensity scores: an introduc- tion and experimental test. Eval Rev 2005;29:530–58. [35] Johnston LD, O’Malley PM, Bachman JG, et al. Demographic sub- group trends for various licit and illicit drugs, 1975–2005. (Monitor- ing the Future Occasional Paper No. 63). Ann Arbor, MI: Institute for Social Research; 2006. Available at: http://monitoringthefuture.org/. Accessed January 17, 2007. [36] Bukoski WJ. A framework for drug abuse prevention research. In: Leukefeld CG, Bukoski WJ, eds. Drug Abuse Prevention Intervention Research: Methodological Issues. NIDA Research Monograph 107. Washington, DC: U.S. Government Printing Office, 1991:7–28. [37] Newcomb MD, Bentler PM. Substance use and abuse among children and teenagers. Am Psychol 1989;44:242–8. [38] Yamaguchi K, Kandel DB. Patterns of drug use from adolescence to young adulthood: III. Predictors of progression. Am J Publ Health 1984;74:673–81. [39] Zapert K, Snow DL, Tebes JK. Patterns of substance use in early through late adolescence. Am J Commun Psychol 2002;30:835–52. [40] Gallagher KM, Stanley A, Shearer D, et al. Implementation of youth development programs: promise and challenges. J Adolesc Health 2005;37:S61–8. [41] Catalano RF, Berglund ML, Ryan JA, et al. Positive youth develop- ment in the United States: research findings on evaluations of positive youth development programs. Prev Treatment 2002;5:1–106. 247 J.K. Tebes et al. / Journal of Adolescent Health 41 (2007) 239–247