Thursday, May 19, 2016

What does Project Coach Cost in Comparison to Other Out-of-School Programs?


 Don Siegel


While questions relating to the actual or potential benefits accruing to participants in youth development programs can be difficult to answer at any particular point in time, the question of costs is of immediate interest to those involved in starting, scaling, or funding a program. Although the cost question seems fairly straight forward, it really is a bit more complicated than most people would think. Granting that actual cash expenditures for operations are included in all analyses, things get a bit trickier when trying to estimate the value of in-kind contributions, the use of free or reduced-cost facilities, or how to value the time and expertise of volunteer advisers, tutors and mentors. As well, programs also vary with regard to the breath and depth with which participants engage. Some programs are designed to take large numbers of youth who are provided minimum dosage and supervision, while others take a much smaller number of participants, but who are provided many hours of guidance in a smaller number of activities. This breath-depth variation also plays a role in computing and understanding a program’s cost. 

Nonetheless, having acknowledged the difficulties of computing costs, the Wallace Foundation[1] made an effort to do so, and to provide some benchmarks. Given the “apples and bananas” nature of such analyses, they examined the costs for 111 high quality programs located in six cities (Boston, Charlotte, Chicago, Denver, New York and Seattle). High quality was defined as programs having a high attendance rate (averaging 79% or greater), a high staff/youth ratio (averaging 1:8.3 or better), highly qualified staff (based on educational attainment and training - 30+ hours), and having leadership opportunities for older youth. Included in calculations were: staff salaries and benefits, space and utilities, administrative support, transportation, student stipends, and miscellaneous (snacks, meals, materials, staff training, etc.). Although their sample contained programs varying in size, focus, content, age groupings, location, staffing, ratio of staff to children, facilities, in-kind contributions, management and hours of operation, they did acknowledge that the sample was nonrandom, and that one must be careful when using their data to generalize to the universe of Out-of-School programs.

Cross tabulating programs by whether they were primarily for elementary students (ES), high school students (HS), or middle schools students (MS), they produced the estimates found below. On average, the cost of programs for elementary students (ES) was $6/hr. per slot[2], for high school students (HS) $10/hr., and for programs that had ES, middle school (MS), and HS students it was $9/hr. For programs that were solely focused on academics, the average slot cost was $12/hr. Ranges from the 25th to 75th percentile for each program category are also included.


Wallace (Slot Costs)
Range (25th to 75th percentile)
ES/MS Students Only
$6/hr
$2-$7/hr
HS
$10/hr
$4 - $12/hr
ES,MS,HS
$9/hr
$5 - $12/hr
Academic Focus
$12/hr
$5-$12/hr








Where does Project Coach Fall in this Matrix?

Given these values, I was wondering how Project Coach fared when compared to these estimates. Using a similar algorithm as that used by the Wallace Foundation, I estimated slot costs/hour of Project Coach to be $7.03[3]. This falls on the lower end of the range ($5 - $12) for programs with ES, MS, and HS students, and significantly less than the $9 average computed in the Wallace Database. While Project Coach seems relatively inexpensive compared to other programs, a likely reason for this is that, in contrast to the Wallace Foundation that found staff salaries, administration, and benefits to represent approximately 70% of a programs budget, staff salaries, administration, and benefits in Project Coach represent only 36% of the budget. As well, in the Wallace database space and utilities represented 13% of budgets, while in Project Coach space and utilities are in-kind contributions from Smith College and the Springfield School Department. As an aggregate, we see that collaborating with a college, and the school department in which a program is embedded can offset many expenses.

In computing costs, one can always jump to the conclusion that less expensive also means lesser quality. But, Wallace also compiled a list of nine indicators of quality. Many of these indicators have to do with staff training and evaluation, and the ratio of staff to youth. Using these criteria, Project Coach fared quite well. For example, staff meetings take place at least twice a month”. Project Coach has one or more staff meetings each week. Another guideline is that staff members are referred to training sessions. Project Coach has one or more training sessions each week. Guideline 8 states that programs operate with a low-staff-to-youth ratio (less than or equal to 1:10). In Project Coach the ratio of staff to HS students is 1:3[4], and for ES it is 1:3.5[5].

While it is true that variability across programs makes direct cost comparisons inexact, doing such an analysis provides greater understanding about how a program’s dollars are being spent, and how in-kind contributions can off-set out-of-pocket expenses. This brief analysis of Project Coach shows that it is a relatively lower cost program, that also meets the standards of high quality. Yet, it also shows that a large part of it’s cost effectiveness results from it collaborating with Smith College and the Springfield School Department. 


For those interested in estimating the cost of a program the Wallace Foundation provides a cost estimator calculator at http://www.wallacefoundation.org/cost-of-quality/pages/default.aspx.






[1] http://www.wallacefoundation.org/knowledge-center/after-school/key-research/Documents/The-Cost-of-Quality-of-Out-of-School-Time-Programs.pdf
[2] Slot cost/hour is a standard metric used by researchers who compute program costs. It is the cost of serving one more youth over the course of a program schedule (e.g., having an average daily attendance—ADA—of 101 versus 100). Cost per slot is calculated by dividing the total cost by a program’s ADA.
[3] I used 90% attendance for MS/HS participants and 80% attendance for ES students.
[4] This ratio goes to 1:1 when HS students are tutored by Smith College Students.
[5] This low ratio is a result of MS/HS students being considered staff when tutoring and coaching ES students.

Monday, February 1, 2016

Project Coach and Community Development

Don Siegel


One of the most fascinating social experiments of the last quarter century has been the government’s Moving to Opportunity Program (MTO). In a nutshell, the Department of Housing and Human Development (HUD) ran an experiment during the period 1994-1998 to determine the effects of living in neighborhood’s varying in prosperity on the economic fortunes, and the heath and well-being of poor families. Neighborhoods differing on average income also differ on such things as, quality of housing, social cohesion, family structure, safety, employment opportunities, prevalence of parks and playgrounds, school quality, access to healthful foods and health services, and the presence or absence of such things as gangs, violence, and drugs. Given the array of factors that differentiate wealthy from poor communities, a general hypothesis upon which this experiment was based is that where one lives will largely determine what one becomes.
The way the experiment worked was that from 1994 to 1998 HUD awarded housing vouchers to 4604 low-income families living in high-poverty neighborhoods in five cities. Most heads of households in this experiment were African-American or Hispanic females who had less than a 40% high school graduation rate. As well, a large percentage indicated that they had signed-up for the MTO initiative because they wanted to leave gang and drug infested neighborhoods.

What made this experiment unique was that families were randomly assigned to one of three groups:
·      Group 1 received a housing voucher, which restricted them to relocating to a low poverty neighborhood (poverty rates below 10%).
·      Group 2 received a housing voucher that had no restrictions on where it could be used.
·      Group 3, which could be thought of as a control group, just received housing assistance in the projects where they currently lived.
The ultimate question was what effects, if any, a poor family experienced if they relocated to a neighborhood inhabited by more affluent cohorts.

Results

Surprisingly, after 15 years, no economic differences were found for adults moving to a more affluent community. That is to say, income was approximately the same whether or not an adult moved to a wealthier community or stayed in a poorer community. As well, only marginal, but non-significant differences were found in the physical and mental health of adults who had relocated. Clearly, the lack of effects on these important variables were not what HUD had hoped for when they crafted MTO.

However, these findings, or lack of findings, may not tell the whole story as a somewhat curious measure of self-reported well-being did show that, despite non-significant results in what might be conceived as more substantive measures, adults who moved to less distressed neighborhoods perceived themselves to be happier and better off. While it is always a bit speculative to quantify psychological states in economic terms, a recent study stated that such a psychological profile was typical of people earning an additional $13,000.[i] In a sense then, poorer adults were receiving a real increment in their psychic income from living in a wealthier neighborhood that was not associated with an increment in their real income.

This gets even more interesting as another, more recent study, examined what happened to the children who were part of the various MTO groups by the time they got to their mid-twenties and older.[ii] Results showed a clear dosage effect. Children who were less than 13 (average age 8) when they moved to a wealthier neighborhood, who had had 9.8 years, on average, of exposure, earned $3477 more a year than controls, were more likely to attend college (16% better than controls), attended better colleges, were also more likely to live in lower-poverty neighborhoods as adults, and less likely to be single parents themselves. On the other hand, children who moved to wealthier neighborhoods between the ages of 13-18 (average age 15), were associated with negative effects. By their mid-twenties they actually earned $967 less per year than controls, and fared poorer than or equal to controls on other things such as college attendance, college quality, where they lived as adults, and whether they were single parents themselves as adults. Consequently, it appears that MTO had different effects depending on when a child moved. For younger children, the effects were positive. For older children, they were negative. While explaining such differential effects can only be speculative, arguments have been made that positive effects for younger children were a result of exposure to positive experiences over a longer period during critical developmental years. On the other hand, negative effects for older children have been attributed to the disruption of social networks that are critical during the adolescent years.

What does MOT Mean to Programs such as Project Coach and Community Development?

From a wider perspective, moving to a lower poverty neighborhood, had consequential long lasting effects on younger children that were transferred, in turn, to their own children. More so, the effects do not appear to be mediated by family income, but by a neighborhood’s economic well-being, which, in turn, has an impact on an array of environmental factors. While the data are clear that MTO has a powerful effect on changing younger children’s lives, and the lives of their children, it is not clear on what specific neighborhood factors were responsible for producing different life trajectories for these children. Seemingly, if these could be identified and transported across neighborhoods, we might anticipate better life outcomes for youth growing up poor.

Given that our government is not about to provide housing vouchers for millions of poor families to relocate to more affluent neighborhoods, it seems reasonable to ponder how those things that poor families experienced in wealthier neighborhoods can be replicated in poorer neighborhoods. Clearly, this seems like a reasonable question to ask, but those of us who do community work every day know that it is not easily answered. Communities are complex places, and the term emergent system, seems to characterize such geographical enclaves.  In short, healthy neighborhoods require many ingredients that interact in complex ways. In such a system, the sum of interactions is greater than all of its parts. Furthermore, no one really knows how, or is able, to build the ideal community from a well-defined blueprint over which they have the capacity to control inputs and outputs.

Yet, experiments like MTO appears to be telling us that children are impacted by where they live and grow-up, that effects are cumulative, and track into adulthood. While clarity does not exist on which factors are most critical to the neighborhood effect a simple message may be that many components are part of the mix, and that neighborhoods that wish to be transformed need to work on anything and everything. These should include, but not be limited to such things as: (a) physical aspects, which would comprise the quality and maintenance of housing, parks, libraries, and other public spaces; (b) the quality of schools, (c) the availability of medical services; (d) easy access to supermarkets that carry a wide variety of foods; (e) safety in the streets and on playgrounds; (f) support for businesses that provide goods, services, and employment; and  (g) promoting social cohesion that engages citizens in community decisions about things that affect their lives.

Within this context, programs such as Project Coach play a critical role. While not the universal panacea for all the challenges that distressed neighborhoods face, it is an important ingredient that goes into the stew of factors that can make a community more livable.  Project Coach on face value may be viewed as a sport’s program, but by design sports are simply the medium through which youth are engaged so that they can build: (a) relationships between young children who get opportunities to play various sports and adolescents who serve as their coaches and mentors, (b) relationships among adolescents and their young adult mentors and tutors who help them to meet the daily challenges of growing up in an underserved community, and (c) lifeskills and character skills that help youth to achieve the same sorts of things that the younger children in MTO achieved. Project Coach also provides a means for bringing children’s parents and other relatives together to observe them in action, and for parents to participate together with their children in various activities during monthly community nights.

Surely, Project Coach, or any other program alone, does not contain the full array of benefits that children obtained from MTO, but it is the type of ingredient that can be added into the stew that ultimately makes a neighborhood a better place to live. Clearly, the MTO voucher program makes sense, but for the vast majority of families who will never see a voucher and continue to live in poor neighborhoods, we should continue to offer residents programs, like Project Coach, that make where they live a bit more like the places to which MTO families moved. 




[i] Ludwig, J., Duncan, G. J., Gennetian, L. A., Katz, L. F., Kessler, R. C., Kling, J. R., & Sanbonmatsu, L. (2012). Neighborhood effects on the long-term well-being of low-income adults. Science, 337, 1505-1510.
[ii] Chetty, R., Hendren, N., & Katz, L. F. (2015). The effects of exposure to better neighborhoods on children : New evidence from the moving to opportunity experiment Cambridge, Mass. http://scholar.harvard.edu/files/hendren/files/mto_paper.pdf.

Thursday, November 19, 2015

What is Sport Based Youth Development?


 Don Siegel 

A recent study[1] reports that participating on a sports team can help children, kindergarten to grade 4, to develop healthy dispositions that generalize beyond sports in positive ways, such as by better engaging in classroom activities. The gist of this is that sports involvement, in some way, helps children to develop self-regulation skills, which, in turn, fosters healthy student dispositions. This, along with many other studies over the years, supports the notion that sports have the power to teach kids more than the Xs and Os of a particular game. Unfortunately, such studies are not particularly helpful in providing guidance about the processes within sports that have such an important impact.

In actuality this is another instance of what may be labeled “the mere participation hypothesis”. “The Battle of Waterloo was won on the playing fields of Eton”, or “The way to the boardroom leads through the locker room” are other adages that are representative of the general belief that sports teaches unique lessons to kids that transfer beyond the immediate sports context, and have high value in other contexts such as school or employment. However, for those of us who work with youth in sports, and really wish to maximize the impact that sports have on kid’s lives, we really need to know more about what can be taught and learned in our activities and transferred to other contexts. Mere participation is probably not enough.

A number of years ago sport psychologist Terry Orlick argued that while sport provides a wonderful environment for learning many important life lessons, it also has the potential to be destructive, as he conveyed:

For every positive psychological or social outcome in sports, there are possible negative outcomes. For example, sports can offer a child group membership or group exclusion, acceptance or rejection, positive feedback or negative feedback, a sense of accomplishment or a sense of failure, evidence of self-worth or a lack of evidence of self-worth. Likewise, sports can develop cooperation and a concern for others, but they can also develop intense rivalry and a complete lack of concern for others.

In essence, Orlick is telling us that like most activities in which youth engage, positive or negative outcomes can result. It all depends on what children experience. As with any educational endeavor, positive effects are more likely to ensue in a positive and enriched environment that is overseen by a leader who focuses on and promotes a positive, process-oriented curriculum. But, research has also shown that engaging on teams overseen by irresponsible persons can actually promote moral decay, academic failure and depressed life quality.
In an attempt to provide specifics about the positive attributes that can be taught to youth in sports, and the best strategies for doing so, the Collaborative for Academics, Social, and Emotional Learning (CASEL)[2] provides some excellent guidance. Aggregating the findings from 317 studies involving 324,303 children on the impact of social and emotional learning (SEL) programs, CASEL concluded that programs can and should teach the following:
·  Self-awareness: accurately assessing one’s feelings, interests, values, and strengths; maintaining a well-grounded sense of self-confidence;
·  Self-management: regulating one’s emotions to handle stress, controlling impulses, and persevering in addressing challenges; expressing emotions appropriately; and setting and monitoring progress toward personal and academic goals;
·  Social awareness: being able to take the perspective of and empathize with others; recognizing and appreciating individual and group similarities and differences; and recognizing and making best use of family, school, and community resources;
·  Relationship skills: establishing and maintaining healthy and rewarding relation- ships based on cooperation; resisting inappropriate social pressure; preventing, managing, and resolving interpersonal conflict; and seeking help when needed; and
·  Responsible decision making: making decisions based on consideration of ethical standards, safety concerns, appropriate social norms, respect for others, and likely consequences of various actions;
As the report conveys:
Students who appraise themselves and their abilities realistically (self-awareness), regulate their feelings and behaviors appropriately (self-management), interpret social cues accurately (social awareness), resolve interpersonal conflicts effectively (relationship skills), and make good decisions about daily challenges (responsible decision making) are headed on a pathway toward success in school and later life.
Furthermore, CASEL concluded that the most effective youth SEL programs developed these assets when they followed a SAFE pedagogy. That is, they had a curriculum that was:
·  Sequenced: Does the program apply a planned set of activities to develop skills sequentially in a step-by-step fashion?
·  Active: Does the program use active forms of learning such as role-plays and behavioral rehearsal with feedback?
·  Focused: Does the program devote sufficient time exclusively to developing social and emotional skills?
·   Explicit: Does the program target specific social and emotional skills?
What we learn from the CASEL Report is that rather than simply rolling the dice and hoping that something good will come out of kids participating on a sports team, program designers and coaches need to be deliberate about what they wish to teach their players. If they intend to go beyond the Xs and Os and develop their players’ self-awareness, self-management, social-awareness, relationship skills, or capacity to make responsible decisions, then they need to identify and deploy activities that help kids acquire requisite knowledge and develop associated skills. As well, just as coaches teach progressions in technical and tactical skills, they need to be planful about how to teach SEL attributes. For example, if they intend to teach self-management or relationship skills, then they need to be focused and explicit about doing so, and craft progressions that make sense. Surely, the emotional highs and lows occurring within sports are fertile ground for teaching youth techniques to manage emotions better. As well, teaching kids how to be supportive and effective team members, and how to interact in a civil manner with adversaries provide many opportunities for teaching relationship skills. With so many situations and interactions that occur during sports, one would think that it has the power to provide a plethora of teachable moments during which lessons can be repetitively reinforced in meaningful ways. However, as CASEL suggests, there needs to be a degree of explicitness and deliberateness to such teaching if SEL is to occur.

In summary, this is what sport based youth development is all about. Identifying a set of SEL attributes and teaching them in a coherent fashion within the context of sports. As well, if we expect such learning to generalize beyond sports, we need to be explicit about how this can be done.
A wonderful example of this was revealed a few weeks ago by one of our Project Coach 3rd graders who had been having problems fighting with teammates and classmates in school. He was proud to tell our Project Coach director that his teacher had just named him student of the week, as he had been able to stay out of trouble and also get A’s in all of his work. When asked how this came to be he said that his coach had taught players that when they became upset about something, they should take a few deep breaths and then count to 100 to calm themselves. He said that he started to do this in school, and it helped to keep him out of trouble when he became agitated about something. It also helped him to refocus and concentrate on his work. These are the sorts of connections that make sports into something more than recreation or just learning about the Xs and Os. This is sport based youth development at its best.



[1] Piché, G,  Fitzpatrick, C, and  Pagani, L. (2015, Sep-Oct). Associations Between Extracurricular Activity and Self- Regulation: A Longitudinal Study From 5 to 10 Years of Age. American Journal Of Health Promotion Vol. 30 (1), pp. e32-40.
[2] The positive impact of social and emotional learning for kindergarten to eight-grade students: Findings from three scientific reviews, Technical Report, 2008; http://www.casel.org/library/2013/11/1/the-positive-impact-of-social-and-emotional-learning-for-kindergarten-to-eighth-grade-students