Search Cornell

Collaboration lowers incidence of physical restraint for youth in care

(0) Comments  |   Tags: article,   collaboration,   Elliott Smith,   evaluation,   Michael Nunno,   RCCP,   research,   residential care,   youth,  
Share

Michael Nunno and Elliott Smith

Michael Nunno and Elliott Smith

By Sheri Hall for the BCTR

Two BCTR researchers have been working with a Connecticut child welfare agency to implement and evaluate a program that promotes evidence-based approaches in supporting troubled youth. The Cornell researchers and two agency administrators published the results of their collaborative effort in March in the journal Child Welfare under the title “Benefits of embedding research into practice: An agency-university collaboration”.

Since 2009, Michael Nunno and Elliott Smith, members of the research team for the Residential Child Care Project (RCCP), have consulted with Waterford County School in Connecticut, which provides residential and day care to youth with mental health problems, behavioral issues, addiction and emotional problems.

A team of agency executives, clinicians, supervisors and staff members worked with RCCP staff and consultants to learn about and implement the Children and Residential Experiences: Creating Conditions for Change (CARE) program model.  The CARE model is a research-informed framework created at the BCTR by Martha Holden and her RCCP colleagues that focuses on improving interpersonal relationships between caregivers and youth.  Nunno and Smith were part of the effort to examine if CARE was making a difference in the day-to-day life of the children and staff. 

After the school implemented the program, agency administration reported a substantial decrease in physical restraints among the school’s residential population.  Physical restraints are safety interventions that hold a youth in order to contain physical behavior that is likely to result in injury to the youth or others.  They are, however, not without risks to both the child and the staff since they can have harmful or even fatal consequences. 

“The wonderful thing about the Waterford Country School from an evaluator’s perspective is that it has a thirty-year history of collecting and publishing administrative data on measures that matter to practitioners,” Nunno said.  Our job was to portray the data in relevant and meaningful ways so that it could inform practice, soften professional resistance to change, and add to the growing evidence that relationship-based, trauma-informed practice models can create safe and therapeutic physical spaces.”

“By examining the data, we documented a 48 percent decrease in restraint events within Waterford’s residential and shelter settings,” he said. “We were able to verify the staff perceptions and narratives that the Waterford agency was becoming a safer, calmer place.” 

Yet not all Waterford programs saw this decline.  “The day-school data showed an increase in restraints in the corresponding time frame,” Nunno said.  “Although we were all surprised at this finding, our analysis triggered the agency leadership to examine the children’s social and emotional regulation needs.  They involved day-school teachers and children’s families who designed unified approaches to meet those needs.  Within months of implementing these strategies we saw a significant decrease in the use of restraints.”

The partnership between RCCP and the school demonstrates RCCP’s success at monitoring and detecting problems, guiding solutions, improving practice, supporting learning organizations, and contributing more broadly to evidence-based practice. 

Save

Save

Save

(0) Comments.  |   Tags: article    collaboration    Elliott Smith    evaluation    Michael Nunno    RCCP    research    residential care    youth   
Share

Improving the health of military families

(0) Comments  |   Tags: Brian Leidy,   evaluation,   exercise,   family,   health,   military,   Military Projects,   nutrition,   publication,   smoking,  
Share

BPC_150909_c_AR2_ExecutiveSummary.inddBy Sheri Hall for the BCTR

BCTR researchers have spent two years helping to improve the health and wellness of military members and their families.

The BCTR’s Military Projects team partnered with researchers from the Cornell Office for Research on Evaluation (CORE) to measure outcomes from the Healthy Base Initiative, a short-term project to demonstrate how healthy eating, exercise, and tobacco cessation can improve the lives of active service members and their families. The results were published earlier this year.

Their work was funded by a grant from the National Institute of Food and Agriculture, United States Department of Agriculture and the U.S. Department of Defense.

The Department of Defense launched the project at 14 pilot sites across the world. First, they assessed the health and wellness of the military community at each site, then they implemented a variety of initiatives designed to improve health and wellness, such as fitness programs, menu labeling, cooking seminars, and tobacco cessation campaigns.

BCTR and CORE researchers worked with the Department of Defense Military Community & Family Policy and other researchers to evaluate which initiatives worked the best to help military families lead healthier lifestyles and develop conclusions that will inform a larger wellness initiative across the Department of Defense.

“Health care costs continue to be a large and rapidly growing part of the military budget and include the cost associated with active duty service members and their families as well as military members in the reserve components, retirees, and veterans,” explained Brian Leidy, senior extension associate at the BCTR and director of Military Projects. “Just like in the civilian population, metabolic diseases which are largely preventable through proper diet, exercise, and avoiding tobacco play a major role in the acceleration of these costs.”

The project identified a wide range of recommendations such as encouraging different service branches within the military to work together on health and fitness initiatives, finding ways to offer more healthy food options within military communities, offering childcare while adults exercise or participate in wellness activities, and creating more tobacco-free areas.

You can read the full list of recommendations in the Healthy Base Initiative executive summary and full report.

Save

Save

Save

Save

(0) Comments.  |   Tags: Brian Leidy    evaluation    exercise    family    health    military    Military Projects    nutrition    publication    smoking   
Share

ACT for Youth: Measuring Positive Youth Development

(0) Comments  |   Tags: ACT for Youth,   Amanda Purington,   Christine Heib,   Dora Welker,   evaluation,   youth,   youth development,  
Share

A myriad of evidence-based programs exist to help young people develop positive life skills and avoid risk behaviors. Evaluations of program success are frequently focused on whether programs directly reduce negative outcomes for youth. An alternative approach can be to measure Positive Youth Development (PYD) outcomes resulting from a particular program. Program evaluation with the PYD approach places the focus on positive, healthy outcomes for youth, rather than a focus on reduction of negative outcomes.

In 2015, the New Jersey Department of Health (NJ DOH) asked the ACT for Youth Center of Excellence to provide program evaluation of Wyman’s Teen Outreach Program® (TOP®). TOP® is an evidence-based program that offers young people community-based, service-learning opportunities in order to build on their strengths and reduce risk behaviors. The NJ DOH facilitates the implementation of TOP® and has traditionally measured its success using Wyman’s own pre- and post-surveys. While these surveys provide useful data, the NJ DOH wanted to augment the evaluation with a PYD approach. The ACT for Youth team, including Amanda Purington, Christine Heib, and Dora Welker, was contracted to identify specific PYD measures that could be used in conjunction with the existing evaluation approach, and to develop a survey tool for measuring progress in PYD outcomes.

act pyd

Figure 1. Relationship between subscales, Five Cs, and PYD; from Geldhof et al., 2013.

To develop the new surveys, ACT for Youth used the Positive Youth Development Inventory – Very Short Form.These measures are based on the Lerner and Lerner “Five Cs” model of PYD, which encompasses the “Cs” of youth development: Competence, Confidence, Connection, Caring, and Character. Each of these “Cs” are measured using subscales that assess different aspects of youths’ lives. The PYD surveys created by ACT for Youth incorporate these subscales to get a more comprehensive picture of youths’ lives and the impacts of the TOP® program.

While funding for the evaluation came from the NJ DOH, implementation was done through the Central New Jersey Family Health Consortium, Inc. (CJFHC). Youth completed the PYD pre- and post-surveys at five separate programming locations where TOP® was implemented during the 2015-2016 school year. Additional information such as demographics, living situation, year in school, and school attendance were also collected, enabling ACT for Youth to conduct a more comprehensive analysis of program effects.

After looking at results of the PYD pre-surveys, ACT for Youth found high baseline scores for all five of the Cs. This was a very positive finding -- however, it did create a challenge for evaluation, since with such high initial scores there was not much room for further improvement. While the evaluation did not detect much improvement, and saw some decreases, overall these high scores were largely maintained at post-survey.

To better understand differences among the results, the ACT for Youth team also analyzed pre- to post-survey differences in relation to a variety of grouping variables, such as attendance and “baseline risk,” which was measured on the Wyman pre-survey. Examples of risk factors in this survey include: “failed a course,” “been suspended,” “been pregnant/caused a pregnancy,” and more. ACT for Youth grouped and coded these risk factors into “academic baseline risk” and “sexual health baseline risk” and used these baseline risk levels to further assess TOP® program efficacy, taking into consideration the diverse backgrounds of youth participants and their very different starting points at the beginning of TOP® programming. The goal of this analysis was to see if TOP® is more effective among certain distinct populations. Interestingly, some of the findings from this analysis pyd chart 1suggest that TOP® implementation may have some of its most positive effects on the most vulnerable youth.

Creating and using the PYD surpyd chart 2veys allowed the evaluation team to assess positive youth development, alongside traditionally measured negative risk factors, for a more comprehensive and optimistic evaluation of TOP® in New Jersey. While it was unexpected that a number of items moved in the undesired direction from pre- to post-survey, several reasons might explain this. First, as mentioned, with highly positive baseline scores it is difficult to achieve significantly higher results at post. Additionally, it is possible that implementation of the TOP® program mitigates what would otherwise be even greater declines in certain areas. In the future, this hypothesis could be explored by having youth not involved in TOP® programming also complete pre- and post-surveys as a way of assessing a control group.

Save

Save

Save

(0) Comments.  |   Tags: ACT for Youth    Amanda Purington    Christine Heib    Dora Welker    evaluation    youth    youth development   
Share

Doing Translational Research podcast: Brian Leidy

(0) Comments  |   Tags: Brian Leidy,   evaluation,   family,   military,   Military Projects,   podcast,  
Share

Evaluating Military Family Programs
Tuesday, November 8, 2016

Brian Leidy
Bronfenbrenner Center for Translational Research, Cornell University

(0) Comments.  |   Tags: Brian Leidy    evaluation    family    military    Military Projects    podcast   
Share

ACT for Youth at American Evaluation Association conference

(0) Comments  |   Tags: ACT for Youth,   Amanda Purington,   evaluation,   Jane Powers,   Mary Maley,  
Share

Powers, Purington, and Maley

Powers, Purington, and Maley

This October, staff from the ACT for Youth Center of Excellence participated in the annual conference for the American Evaluation Association held in Denver, CO. For the conference, Jane Powers, Mandy Purington, and Mary Maley organized a panel on the theme of building capacity to strengthen youth programming through the use of evaluation findings. The ACT team described how the Center of Excellence has been supporting the implementation of evidence-based teen pregnancy prevention programs. Through case examples, they illustrated how implementation data are summarized and made accessible to program staff, and how these data help staff reflect on evaluation findings and identify ways to improve fidelity and quality. Colleagues from the University of Wisconsin joined the panel to present on their work in Madison with community program staff, educators, and youth.

In a demonstration session, the ACT team described the three-phase needs and resources assessment process they developed to identify gaps in local supports for expectant and parenting young people. Their approach includes a community partner brainstorm phase, a key informant interview process, and youth focus groups with expectant and parenting young people. They described how the information gained from this process led to action planning for each of the participating communities.

Finally, Jane Powers served as a discussant on a panel organized by Abe Wandersman addressing the issue of organizational readiness for implementing innovations. The three papers in this session focused on how to assess, build, and evaluate organizational readiness.

(0) Comments.  |   Tags: ACT for Youth    Amanda Purington    evaluation    Jane Powers    Mary Maley   
Share

The Military Projects conducts training for Army Reserve leadership

(0) Comments  |   Tags: evaluation,   military,   Military Projects,   training,  
Share

On April 15, 16, and 17, the BCTR’s Military Projects and the Cornell Office for Research on Evaluation (CORE) hosted a three-day training program on evaluation capacity building for the leadership of the Army Reserve Family Programs (ARFP). The group attending consisted of 44 Headquarters, Regional, and command support level staff who are responsible for the management and delivery of training and family support programs throughout the United States and its territories. The training served as the launch of a two-year partnership beginning April 1st to develop performance metrics and measures of effectiveness for the family support programs that the Army Reserve provides for all Reserve soldiers and their families. This work is funded by an award through United States Department of Agriculture (USDA) National Institute of Food and Agriculture (NIFA) and the Army Reserve. In addition, the partnership will develop a standardized needs assessment to gather community input on the needs of Reserve soldiers and family members which will be used by each Family Programs Center to prepare for their tri-annual accreditation review.

During their three days on campus, the group began the process of developing pathway models for their programs and services. These models will be finalized during a one-day follow up training in Boston in August. During the time between now and August, concept mapping will be conducted with a large group of program stakeholders to assure that the models incorporate the broadest range of perspectives of those who have a vital interest in the program. Evaluation plans will be developed from the finalized models and outcome metrics will be selected from the evaluation plans for piloting. As outcome metrics are validated they will be incorporated into the ARFP’s client tracking system to be used for ongoing monitoring and evaluation. The work begun in Ithaca in April will serve as the basis for a long and careful process to develop valid and credible measures of these critical programs and services.

 

(0) Comments.  |   Tags: evaluation    military    Military Projects    training   
Share

Military Projects renewed to evaluate U.S. Army’s Family Programs

(0) Comments  |   Tags: Brian Leidy,   evaluation,   Marney Thomas,   military,   Military Projects,  
Share

The Military Projects within the Bronfenbrenner Center for Translational Research was recently awarded $267,372 by the National Institute of Food and Agriculture, United States Department of Agriculture and the U.S. Department of Defense under Agreement No. 2011-48746-31000 to conduct program evaluation and needs assessment for Family Programs in the United States Army.

This is a renewal of work begun more than ten years ago that includes the development of performance and outcome metrics to be used by Family Programs across the Army and assisting local Army installations as they carry out needs assessment in preparation for their tri-annual accreditation process.  The Family Programs in the Army provide education, counseling, and support to help military families deal with the unique stressors of military life. Military families typically experience lengthy family separations, frequent moves, and isolation from family support networks while serving in locations across the country and overseas.

Brian Leidy and Marney Thomas

This work will be led by Brian Leidy and Marney Thomas, both Senior Extension associates at the center and will be carried out between September 1, 2012 and August 31, 2013.

(0) Comments.  |   Tags: Brian Leidy    evaluation    Marney Thomas    military    Military Projects   
Share

Talks at Twelve: Monica Hargraves, Thursday, March 15, 2012

(0) Comments  |   Tags: BCTR Talks at Twelve,   CCE,   CORE,   evaluation,   Monica Hargraves,  
Share
 

The Evaluation Partnerships: A Systems Approach to Program Evaluation
Monica Hargraves, Manager of Evaluation for Extension and Outreach, CORE

Thursday, March 15, 2012
12:00-1:00pm
Beebe Hall, 2nd floor conference room



The Cornell Office for Research on Evaluation (CORE), led by Professor William Trochim, has been conducting and applying research on a new approach to evaluation planning and evaluation capacity-building. CORE’s approach is grounded in an evolutionary, “systems” view of programs – basically, that programs are systems nested within organizations which are nested within larger entities and so on, and that they evolve and change over time. Simple as these principles may seem, this systems approach has significant implications for how to conduct evaluations. CORE has developed a Protocol for this approach to evaluation and has used it in “Evaluation Partnerships” with more than 50 Cornell Cooperative Extension (CCE) programs as well as a number of outreach programs at research centers across the US. Monica Hargraves leads the facilitation of CORE’s Evaluation Partnerships with CCE. She will introduce the Protocol and talk about how it has worked in practice within Extension programs, the impacts it has had on program staff and their work, and its implications for building the research base for programs and their evaluation.

Monica Hargraves has a Ph.D. in economics and spent the early part of her career at Brown University and in the Research Department of the International Monetary Fund. Her growing interest in more applied, community-based work precipitated a significant career shift. In 1998 she joined the staff of Cooperative Extension of Tompkins County, where she ran volunteer and educational outreach programs and later moved into organizational roles focused on valuing the work of Extension, internal reporting systems, strategic planning, and evaluation. She joined CORE’s research team in 2008, and works with Extension organizations and programs across NYS to build capacity for evaluation. Her research and applied interests involve integrating evaluation into the on-going work of program management and program development at all levels of the system, and improving the understanding and ultimately the valuation of Extension work.

Lunch will be served.  This event is free and open to all.
Metered parking is available across Plantations Rd. in The Plantations lot.

(0) Comments.  |   Tags: BCTR Talks at Twelve    CCE    CORE    evaluation    Monica Hargraves   
Share

Talks at Twelve: Margaret Johnson, Thursday, February 23, 2012

(0) Comments  |   Tags: BCTR Talks at Twelve,   evaluation,   Margaret Johnson,   PAM,  
Share
 

Building Strong Evaluation Policy in Organizations
Margaret Johnson, Policy Analysis & Management and Finger Lakes Center for Law and Social Policy

Thursday, February 23, 2012
12:00 - 1:00 PM
Beebe Hall, 2nd floor conference room



Lunch will be served. This talk is open to all.  Metered parking is available across Plantations Rd. in The Plantations lot.

To generate a set of relevant types of evaluation policy for the US federal government context, Margaret Johnson conducted a survey of 600 members of the American Evaluation Association in 2009. Participants were asked to brainstorm examples of evaluation policy and then sort and rate them. Results were analyzed using a concept mapping technique and then used to create an evaluation policy inventory instrument. Her presentation will focus on the evaluation policy inventory instrument, with step-by-step instructions for its use in organizations.

Margaret A. Johnson, PhD, is a January 2012 graduate of the Department of Policy Analysis and Management (PAM) with a focus on program evaluation and a special interest in helping organizations develop sound evaluation policies. Current work in the Department of PAM includes two Cooperative Extension-funded outreach and research projects: one to develop and evaluate a curriculum to teach older teens about the health care/health insurance system, and another to investigate how unmarried parents make decisions about child custody. She is also an associate of the Finger Lakes Center for Law and Social Policy, which provides program evaluation and consulting services to state and local governments and other nonprofit organizations.

(0) Comments.  |   Tags: BCTR Talks at Twelve    evaluation    Margaret Johnson    PAM   
Share