Evaluation
Our approach to evaluation design is rooted in our overall organization mission of “helping the people who help people.” We base our approach to each project on the concepts of public justice, empowerment, and mutual respect while using an ecological framework. An ecological framework values input from all stakeholders from various settings such as students in classrooms, principals and other administrators, community leaders, and policy makers. It examines individuals and communities within their contexts to determine weaknesses and strengths regarding the ‘person-fit’ environment, and recognizes the multiple influences (e.g., political, economic, cultural) occurring within each setting. BSRI also works individually with all clients to determine whether evaluation services constitute human subjects research and whether Institutional Review Board (IRB) approval is needed or preferred. Our evaluation plan consists of the following general steps with stakeholder input at the forefront:
1. Stakeholder Input
2. Theory of Change & Logic Model
3. Process Monitoring
4. Outcome Evaluation
5. Dissemination and Feedback
Needs Assessment
BSRI has experience conducting needs assessments for organizations and specific communities. We use multiple sources of data including both quantitative sources (archival data, survey data, county/local statistics) and qualitative data sources (key informant interviews, focus groups, case/record reviews). BSRI has expertise in focus group research, and can quickly plan, moderate, analyze, and interpret findings. Qualitative and mixed methods research can be done at all phases of the evaluation plan, and are helpful in learning about a population’s perspective to certain services and programs. Finally, survey administration and personal interviewing can also be used to determine needs or gaps in service, and to ensure all projects at hand are understood on a personal level. BSRI has the capability to administer surveys in person, on computers, and via web-based applications. Data administered electronically is easily transferred into statistical databases for storage and analysis.
Formative Evaluation
Organizations often believe in their programs and strategies but may have trouble articulating why and how their programs work to funders and other stakeholders. BSRI can work with your organization to illustrate the how and why using facilitation techniques to create a theory of change and/or logic model. The Theory of Change is a birds eye view process map of your program and can be a very valuable program summary or graphic depiction. The logic model is a detailed ground-level road map of how and why your program works.
Process and Outcome Evaluation
Process Evaluation
BSRI is trained and certified in the NIATx Quality Improvement Process which uses rapid cycle change techniques. NIATx, designed and supported through a collaborative sponsored by the University of Wisconsin School of Engineering, is a model of process improvement specifically for behavioral health care settings to improve access and retention in treatment. BSRI is also trained and certified in Lean Six Sigma techniques for organizational change.
Outcome Evaluation
Outcomes are often what funders and key stakeholders are most interested in when it comes to program performance. BSRI can help your agency specify program outcomes, link data to outcomes, and examine how those results measure up relative to other programs doing similar work. BSRI is always interested in working with agencies to gauge community-level impacts on population health.
In 2016, BSRI brainstormed, invested in and developed the “Data Outcome Evaluation System” (DOES). This data entry system is a fully secure and private web-based platform. All data can be viewed by the client and evaluation team in real time and is customizable based on client needs. BSRI can also analyze this data to produce data dashboards for clients based on priorities within evaluation.
Fidelity Monitoring
Fidelity monitoring is particularly useful when your organization is implementing an evidence-based practice or intervention and refers to whether the program or strategy is being implemented in the way that developers intended. Measuring adaptations to programs and strategies is necessary to make sure expected results occur without unintended consequences. BSRI has a great deal of experience in fidelity monitoring with evidence-based practices through record reviews, qualitative interviews, and measuring other process and outcome indicators of a program. We can help you tailor an intervention to best serve your community and help you measure and document changes along the way so if something is working, you know what it is and vice versa.