We measure our success using a research method called observational measures to evaluate our youth's growth three times a year - the beginning of a school year, middle, and end.
What do we measure? The Five C's of the 21st Century Skills
ProAct Indy strives to develop the “four C’s” of 21st Century skills with participating youth and adults who engage in our programs. The four 21st Century Skills are communication, collaboration, critical thinking, and creativity. The fifth “C” that ProAct develops is confidence. These are skills that students need to develop to be successful in the 21st century. ProAct develops these skills through weekly social equity workshops and monthly service projects that the youth participate in throughout the Indianapolis community.
We measure our success using a research method called observational measures to evaluate our youth's growth three times a year - the beginning of a school year, middle, and end. The benefits of using observational measures include that the measurement is based on students’ language and actions, and is related to their schooling, which is more personable for the students. This creates a meaningful experience for students and provides the opportunity to present problems that are similar to what the students already face.
How do we measure these skills? Youth Program Evaluation Rubric
In partnership with the University of Indianapolis, ProAct has designed a proprietary evaluation rubric to be able to measure the skills of the “four C’s” plus confidence. Our rubric is adapted from a rubric provided by the Houston Independent School District.
Each skill being assessed was operationalized by defining what actions constitute each skill. The actions that constituted showing communication skills include being able to communicate orally, being able to use written communication, and being able to listen. Collaboration skills are demonstrated by students through negotiation, being able to work with others, contributions, responsibility, and respect. Creativity is demonstrated by students through flexibility, elaboration, curiosity, and risk-taking. Critical thinking is measured in students by identifying issues/seeking information, using imagination or brainstorming, selecting or developing a plan, implementing a plan, and improving or evaluating it. With confidence, since it is not considered a 21st Century Skill, it was operationally defined through the definition that ProAct uses. For a student to demonstrate confidence, they must demonstrate the organization's values and optimism towards the project/activity and others.
Success is measured by a 5% or more increase in a group that demonstrates "excelling" measures in our rubric.
How do we mitigate the potential for bias in our observational evaluation of youth?
One consideration in developing the evaluation we have addressed is the potential that evaluators may likely present with some sort of bias (Center for Assesment 2020). This is something that our team has intentionally thought through and is aware of, and it has been marked within the management and implementation processes of our evaluation delivery.
An example of bias that could occur could be a language barrier.
It is important that the individuals assessing the students are able to understand them. It is important for ProAct to be able to provide instructions for the evaluation activities in another language.
The way we have lessened the prospect of bias being presented in our evaluation process is to require two evaluators to observe our youth when we implement our process. We then take the average of the two scores as the final assessment of each of the groups observed.
For a complete understanding of the rubric and instructions, we will share them upon request at our discretion.
Please email our training team at learn@proactindy.org for more information.