How to Measure Soft Skills
Introduction
International youth development programs with soft skills interventions need to be able to accurately measure soft skills among their youth beneficiaries in order to assess participants, improve programming, know whether interventions are improving the skills, and determine whether skill acquisition has an impact on outcomes. Although there may be consensus that soft skills are important, there is less clarity on how to measure them. We conducted an inventory of measurement tools using objective criteria. Our review focuses on the skills identified in the two literature reviews (Lippman et al., 2015 and Gates et al., 2016) as enjoying strong and wide-ranging support across multiple outcomes.
This study provides international youth development programs with a thorough overview of soft skill measurement methods, from a practitioner perspective.
By applying a precisely defined set of criteria (page 33), YouthPower Action identified a set of 10 tools (page 50) that measure the three skills in the center of the Venn diagram — positive self-concept, self-control, and higher-order thinking skills. These tools may be promising starting points for programs searching for measures that are available now.
Types of Soft Skills Measures
Measurement tools may be used for several different purposes for international youth development programs: 1) to inform program participants of their progress; 2) to provide programs with information for the purpose of better implementing their activities; 3) to describe or monitor the progress of youth at the group level within a program; and 4) to evaluate the effectiveness of a program in developing skills or having an impact on specific outcomes through skills development.
Measures may differ by their use in form, content, the nature in which scores are reported, and the level of standards applied. Multiple types of measurement tools exist, and there are a number of different ways to organize them (ETS, 2012; Duckworth and Yeager, 2015; Kyllonen, 2015; Soland et al., 2013; and Stecher and Hamilton, 2014).
The YP Action inventory organizes measures according to four categories (page 20): 1) self-reports and self-ratings; 2) reports and ratings by others; 3) performance assessments and simulations; and 4) mixed methods measures.
Several challenges (page 24-26) plague the field of soft skills measurement, including:
1) Ensuring technical quality;
2) Addressing bias in self-reporting and other reporting methods;
3) Measuring change over time; and
4) Developing and adapting tools for use across cultures and differing resource levels
Methodology for Instrument Review and Screening
The measurement tool review process began by identifying all tools (page 37) that might measure one or more key soft skills of interest. Tools were first screened for relevance to the key skills, age appropriateness, and cost. Those that passed the initial screen were then reviewed in more depth for evidence of validity, reliability, outcomes of interest, ease of use, evidence of international use, and assessment type (page 31).
Then, the tools were divided into three groups based upon the degree to which they met all the criteria: high (meeting between five and seven criteria), medium (meeting from three or four criteria), and low (meeting fewer than three criteria).
We used this data to create an inventory of 74 soft skills measurement tools (page 33).
We also developed a shortlist of tools (page 50) that measure the three skills in the center of the Venn diagram —positive self-concept, self-control, and higher-order thinking skills. These tools may be promising starting points for programs searching for measures that are available now. The tools have been used for different purposes, including program evaluation (the Chinese Positive Youth Development Scale and the Jamaica Youth Survey), group performance assessments (the California Healthy Kids Survey: Social and Emotional Health module and the SENNA surveys), and individual assessments (all the remaining tools). All of the instruments use self-reports by youth, except for the Knack game, which is a performance assessment.
Towards a New Measure
Our research revealed that, while strong instruments exist, there is no single existing instrument that:
- Encompasses all key skills identified as most important across three domains (workforce, violence prevention, and SRH);
- Is suitable for measuring change in skill levels over time among youth in international development programs; and
- Meets other key criteria for use by youth programs (including ease of administration, validity, and reliability).
Therefore, YouthPower Action is developing a soft skills assessment tool that will focus on measuring three soft skills: positive self-concept, self-control, and higher-order thinking skills, in addition to social skills and communication. Youth development programs will be able to use the tool to assess youth’s soft skill levels within the context of a program. Key components of the tool include a youth self-report with anchoring vignettes and a short observer report tool.
Resources
Types of Soft Skills Measures & Challenges:
Duckworth, A. L., and Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237-251.
Educational Testing Service (ETS). (2012). Assessment Methods.
Kyllonen, P. C. (2015). Designing Tests to Measure Personal Attributes and Noncognitive Skills. In Suzanne Lane, Mark R. Raymond, Thomas M. Haladyna (Eds.,) Handbook of Test Development. Abingdon: Routledge
Soland, J., Hamilton, L. S., & Stecher, B. M. (2013). Measuring 21st Century Competencies: Guidance for Educators.
Stecher, B. M., & Hamilton, L. S. (2014). Measuring Hard-to-Measure Student Competencies: A Research and Development Plan. Research Report. RAND Corporation. Santa Monica, CA.