Monday 14 October 2013

Why 'soft skills' are worth assessing too

US expert says qualities such as resilience, grit and 'stick-to-it-iveness' are key to success in life
By Sandra Davie, The Sunday Times, 13 Oct 2013

Singapore parents are no different from American parents in fretting over education changes, whether the subject is examinations or school admission.

But American education expert Brian Stecher says change, though painful at times, is needed to prepare children for a world environment that is ever-changing and ambiguous.

Announcing various changes to the education system in response to criticism that children face too much stress too soon, the Education Ministry stated two months ago that it was looking into broadening the Primary School Leaving Examination scoring system.

But what made some parents sit up was the ministry's move to also expand the Direct School Admission scheme which allows children with a special talent or ability to secure a place in a secondary school of their choice regardless of their PSLE results.

The ministry suggested taking into account qualities such as a child's character, leadership and resilience - and that left parents asking if secondary school admission might become less transparent and more subjective, because how would such soft skills be assessed?

Dr Stecher, who was in Singapore to speak at a symposium on the teaching and assessment of 21st century skills organised by the US-based Asia Society, says the parents' concerns are understandable, but educators around the world, convinced of the "absolute importance" of these skills, are already looking at ways to teach and assess skills such as communication and digital literacy.

Nurturing soft skills and building character in students is no easy task, he adds.

"Teaching communication skills and digital literacy is one thing, but how do you build character, qualities such as resilience, grit and 'stick-to-it-iveness', which studies show are important for success in college and life? What sort of experiences must you provide students to nurture these qualities in them? There are no easy answers to these questions," he says.

If "teaching" those skills is difficult, measuring and assessing them is even more so.

But he points out that there are schools and organisations such as the Organisation for Economic Cooperation and Development (OECD) - which runs the triennial Programme for International Student Assessment (PISA) test - that are working on interesting ways to measure these qualities.

The 2015 PISA study will not only test 15-year-olds in mathematics and science and literacy but will also test them on whether they are able to collaborate well with others to solve problems.

Singapore - along with Finland, Shanghai and South Korea - was one of the star performers of the 2009 PISA study.

Dr Stecher applauds the OECD's move to broaden its PISA study to measure skills such as collaborative learning, one of the skills seen to be essential by employers.

Another method he considers promising is evidence collection or portfolio assessment used by some American high schools, where a student builds a record of his achievements, both academic and non-academic, over his three years in high school. The portfolio becomes the basis for a holistic assessment of the student.

As for parents' concerns about subjectivity in assessing soft skills, he says there are several ways to ensure fairness and consistency.

But he also notes that in the real world, subjective judgements are made regularly in the workplace.

He says: "Bosses and supervisors make subjective judgements on their employees all the time, but we accept that. It is part of working life. So why won't we accept it when it comes to school life?"

Just as companies have checks to reduce bias, schools too should have systems and measures in place to ensure fair assessment of students. "You try and standardise the judgements  You don't just give teachers a blank piece of paper and ask them to write about a student," he says.

There are various ways to do this. "Schools can come up with a rubric for awarding marks, and then train their teachers to use the rubric consistently, and there can be checks and balances, such as getting more than one teacher to assess a student's work."

What about transparency?

First he points out that a portfolio assessment is in a way more transparent than a pen-and-paper test which parents do not get to see.

He also advocates schools sharing with parents their test objectives and scoring guides.

But he concedes that it can get really difficult when the decisions are high stakes, for example, having an impact on whether a child gets into a top school based on the results.

"The more important a decision is, the more uncomfortable people are," he says, adding that having checks and balances will help.

He also points out that even at the university level, broader admission criteria are increasingly being used in many countries, including Singapore.

To select students for prestigious, limited medical school places, for example, universities rely not only on strong academic results, but also test candidates' aptitude and use interviews and teachers' recommendations.

"Most of us would not want our future doctors to be selected on grades alone. We want people who will be caring, compassionate and who have good communication skills," he says.

Acknowledging that change is uncomfortable and difficult, he says one way to ease parents and teachers into the changes is to try it out on a small scale, as in a pilot.

"This way you can see what works and what doesn't, and make adjustments and changes," he adds.

The outcomes should also be studied over a period. "It is important to look at if, indeed, students acquire those skills you want them to. Does it have an impact on their maths and science and English scores? What about employers - do they see the difference?"

The shift will not be easy - not only for students and their parents, but also educators - but he is convinced it is needed to nurture "broader citizens".

"By that I mean young people who will go on to have multiple successful careers, who will continue learning and who can collaborate with people across all ages and from different cultures," he says.

PISA to test collaborative problem solving

Come 2015, 15-year-olds sitting for the Programme for International Student Assessment (PISA) test will not be tested only on their problem-solving skills in mathematics, science and reading.

They will also be tested on collaborative problem solving. They will be asked to solve a problem by collaborating with a partner, in this case, a software program.

Students will have to use their interpersonal and communication skills to engage the program and pool knowledge and skills to complete a task.

The PISA study, the first of which was held in 2000, is aimed at assessing the extent to which 15-year-old students near the end of secondary education are able to analyse, reason and apply their knowledge and skills in unfamiliar settings so as to meet real-life challenges.

Over the years, educators and policymakers around the world value the information the tests have provided on student and school performance, and how educational systems in various places compare with one another.

The 2009 PISA study which Singapore participated in along with 64 other countries and territories, saw 15-year-olds here ranked fifth in reading, second in mathematics and fourth in science. It was the first time that Singapore had participated in the study. The other top scorers were from Hong Kong, Finland, South Korea and Shanghai.

Explaining the move to measure collaborative problem-solving, the Organisation for Economic Cooperation and Development said much of the problem-solving work done in the world today is performed by teams in an increasingly global and computerised economy. It also quotes a University of Phoenix Research Institute study which identified virtual collaboration as one of 10 key skills for the future workforce.

Dr Brian Stecher, like many educators around the world, welcomes the move to broaden the tests to measure soft skills that are becoming increasingly crucial to thrive in the workplace.

He says the PISA test will be a powerful database that can be used by educators to work out how to teach and assess such skills in their students.

"It will be interesting for example to see if students who score high on individual problem-solving in maths and science, do as well when it comes to having to solve a problem in a team and whether students from some cultures are more adept in working in teams," he said.

Since 2000, a randomly selected group of 15-year-olds take tests every three years in reading, mathematics and science, with the focus on one subject in each year of assessment.

The students and their school principals also fill in questionnaires on the students' family background and the way their schools are run. Some countries and economies also choose to have parents fill in a questionnaire.

In 2000 the focus of the assessment was reading, in 2003 mathematics and problem solving, in 2006 science and in 2009 reading again. The 2012 data collection focused on mathematics and included an optional computer-based assessment of mathematics and reading involving some 30 countries as well as an optional area of assessment: financial literacy, which 19 countries took up.

The 2012 results will be published in December.


Dr Brian Stecher is a senior social scientist and associate director of RAND Education, part of the global policy think-tank RAND Corporation.

His research focuses on measuring educational quality and evaluating education reforms, with a particular emphasis on assessment and accountability systems.

During his 20 years at RAND, he has directed prominent national and state evaluations such as No Child Left Behind, Mathematics and Science Systemic Reforms, and Class Size Reduction.

His measurement-related expertise includes test development, test validation, and the use of assessments for school improvement.

He has served on expert panels in the United States relating to standards, assessments and accountability and has published widely in professional journals. He is currently a member of the editorial boards of journals including Educational Evaluation and Policy Analysis and Educational Assessment.

He received his PhD from the University of California, Los Angeles.

The Global Cities Education Network was established by the US-based Asia Society as a mechanism for educators and decision-makers in Asia and North America to collaboratively "dream, design and deliver" solutions to common challenges which education systems are currently grappling with. Singapore is part of this network.

The previous two symposiums were held by the group in Hong Kong last year and Seattle earlier this year. The third symposium held here last week saw more than 70 participants from 10 cities including Seattle, Denver, Houston, Hong Kong, Melbourne, Seoul, Shanghai and Toronto.


No comments:

Post a Comment