Visit to observe the Qualified Lawyers Transfer Scheme in operation

By Crispin Passmore, Executive Director of Policy & Education, 22 June 2015

A couple of weeks ago, I went to visit the Qualified Lawyers Transfer Scheme (QLTS) testing in operation to gain a better understanding of our current mechanisms for assuring standards at point of entry to the profession. The QLTS is the test which overseas qualified lawyers and barristers of England and Wales must take if they wish to be admitted as a solicitor of England and Wales.

Its introduction in 2011 has opened up the English and Welsh legal market to qualified lawyers from 154 jurisdictions as diverse as Russia, Japan and Argentina, provided they can demonstrate they have the skills and knowledge to practise effectively and safely as an English and Welsh solicitor.

The QLTS consists of two parts: a knowledge test (conducted by multiple choice) of and a series of six half day assessments which test candidates' knowledge and legal skills in their practical application.

The test is written and run by a single SRA-appointed assessment organisation, Kaplan QLTS, which has no involvement in teaching or preparation for the test, and therefore provides complete independence in the testing process. This structure means that all candidates are assessed to the same, consistent standard. Numbers of candidates have steadily grown. In 2013, 709 candidates attempted the QLTS and in the same period 469 candidates qualified as a solicitor through this route.

I've learned that the QLTS methodology which the SRA and Kaplan introduced was ground breaking in building on and adapting lessons learned about assessment and standard setting practices from legal exams in other jurisdictions and medical education.

The first part of the QLTS is the MCT. This requires candidates to answer 180 questions covering the full range of core legal knowledge: contract law, tort, criminal law, constitutional law, EU law, ethics, tax, financial services, solicitors' accounts, property law, equity, business structures. It is not a memory test. Candidates have to use their knowledge of relevant legal principles to identify the best solution to clients' problems. The first challenge is to recognise what the relevant principles are. Unlike a traditional exam in, say, contract law, candidates do not know that the answer must be a contractual one. They must spot the relevant legal issue and then apply it to a practical scenario. Kaplan publish example questions on their website. Try it and see how you get on. I'm not a lawyer at all and I had a go - luckily results are confidential!

In setting the pass mark for the MCT, the Examination Board makes use of the Angoff method. A panel of solicitors decide what percentage of candidates who have just reached the standard of a newly qualified solicitor would answer each question correctly. This is a widely used and internationally recognised standard setting method to ensure the pass mark is related to the difficulty of the questions.

Once candidates have passed the MCT, they must take a practical test, called the Objective Structured Clinical Examination (OSCE). This consists of a total of 18 assessments: one each in interviewing, attendance note/case analysis, advocacy, research, writing and drafting in each of the contexts of business, property and civil and criminal litigation.

The scenarios are complex and realistic and the marking is rigorous. Take interviewing. The client is a professional actor, who is role-appropriate. You don't get a twenty-year old instructing a solicitor on a will leaving their estate to their adult children, or a middle-aged person acting the part of a client arrested for possession of drugs at a rave. The actors assess candidates' client handling skills. They know best how they felt as the client; whether the candidate gained their confidence and whether they understood clearly what the candidate was asking them. Separately, qualified solicitors assess the accuracy of candidates' legal advice. As I sat in the control room watching the action on a bank of screens I was struck by the realism of the assessment. Sadly, for too many solicitors, this will be the last time they are observed in real time to aid their learning and development. It is a well established tool in many industries and businesses - including the SRA.

I had the chance to talk to the actors' supervisor and the overall QLTS lead. The supervisor watches the actors to help ensure consistency. They face relentless and rigorous feedback to support that. The level of training and preparation for each assessment centre impressed me. Role plays of the role plays, mock scoring, group discussion and detailed training are some of the elements that underpin the actual assessment day. This isn't a few teachers marking exam scripts late at night or at the weekend!

For the OSCE, candidates skills and application of law are graded ranging from 'A' indicating superior performance that is well above expected competence to 'F' - namely poor performance that is well below competence.

In setting the pass mark for the OSCE the Examination Board makes use of the borderline regression method. The borderline regression method is widely used in high stakes professional exams. It provides a way of ensuring that the pass mark is related to the difficulty of the assessment.

I observed a cross-section of the OSCE tests. It was clear to me that the mix of skills and knowledge required was right and properly separated the sheep from the goats. Those without the right knowledge could not ask the right questions or present legal information correctly and those without the right skills or who couldn't communicate effectively in English lost their knowledge in the midst of unintelligible waffle.

I was particularly interested in how ethical competence and capacity was tested in this environment. Rather than sit a separate paper on ethics or be assessed through workplace experience, there are questions and issues woven throughout the assessments. The candidates need to recognise the issue as an ethical one when they are focused on legal or other practical skills. That strikes me as both more valid and more realistic.

It struck me as I travelled home that I hadn't wondered what degree the candidates have, which university they'd been to or where they done their training. This appears, on my observation, to be input blind. That must be good for diversity and social mobility because it is relentlessly focused on merit. Too often attempts to improve diversity get challenged on the grounds that we mustn't lower standards (though I've never actually heard anyone argue that we should). This assessment methodology seems to turn the debate on its head with a relentless focus on merit, on consistent and rigorous standards without any nod to background and that seems to me to be a positive.

It is a tough test which provides me with confidence that solicitors qualifying through this route have the right knowledge and skills to practise effectively.