News release
SQE1 pilot "successfully achieves its purpose"
31 July 2019
- Functioning legal knowledge performs well
- Early stage skills assessment requires change
- Independent reviewer says pilot "successfully achieved its purpose"
A pilot to test the first part of the Solicitors Qualifying Examination - known as SQE1 - has shown that it is on course to be a valid, rigorous assessment, while also highlighting the need for more work to see if a skills assessment is appropriate in the first stage of the exam.
The SQE is a new, single assessment for qualifying solicitors and is due to be introduced in autumn 2021. Before candidates can take the SQE2 legal skills assessment, they will need to pass SQE1.
SQE1 mainly focuses on functioning legal knowledge (FLK) - testing the application of key principles of legal knowledge to practical situations.
Conducted in March, the SQE1 pilot was completed by a diverse group of 316 candidates at 44 locations in the UK, and in Singapore and France. Candidates took three FLK papers with a total of 360 single best answer questions.
The second element of the SQE1 pilot tested skills, with one legal research and two legal writing exercises.
The pilot was run by Kaplan, the SQE assessment provider. Kaplan's independently verified pilot showed that the proposed FLK assessment meets high standards.
It was found to be reliable, accurate, valid and manageable. It can appropriately differentiate between candidates - marks on the FLK ranged from 17% to 85%, with an average mark of 50%.
We have accepted Kaplan’s recommendation to further enhance the effectiveness of the FLK assessment by amending the design to two 180 question assessments rather than three 120 question ones. This will improve the reliability and accuracy of the exam, helping to ensure that those who pass deserve to pass and those who fail, deserve to fail.
The results did however raise questions about the validity of the SQE1 skills assessment. Whereas the SQE2 skills assessment will include 14 different elements and generally will be taken by candidates with qualifying work experience, the SQE1 skills assessment would likely be taken before work skills had accrued and comprises three elements.
The low number of exercises was a key factor in this part of the assessment not reaching the high standard of accuracy required. The results also suggested that the skills part of the SQE1 may set an unnecessary barrier to qualification which disadvantages BAME (black, Asian, and minority ethnic) candidates, which would be unacceptable.
As a result, we will not go ahead with the proposed approach to assessing skills in SQE1.
We will work with stakeholders to explore different options, including whether a skills assessment is appropriate at this stage. Skills are tested extensively in SQE2.
Paul Philip, SRA Chief Executive, said: "The SQE will help build trust that all qualifying solicitors are meeting consistent, high standards.
"This pilot brings us a step closer to delivering a world class assessment. It provides confidence that the core part of SQE1 is appropriately rigorous, while helping us to improve it further.
"The pilot was also about understanding what does not work, and there is clearly more to do to establish whether an early stage skills assessment can be sufficiently robust. We will continue to work closely with stakeholders to explore the best options for assessing legal skills through the SQE."
Peter Houillon, CEO of Professional Education & Assessment at Kaplan, said: "An exam leading to qualification as a solicitor must meet the very highest standards to both protect the public and be fair to candidates. Kaplan is committed to ensuring the SQE meets those standards. A key step in this process was this pilot and the analysis of the data it has provided. The evidence suggests that an assessment of the application of the functioning legal knowledge taking the form of two papers of 180 questions each will reach those standards."
Kaplan's approach was validated by an independent reviewer, appointed by the SRA after an open recruitment process.
Geoff Coombe, the SQE Independent Reviewer, said: "The pilot was a rigorous, worthwhile exercise. It successfully achieved its purpose to explore whether SQE1 is a fair, reliable, accurate, valid, cost effective and manageable assessment. The FLK assessment performed well, yet there were significant concerns around the SQE1 skills assessment."
The pilot is part of the SRA and Kaplan’s wider work to develop a world class assessment. Hundreds of stakeholders have been involved in the development of the assessment so far. A pilot on SQE2 is due to run in December.
Results from the pilot can be found here:
Note to editors
The functioning legal knowledge (FLK) assessments were divided into three papers as follows:
- Business law and practice; dispute resolution; contract; tort
- Property practice; wills and the administration of estates and trusts; solicitors accounts; land law
- Public and administrative law, legal system and legal services, criminal litigation and criminal law