Using Questionmark Perception (QMP) in formative assessment
School of Social Science working with School of Education and the Centre for Academic Practice. Primarily deployed in the School of Social Science.
Background
Between 2002 and 2003, a set of three online interactive tests were created by a small team (Jane McNeil, Ann Liggett, Ed Foster and Trevor Pull). These helped students improve their academic writing skills and tested their knowledge of grammar, plagiarism and referencing. The assessment also suggested areas for development and directed participants to appropriate learning resources.
The three student-friendly online tests are respectively called the Grammar Beagle, Referencing Ferret (see Fig. 1 below) and Plagiarism Badger.
Established practice
Prior to these development of the three QMP online assessments the only source of support for these academic writing skills was paper-based or down to individual tutors to address areas for concern. There was no uniformity to the approach.
Rationale
The problems caused by a lack of understanding of the skills required for academic writing were wide-spread across the University. QMP had been explored by one of the team as a resource which was openly accessible and online was felt to address this and help maximise potential participation.
Changes to practice
There are a set of technical skills to be acquired in order to use the software to set up the tests. Time needs to be allocated to this at an early stage to a) more fully understand the potential of QMP and b) allow sufficient time for question development later in the project.
The eLearning advantage
There are several main advantages:
- The learners have a resource which they can use at any time to brush up their writing skills and assess their knowledge.
- QMP is good at highlighting trends, such as main grammatical errors or referencing problems
- It can be deployed anonymously which can be effective if testing is to be on what might be considered a sensitive issue
- The reporting is excellent – it can provide prompts for follow-up action or provide reports for students who could then address the highlighted issues with a student advisor.
Key points for effective practice
There are three main processes to bear in mind:
- Development
- Delivery
- Reporting
Of these, the most time consuming is development, more specifically, the development of the question set. Writing assessments is a highly skilled task and delivering them online is no less skilled and the feedback and resources which are to be offered require careful planning.
An appropriate quality assurance process also needs to be in place. In this case, each individual wrote a set of questions and as a team they worked together to test and revise the questions appropriately.
Conclusions and recommendations
This experience shows that using QMP is not just about summative assessment but that it can play an extremely valid role as a teaching tool in formative assessment too.
Ongoing promotion of the tests would widen their use and draw attention to their availability. The means of dissemination need planning during the development phase of any test.
Contact details
Trevor Pull, and Jane McNeil.
Fig.1 Screenshot of the Harvard Referencing Ferret
