eAssessment

Use of eAssessment in NOW for Designed Environment students
Adam O'Rourke, School of Architecture and the Built Environment

Background

The course in which this case study occurred was a level one, 20 credit module shared between three programmes (approximately 200 students). The teaching has traditionally been delivered face-to-face with a mixture of lectures and seminars, together with additional workshops. Although NOW has been used to host a supplementary website to support the students, no teaching had taken place using it prior to this case study.

Established practice

One hour lecture and then a four hour workshop / seminar which applied the material covered during the lecture. Adam O'Rourke has been delivering the lecture and most of the seminars with some additional support to deliver the workshops / seminars. The core reading material for these lectures came in the form of DVD content from the University of the West of England, entitled 'Construction Site 09 - 10'. This DVD package (hosted online) contained some multiple choice tests which were delivered as part of directed learning for the module. Evidence suggested that students were not getting maximum benefit from this delivery method, and a more formal approach to the test would enhance the student learning experience.

Rationale

The rationale for this project was the educational benefit that could be derived by changing practice. Making Buildings is a technical module and is well suited to assessment through tests and examinations. The established practice of delivering the tests was not robust, and failed to maximise the learning outcomes of the module. By moving the assessment to become part of the VLE means that they are available both on and off campus, they can provide instant feedback and the results can be analysed to produce individual or cross-group reporting. It also enables the mark to be used as a part of the module's formal assessment. At a time when student numbers are increasing rapidly and time and space to support them is limited, a piece of directed learning which can be delivered in a considered fashion offers added value to the student learning experience. The assessment tool in the VLE also gave the chance for the assessments / question bank to be shared amongst colleagues and delivered, appropriately, on other relevant programmes. The potential for use and reuse of the question bank could reach approximately a further 400 students and offers exciting possibilities for the future.

Changes to practice

One of the biggest changes came on a personal level - learning the new tool and working out processes for developing robust sets of questions which extended the work which had been begun by the Construction Web DVD. It wasn't a completely radical change, but there was a need for targeted staff development during implementation which set the foundation for the creation of the online assessments. Working with the Educational Development Unit to work out strategies for migrating questions from the DVD, workflows for creating question banks and developmental testing, enhanced the pedagogy underpinning the use of online assessment.

The eLearning advantage

Benefits for students included a more blended, balanced approach to learning, removing the emphasis on purely coursework and offering students new ways of getting to grips with core technical concepts. More formative feedback is provided and it is delivered at the point of assessment ensuring a strong connection between work done and feedback received.

From a staffing point of view, using online assessment frees up approximately 120 hours worth of direct contact time that would otherwise be spent in seminars / workshops with the students. There is also a large staff saving related to the reduced marking and assessment load. This allows staff time to be redirected to other teaching or research activities. Although this has only benefited other staff so far (since Adam's time has been taken in developing these resources) the online tests can be reused next year and at this point, Adam will benefit from this time saving too. The in-built reporting has also proved extremely useful - revealing that good performance in the test was linked to attendance and is helping to quantify student engagement on the course. The online assessment has also allowed problem areas both for individual students and for the group as a whole to be readily picked up and teaching adjusted accordingly.

Key Points for effective practice

The use of practice formative tests has helped students to become familiar with the test environment and has meant that, with a 95% uptake, students are already becoming used to taking the online assessments.  It has also been important to gain an understanding of the pros and cons of various question types as well as system restrictions (duration of test, use of hints, number of attempts) to ensure that assessment remains robust in its online delivery.

Finding a real purpose for using the virtual learning environment has heightened both Adam's and his students' engagement with the system - moving beyond simply using NOW as a glorified document storage area and deriving real educational benefit.  It is clear that to get the most from NOW, a strong rationale for use is needed and a careful consideration of pedagogic benefits at every stage is also necessary.

Conclusions and recommendations

Using this online assessment has significantly enhanced the student learning experience. Adam has discovered  a way to move beyond the simple adoption of the minimum standards to find a creative, valid use of the tools embedded within the VLE  that has delivered real educational value.  Developing the question bank for use on the programme will be an ongoing process, potentially to be rolled out to other level one and two programmes so building this into teaching practice is important.  As the process becomes more established and matures, it may be of use in level three programmes as well.

In terms of elements to be changed for the future it is acknowledged that the process needs to be continually evaluated - the type of questions, number of questions etc need to be reflected upon - and building in this evaluation and reflective practice into the use of online assessments is worth noting.  Adjusting the questions is an appropriate and meaningful way will ensure that the use of eAssessment on the programme continues to improve and pay dividends for all involved.