Assessment

By Dr. David Towlson, BSc, PhD, MIOSH

In case you have second thoughts already about the rather unexciting title, you should note that this article draws heavily on some far more learned sources referenced at the end, to which you are referred. Assessment is a tool to monitor learning and student achievement. To some it is a pariah and to others it is a friend. Sounds romantic and poetic but we do it all the time and in lots of different ways – from the “quick and dirty” to the ultra sophisticated.

Assessment is generally recognised as having at least three uses (I have neglected to mention the half dozen other household uses that could spring to mind):

• Diagnosis (for example to establish entry standard onto a course)

• Feedback (both to students on their progress and also to teachers on the effectiveness of their teaching strategies)

• Standards (for example, to make sure students have achieved a certain standard – some of which may be used for national comparisons, as is the case with GCSEs and other national punishments).

Of course, in order to have credence and retain secret knowledge amongst educationalists, assessment must have a nomenclature all of its own. It is usually divided into two types: summative and formative. Summative assessment usually occurs at the end of a course or module (typically a formal exam or test), identifying the standard you have achieved after a course of study and so perhaps to enable progression to a higher level. Formative assessment is on-going, occurring throughout the course and is aimed at giving regular feedback on both student learning as well as teaching effectiveness. In practice, both may use similar methods of assessment (e.g. an assignment), but their aims are different. Assessment is integral to teaching and should be valid, reliable and fair.

A valid assessment is one which is appropriate (given the context) and actually measures what it is supposed to measure (wouldn’t that be nice?). For example, it is futile to just have a written, theory test for an essentially practical skill (like driving or skydiving). It should also test to a reasonable depth and breadth; making a multiple choice test so easy that it is possible for anyone to guess the right answer simply because of the ludicrousness of the alternatives clearly tests nothing. It should also be fair; that is, not biased towards any particular group or individual (perhaps because of previous performance or attitude) – every one has an equal opportunity. Reliability simply means that it is consistently capable of producing the same (or similar) results given the same circumstances. Thus, for example, different examiners, using the same criteria should give broadly similar scores for the same piece.

It is worth saying that these are not absolutes. Indeed, no assessment is absolutely valid, reliable and fair (as many students would agree!).

In terms of performance, it is worth noting two referencing extremes. Criterion referencing is simply where assessment is against fixed criteria (agreed beforehand). Thus, using fixed criteria, everyone or no-one might pass a test. The alternative is called norm-referencing. Though this may start out using performance criteria, the results are then statistically massaged (oops, did I say that – I mean normalised) to ensure a broadly similar distribution of grades in candidates from sitting to sitting (so a kind of quota system). Norm-referencing is though, inherently flawed and unfair, since actual candidate performance may vary considerably from year to year. Application of statistical massaging techniques also becomes increasingly invalid as the population sample size decreases. Another distinction which is worth noting is that between competency or non-competency based assessment. Now, I’d like to point out at this stage that the “competency” has nothing to do with the person doing the marking, whatever your views. Competency-based assessment means you are using criteria which assess the behaviours/attitudes, skills and knowledge actually required to competently perform a job (now there’s a novel thought!). This means that the person not only knows but can also apply; such assessments are typically workplace-based, observing the candidate in action. Non-competency-based is everything else.

There is an enormous range of assessment methods and variations within them. Reliance on any single form of assessment is not recommended (just like having the same political party in power for the last 10 years – hm, a little too close to the truth). A range of valid, fair and reliable assessment methods should be used instead. The principal, commonly used methods are summarised below (largely adapted from the references at the end of this article), together with some of their strengths and weaknesses.

Methods		Strengths	Weaknesses
Case study (variable amount of information given about a scenario; students have to analyse and assess the information in answering associated written questions).		Helps assess ability to apply knowledge, reasoning, understanding to a new situation, so is very suitable for high level cognitive learning.	Can be made too complex for the level of the learners and so be confusing (key data can be lost).
Collaborative project work (students work collectively on a project, rather than individually)		Useful where learning from each other is needed or where the project is too large for an individual.	Requires students to have a mature attitude. Can be too easy for the “few” to do all the work and the rest just to sit back.
Multiple choice questions(a bank of questions, each with a choice of answers only one (or combination) of which is correct).		Pretty reliable. Quick. Easy to mark.	Difficult to devise good questions sometimes; a common failing is lack of challenge, the right answer being too obvious because of the available choices – and so rendering it invalid and a poor discriminator.
Assignment/Essay (typically a piece of writing usually addressing a specific subject/question and with word count limits. However, the output of the assignment may well be a presentation, which would then probably be assessed, at least in part, by observation).		Suited to situations which also need to demonstrate research skills, sustaining a written argument (logical progression), presentation skills (if presenting) etc.	Since writing an essay (and also presenting) demands quite a few skills in its own right, it may be considered to be unfair (equal opportunities) if the subject taught does not actually require those skills to be demonstrated. Open to abuse (plagiarism and other forms of “help”) – so it may not be the student’s own work.Very time consuming to mark properly.
Direct observation (tutor observes student performing a task/activity. May involve formal recording of performance).		Particularly well suited to practical subjects, though often used informally to observe individual performance in group work. Immediate.	Mainly tells you about performance, rather than understanding. It can be quite subjective (and so unreliable) unless the assessment criteria are well defined and clearly understood by all assessors.
Oral question and answer (students are asked direct oral questions by tutor – usually during class but sometimes during a viva examination).		Immediate feedback. Interactive. Generally high validity.	Could be considered less reliable (questions may vary considerably in difficulty between students). Questions do need to be phrased properly and people not “victimised”. Answers almost never recorded.
Problem sheets (students work through written questions, usually individually, and these are gone through in the next class session, usually as a group. Assessment may be self or peer).		Very useful to practice application and review of cognitive-based materials. Helps identify areas of difficulty for individuals and group as a whole. A range of question difficulties can be devised which will challenge different levels of student learning (diversity).	May be time-consuming to devise.
Examinations (use a variety of written questions etc as already stated, but under timed conditions. Sometimes these may be set as “mocks”, or practice, to assess readiness).		Secure, highly controlled (invigilated); difficult to cheat. If well-designed, can discriminate between different levels of student.	Examinations require some skills in their own right (working under time-pressure etc) and, if the subject does not require this to be demonstrated, may not be considered entirely “valid” – discriminating in favour of those students that are more suited to exams.
Self-assessment (students assess their own performance. Used to complement tutor assessment).		Can aid transparency of the assessment process.	Must be clear on the assessment criteria. Student assessment of themselves may not be realistic or reliable.

As you can see from the table, assessment doesn’t have to be formal. Just because it is informal doesn’t mean it isn’t valuable. My experience is that much assessment in the training area is undoubtedly informal. That is, the results are not formally recorded. Even for formal written questions, the answers may be discussed in class without allocating a mark. Instead, the emphasis is more on informal assessment of understanding and rapid correction of misunderstandings. Thus, students are continuously given feedback about their learning and progress. This promotes learning in many ways. Firstly, it is highly motivating for students; if they get things wrong, they can be sensitively corrected (this may require a little attitude adjustment…) and encouraged to move on; if they get things right, they develop confidence to tackle even more difficult things. All of this feeds back into the trainer’s own training evaluation. If students have not clearly grasped a concept, this triggers a recap or use of other illustrations to make the point. Depending on the subject, you can also use such identified areas of difficulty to benefit the next student cohort – reorganising the emphasis of the lesson (and associated resources) to better effect.

Without some form of assessment, both students and trainers miss out. Assessment may be a dirty word but someone has to do it….

References:

This article draws upon the following useful introductions to assessment principles and methods, to which the reader is referred for further reading: Reece, I. And Walker, S. (2003), Teaching Training and Learning, 5th Edn., Chapter 7; Donley, J. and Napper, R., Assessment Matters in Adult Learning, (1999), available from National Institute of Adult Continuing Education (NIACE – www.niace.org.uk); ATHERTON J S (2005) Teaching and Learning: Assessment [On-line] UK: Available: http://www.learningandteaching.info/teaching/assessment.htm.

Back to top