English Exam Fiasco

In the whole of this sorry mess which is the exam debacle in England nobody, to my knowledge, has mentioned the technical aspects of assessment, namely norm referenced and criterion referenced examination systems. From what I have read and heard, the June exams have basically been norm referenced, i.e. a fixed percentage of the total cohort of examinees are allocated to each grade so the grade pass mark will go up and down depending on the degree of difficulty of the exam and the severity of the marking. But in every year, for example, the top 10% of of the marks awarded will be allocated to A grade and so on.

However it must also be apparent that in previous years the exams have been criterion referenced. In other words if somebody sitting the exam gets a certain set of the questions correct, an A grade is awarded and similarly for each grade. No account is taken of the numbers achieving any grade. This must have been the case if more and more pupils were getting higher grades over the last 20/30 years as has been alleged. Norm referencing would not allow this to happen. For criterion referencing to be fair from year to year, a great deal of trouble and effort must go into setting very comparable exams year after year in terms of the criteria to be used and the marking must be very carefully moderated. In the case of the exams board in England this has not been happening, probably for commercial reasons.

To me it is obvious that the goal posts have been changed this year but the norm referencing has only been applied to the June results. This is effectively what was said this morning by the OFQual Regulator speaking on Radio 4 though she did not use the technical term.

The case for unfairness is not between the January and June results but between the June results and previous years when none if any norm referencing has been applied. The goal posts have been changed between 2011 and 2012, not within 2012.
This entry was posted in Uncategorized by robthill. Bookmark the permalink.

About robthill

I am semi-retired ICT Staff Tutor in Dundee, Scotland and an online facilitator. The views here expressed are my own somewhat quirky feelings about the world as I see it, no doubt just as prejudiced as most other bloggers.

2 thoughts on “English Exam Fiasco

  1. Great post. I missed the radio 4 interview but it seems to me that there are actually 3 referencings potentially going on:Straightforward, logical criterion referencing which was used in the legacy specification and which teachers use to mark essays and writing out of, for the sake of argument, 40. If you can do x, y, z skills, you get 25 marks. This could conceivably result in either 100% or 0% getting a certain mark. Norm referencing so that, as you say, a certain percentage are permitted to obtain a*, a, b etc. which Ofqual are denying has happened. But Ofqual are saying that grades have been referenced against the results of 2011’s legacy specification in something called "comparable outcomes". Now I am a bear of very little brain so I don’t fully understand this but it suggests to me that somehow 2012 unitised results have been referenced against 2011 legacy results. At this point, my head starts to ache, the vein on my neck bulges just a little too far! BUT what I do get is that, whether Ofqual, AQA, OCR of whoever are norm referencing or comparable outcome referencing, the results are being divorced from criteria. A C grade student from a June submission has a different level of skills than a January 2012 or a June 2011 legacy student. How do HE providers manage with disparate levels of skill that the grade does not differentiate?Head aching on last day of holiday now.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s