Category Archives: Staff Quality Audits

Two is Good, Four is Better: On Inter-Subject Comparability.


What’s in a number?

In the old days it was all As, Bs, Cs, Ds… Now we have the 1 to 9 but it really doesn’t matter what you call it the idea that an A is an A or a 9 is a 9 is sacrosanct. A** or 9 or turn it up to 11, this is the bestest, most excellentest, the brightest…

When you see a report card and it’s all nines except in one subject it’s a six you think, oh dear, not so good at Maths or Art or whatever… but what if that assumption was wrong? Maybe the grade given by the teacher in that subject is a sign that the child is equally brilliant in that subject as the subjects in which the child achieved a 9… or is even more brillianter!!! Ridiculous, you’d say, it is clearly that the child is putting in less effort or is ‘just not cut out for that sort of thing’ but what if your assumption was wrong? What if nobody knew whether a 9 in Art was equal to a 9 in Physics and equal to a 9 in History? And even at GCSE… What if pupils make choices about what A levels to take and they drop a subject because they only got a 7 in it? Two As and a C for Uni not good enough, maybe, maybe not… And what about dropping subjects to take for GCSEs at option time? Let’s say your key stage 3 marking is based on some solid foundation like GCSE grades and a child got a 6 in French so decided not to take it, sensible?

Performance related pay… She always gets great results in Geography, whereas his results in Latin are always a couple of grades lower…

You would have thought all this was based on firm ground. The data doesn’t lie…

But no-one knows whether a 9 in D&T is equal to a 9 in English Literature. This is the issue of Inter-Subject Comparability and if you have ever assumed that the same grade in one subject is equal to that in another you have been mistaken, I repeat, no-one knows.

There is presently no requirement in our regulations for the exam boards to align subjects…

Although thinking has advanced considerably over time, we still see huge disagreements concerning how best to define and conceptualise inter-subject comparability, let alone how best to monitor it, let alone how best to respond to monitoring outcomes…

Ofqual 2015…

If you have a great assessment model in your school do you assume that the same mark in one subject is equal to that in another? If you feel that it is, how do you know? What decisions do you make if you assume all are equal? Should you review how these decisions are made if they are made on the assumption that all is equal?

Some subjects are more equal than others.

When Things Go Mad: The Destructive Power of Ideas

Over a hundred years ago, the German poet Heine warned the French not to underestimate the power of ideas: philosophical concepts nurtured in the stillness of a professor’s study could destroy a civilisation.  Isaiah Berlin
Yesterday I read a post by @teachertoolkit about a measure he employs in his school called ‘book look’. In his blog he wrote that the measure was to check on: “…marking, workload and quality assurance of feedback and assessment” As my main teaching subject is a practical one I thought I would ask the perfectly reasonable question:

Screen Shot 2015-11-26 at 08.00.47.png

My thought being that in subjects where much of the feedback, teacher, self and peer assessment is done verbally that instead of demanding those subjects should do unnecessary written work to fit in with the book look system, that the system might bend in a way to recognise the richness of this type of assessment.

Strangely, before I was able to receive an answer, this happened:

Screen Shot 2015-11-26 at 08.11.26.png

This was an odd diversion. Clearly Paul, who is an Ofsted inspector, had been into Teacher Toolkit’s school and had seen the ‘book look’ in practice, he implies here that he likes it. He also makes a leap from my tweet to imply that I meant practical subjects don’t mark. No matter, I had a right of reply:


Screen Shot 2015-11-26 at 08.20.45.png

And I expected it to be left there, after all, I was asking a question to which I hadn’t received an answer. My ulterior motive is to avoid subjects with a large practical component having to write everything down – where currently they are more effective by not having to log everything.

Screen Shot 2015-11-26 at 08.25.48.png

Now Paul has begun to question my understanding of ‘practical subjects’ – I have taught drama for twenty-five years, so this certainly ‘bristled’ with me. All I can think is that Paul is deliberately misunderstanding me because I’m asking about the practical component of practical subjects.

Screen Shot 2015-11-26 at 08.30.16.png

I know they all do written work. I am waiting for an answer from Mr. Toolkit about how they approach practical subjects, in other words do they want ‘everything’ logged in written form?

Screen Shot 2015-11-26 at 08.39.24.png

This is where things went mad and we went into meltdown. I had spent the day leading a course on the new exam specs for drama so Paul’s comment that I need to look at the exam specs was particularly hilarious.

The meltdown:

Screen Shot 2015-11-26 at 08.44.19

Screen Shot 2015-11-26 at 08.45.32.png

From this point the whole discussion went even further down hill, it is on twitter for those you who wish to see a spat about ‘you said this’, ‘you said that’.

There is something far more important going on here, the original book look idea is a bureaucrat’s dream but it falls apart when faced with practical subjects. Either it means that subjects with a sizeable practical  component will be asked to do more, unnecessary, writing or the system that has been brought in will miss the feedback given verbally and understood physically in, say, drama and PE.  Will this result in teachers at the school putting too much emphasis on written feedback and ignoring the richness of other ways of teaching and assessing?

Paul Garvey clearly thought I had no idea what I was talking about and the best I can think of him is that he misunderstood the nuance of my question. He seems to have a huge vested interest in ‘book look’. As a consultant he had visited Teacher Toolkit’s school and, as mentioned above:

Screen Shot 2015-11-26 at 08.11.26.png

has given ‘book look’ his blessing. As an Ofsted inspector his word carries power, I can see many schools jumping on the book look book check and adapting it for use in their schools. This would be a disaster for practical subjects and, I would venture, for all subjects. Read this excellent blog from Greg Ashman where he examines the thinking behind ‘book look’.

Since I pointed out the problem with practical subjects Teacher Toolkit has removed the bit of the blog where he wrote about checking: “…marking, workload and quality assurance of feedback and assessment” maybe because it doesn’t work for a large part of the feedback that is done in practical subjects. However, the tool doesn’t cover practical elements.

Screen Shot 2015-11-26 at 09.15.54.png

The ‘yellow box’ areas don’t help for practical work – how could a ‘diagnosis’ of books pick up the improvement in a girl’s ball skills in football or a boy’s vocal quality in drama unless it is written down? The point is the writing down of these things interrupts the teaching of these things in the subject, slows them down,  and, in all likelihood, lowers the standards of teaching and learning in the practical element of practical subjects. By writing about your practical work, you cease to truly understand it. It would be like teaching a child to walk by insisting she wrote all the stages down, set targets etc. when, in real life, the understanding of how to do these things is quite mysterious to us. Each child is different, they respond, in practical work, to coaching in real time, not to stopping and starting and writing about it.

That this check seems to expect teachers to spend a lot of time giving written feedback, and what is assessed will be what is done, I think the above book checking tool will mean that staff will have to spend more and more effort and time with written work and feedback and reduce the time and the quality of verbal feedback. This excellent blog by Andy Day reflects on how these checks distort education. What a pity  to put all the effort into written feedback especially if verbal feedback is a far more effective method of raising standards.

Ideas nurtured in the stillness of a senior leader’s study could destroy the quality of education in a school. Unnecessary writing of targets etc. is the bane of practical work please stop it happening in schools just so that a leadership team needing something to do can check it and Ofsted inspectors needing to justify their consulting business sidelines can tweet about it. Hopefully the book look tool will ignore practical work and, maybe, ignore books.