A topic that we discussed at the last OPS Council meeting was the thorny subject of judging! This year, for the second time, we had the annual competitions judged by a team of three judges (Ian Aldcroft, Gordon Jenkins and Dianne Owen).
The scoring machine
Also this year, we used John’s scoring machine, which he designed specifically for the purpose along with writing the competition software. We also use the machine at our Impact Group meetings.
The scoring machine has three handsets, so that each judge can score each image presented to them. Customarily, a scoring machine has 4 buttons, marked, 2, 3, 4, 5 respectively. The 1 score is not used in this country, though in some countries abroad, the judges mark 1-5 or 1-9. A very little arithmetic tells us that the images here will be scored from 6 to 15.
The signals from the score pads are sent to a laptop computer. The competition software calculates the total score for each work and writes it to a file on the hard drive. The score is called out by the computer operator.
How it all works
Beforehand the judges have been briefed. A typical briefing will be “Use the 5 button if you think the image is especially strong/if you want the image to receive an award”, “use the 4 button if the image is above average standard/you want to look at it again” , “use the 3 button if you think it is average/OK”, “use the 2 button if the image is weak and you definitely don’t need to see it again”. Depending on what the organisers want, this briefing may change…in the case of choosing an exhibition, 5 may mean “look at again” 4 could mean “include in exhibition” 3 could mean “not quite exhibition standard” and 2 “poor”. For this reason, you cannot compare in any way a mark out of 10 given by a single judge to a mark out of 15 from 3 judges. The 3 judges system could equally be scored with ABCD, where AAA is 3 top scores and BCC is below the score to see again. Some AV judges used letters rather than numbers to score.
Using three judges has the great advantage that individual preferences and even prejudices in respect of subject matter, treatment etc, should be diluted, giving a fairer mark for the work. On the downside, it does mean that your image needs to communicate immediately to the viewer…the shy violets that need time to be digested and appreciated, anything that doesn’t have immediate impact, will tend to suffer under this system, because the judges will make up their minds in the first 3-5 seconds in many cases. The quicker the score comes up, the quicker you realise, as an entrant, that they have either got the point or missed it!!!
Depending on the proportions of each score at the end of this first run through, the top marked images will be held back. Lets say we want about 20% of the entry to gain awards and there are 25% gaining a 12 or over, the judges will be asked to review the 12s and above and pick their awards from those. If only 10% have scores of 12 and above, the 11 scores will also be counted in, or the number of awards may be reduced to 10%. Scores of 10 rarely get a second look. Only if your work gets that second look does it stand a chance.
The negotiation
It is at this stage that the three judges negotiate…when they review the work they will either quickly come to a consensus about the winners, or they will negotiate and the stronger wills may prevail. The winning work will be the one that all three judges can live with – it may not be their personal preference, but if all three have a different preference, they have to compromise. Most experienced judges are used to doing this and don’t find it difficult. Only occasionally do you get a judge who tries to coerce the other judges into thinking their way. Once the winners are chosen, the remainder of the work will be given VHC, HC or commended, or no award at all, as the judges wish.
Something that is often a puzzle to relatively inexperienced members, is that a picture that received a score of 14 may not end up amongst the winners and a picture that scored 12 might…this is because the judges will score initially on content and impact…when they get to see the work close up (and this especially applies to prints), they may see a fault that had not been noticed before, or just find the work doesn’t have lasting appeal, leading them to discard the work at this stage and choose something that received a lower score but looks better or whose impact is more enduring on close-up appraisal.
The audience point of view
Advantages:
The audience can see how higher level competitions (regional, nationals, internationals) work. Almost all higher levels competitions run this way.
The judges can get through a lot more work on the night. PI and prints can be judged the same evening/day.
The system is thought to be fairer and more balanced.
Disadvantages:
There is no feedback given about the unsuccessful images…only the awarded images receive comments afterwards.
It’s not the best of entertainment.
Discussion:
There is a school of thought that says that images entered into our annuals should have been seen beforehand by club judges or at an internal appraisal night, allowing sufficient feedback to the individual photographers on their work. Work that is newly entered into the annual can afterwards be entered into quarterlies or appraisals to gain feedback, so, there are plenty of times when appraisal can be given outside of the annual judging.
The scoring machine system does enable a huge through-put in an evening. Without it, we would probably return to the former system of having one judge for the PI and one judge a different night for the prints….though in our case it would have been a bit of an insult to ask a judge to come any distance to judge our 24 print entry this year. In the case of the projected images, we would have had to limit the number of entries as there were about double the amount a single judge should be asked to comment on in an evening.
As far as the audience is concerned, the three judge system does lead to pauses during the evening whilst the judges are conferring and that maybe doesn’t make for a very entertaining evening. Some clubs do their annual judging outside of a club meeting, maybe on a Saturday or a different day of the week, with only the competitions team present and this could be an alternative for us.
It would be interesting to get some feedback from our club members as to how they would prefer the annuals to work. Please click on “leave a comment” below and don’t forget to sign in if you are already a Blog member so your comment won’t have to go through Mike or me to be moderated before appearing on site.
Chris Widdall