Good morning, everybody. Welcome to this morning's meeting of the Education, Culture and Sport Committee. I extend a particular welcome to Irene McGugan, who is joining us for the first time. There will be more of this to come, Irene, so you must have been very wicked.Our first set of witnesses this morning are members of the teaching trade unions. We are grateful for your attendance and for your written submissions, which we have had an opportunity to examine. I intend to open up the meeting to questions immediately. As ever, we will be pressed for time. I will first ask individual members of the committee to ask relevant questions. There may also be some follow-up questions. If members would like to intervene and I am not looking at them, they should wave at me—I will try to bring them in where possible. We will kick off with higher still development.
Official Report · Committee
School Exams
Speakers
Choose an MSP to read only their contributions- David Elliot Former Director of Awards, Scottish Qualifications Authority 57
- Michael Russell SNP 46
- Ron Tuck Former Chief Executive, Scottish Qualifications Authority 41
- Ken Macintosh Lab 38
- Mary Mulligan Lab 35
- Brian Monteith Con 34
- Alex Easton Headteachers Association of Scotland 29
- George MacBride Educational Institute of Scotland 21
- David Eaglesham Scottish Secondary Teachers Association 18
- Jamie Stone LD 14
- John Kelly National Association of Schoolmasters and Union of Women Teachers 13
- Ian Jenkins LD 13
- Pat Cairns Headteachers Association of Scotland 13
- Margaret Nicol Educational Institute of Scotland 12
- Cathy Peattie Lab 11
- Nicola Sturgeon SNP 10
- Johann Lamont Lab 10
- Andrew Shanks Educational Institute of Scotland 5
- David Eliot 1
The record
422 contributions · showing 1–200What were the effects of the rate of introduction of higher still? What are your thoughts on the advisability of the manner in which it was introduced? In previous meetings, the committee has touched on the issue of phasing. This is a key area for the committee, and I would be grateful to hear your comments.
All the teacher trade unions were deeply concerned about the initial timetable for the introduction of the higher still programme. We argued strongly for and welcomed the postponements that were implemented under the Conservative and Labour Governments that were responsible for the programme before the Scottish Parliament was set up. The Educational Institute of Scotland was deeply concerned that there was pressure on teachers in Scottish schools to implement the programme before they were ready. We were pleased by the decision to phase in the programme in those subjects where people felt that it could not readily be implemented during the first year of higher still. We welcomed the fact that teachers were not put under pressure to do that.We felt that phasing in a more general sense—piloting or introducing higher still for one subject before it was introduced for another—was inappropriate for a number of reasons. If a programme is piloted in a particular subject, that means that more pressure is put on teachers of that subject than on their colleagues. If the higher still programme had been piloted in particular schools or establishments, that would have led to considerable difficulties, because higher still is aimed not solely at young people in school, but at colleges and people in adult and community education and other forms of lifelong learning. It would have been difficult to have had any meaningful form of piloting.
I do not disagree with anything that George MacBride has said. One of our concerns—and I am not sure whether this is political with a large or a small P, because it was the case under both Administrations—was that there seemed to be a reluctance to accept the professional judgment of teachers that the programme was not ready to be implemented. There was a political will that the timetable should be met. When Brian Wilson was minister of state at the Scottish Office, he was heard at one meeting to say, "Why am I hearing this for the first time?" There was a reluctance on the part of Her Majesty's inspectors of schools and, possibly, the higher still development unit to give ministers messages that they might not have wanted to hear.
I echo what John Kelly and George MacBride have said. From our point of view, the key to this is the extent to which the advice—the virtually unanimous but separately arrived at advice—of the teaching unions, representing the whole profession, was ignored for what can only be regarded as a narrow political purpose. Like John Kelly, I will not say whether that was political with a capital P. Teacher unions were castigated for being luddite and for refusing change. We were seen as reactionaries who were holding up the process. However, all the time we were totally right, as we were reporting back what practitioners were saying in the classrooms—that there were fundamental problems.There was no way in which we could know the detail of the difficulties that finally arose at the Scottish Qualifications Authority. Had we known that detail at any point, we would certainly have said something—we would have been culpable had we not said something. We cannot pretend that this is all about one episode. It is quite clear that there was a political determination that higher still should go ahead, that it should go ahead in a particular way and in a particular time frame, and that nothing should stop it, least of all genuine reporting by the profession of serious problems of implementation. We made the point repeatedly that no teacher in Scotland would have benefited by a penny from delaying higher still. However, we were still told that we were putting our narrow interests ahead of the interests of the pupils. We said repeatedly and in public that the unions and the members whom we represent were putting the interests of the pupils first.
Are you suggesting that there was what verged on a reckless haste in implementing higher still, despite warnings from the unions?
I do not know whether the haste was reckless or whether it was otherwise motivated. However, there was a determination to implement the programme within the agreed time scale, regardless of any objections.
Do you agree that there was undue and possibly damaging haste?
There was haste that eventually proved to be damaging. It created problems for schools when they came to decide whether they should go ahead with higher still. It would have been far better if we had been able to proceed on a properly planned, fully resourced and fully ready basis. That was not the situation in which we found ourselves. Schools had to make difficult decisions about whether to present in higher still or in the traditional higher.
I would not use the adjective "reckless", but undue haste was an important factor. Many teachers in schools and members of all the teacher trade unions perceived a determination to push ahead with higher still. We associate that with the general culture in the management of Scottish education, which until now has operated on a top-down model where decisions are made without involving teachers in much of the detailed planning. That led to many of the practical difficulties, and we had to push extremely hard for the additional resourcing that finally underpinned the higher still programme. Even when Brian Wilson, as minister of state at the Scottish Office, was announcing the postponement by a year of the introduction of the higher still programme, leading members of the Convention of Scottish Local Authorities clearly disagreed and argued that teachers, as employees, should implement it with undue haste. We would therefore not identify any political party, but we would identify a political culture in Scottish school education.
I am anxious to get to the heart of what you think has or has not been going on with HMI. We heard that HMI was perhaps not carrying messages back to ministers, although it should have been. Can you expand on that and say why, if that is the case, that did not happen?
In our written evidence we have indicated that, at meetings with ministers, we were presenting our concerns. Certainly my organisation did so, and I imagine that other organisations did likewise. Initially, there was a clear trend to dismiss those concerns as unimportant and not time constrained. As time went on, it became clear that ministers were getting conflicting advice. They were getting advice from us saying one thing and advice from other sources—HMI and presumably others—saying very different things. It became clear from casual conversations with ministers that they were puzzled about which advice was correct. They were convinced by what they heard from teacher trade unions, but they were also being given advice by their usual sources. We were very much at odds. I remember a particularly vitriolic meeting in Dover House, at which one of my colleagues was challenged about the accuracy of the points that he was putting across on the higher still developments. We vigorously reinforced those points, but it was clear that the minister had a real dilemma as to which line was tenable.
During the first year of implementation, we found that HMI, quite rightly, picked up the positive view of what was happening in schools, but seemed to be failing to pick up the problems and the more negative aspects, particularly in relation to internal assessment. HMI's view did not reflect the reality that most of us found in schools. Although it has been picking up curricular issues in school inspections, it appears to have failed to pick up the administrative difficulties. To that extent, we certainly have concerns.
As a general observation, the problem is not with personalities in the inspectorate but with the role of the inspectorate as it has been constituted in the past 10 years. In our evidence on the Standards in Scotland's Schools etc Act 2000, we pointed out that the inspectorate is now both the generator of policy and the policeman of policy, which cannot be right. If the inspectorate is pushing higher still—and it could be something else tomorrow—is it the best-suited body to listen to and represent the problems that might occur in implementation?
In the Scottish Secondary Teachers Association submission, one statement seems to sum up everything about the implementation phase. It says: "Principles dominated practicality throughout the process".Can you give examples of that? I think that everyone accepts that the Scottish Vocational Education Council modules were not doing the job and that intermediate 1 and 2 would have been a new development with status, but there were practicalities involved in the unitisation and administration. How did principles dominate practice in those areas?
I was referring to something more general. There seemed to be a principled determination that this was some kind of flagship programme that had to go ahead. We were unclear why a programme that was devised under one Administration and taken up under another Administration should suddenly become that second Administration's flagship programme. It seems that there was multifaceted ownership of it. It was important—nobody was under any illusion about the fact that it was a vital programme that had to go ahead. However, the principle seemed to be that it had to go ahead in a certain time frame and on a certain basis and that, if it did not, the Administration might be seen to have failed. We did not think that the Labour Administration would be failing if higher still was suspended or postponed for a further period to enable it to be implemented properly. In that sense, our comment was not a political reflection. However, there seemed to be a political determination that it must go ahead; perhaps it became the educational millennium dome, which is unfortunate. The practicalities at the time were that there were problems with the delivery of materials. At times we were being told things that were simply untrue. In the experience of classroom practitioners, the materials were not arriving in schools, but we were being told that they had arrived. That was a difficult situation. Our members were reporting that the materials were not there, which meant that we could not progress. However, the principle said that higher still was so vital that it must go ahead. We agreed that it was vital and that it had to go ahead, but not in that time frame.
To borrow the SSTA's phrase, we believe that the principles that underpin the higher still programme are important ones that should be realised in Scottish education. They build on the success of our comprehensive schools and seek to promote social inclusion by bringing in those who are socially disadvantaged. They seek to recognise progression, by building from standard grade through higher still and into higher level qualifications. They seek to ensure that adults have access to further education and they break down the traditional divide between academic and vocational education. All those principles are important and should be sustained, whatever the outcome of this and other inquiries into recent events. We strongly agree with the SSTA about the issues that David Eaglesham has just described. Those principles were understood and promoted even when there was clear evidence on the ground that there were practical problems that were not being rapidly or efficiently addressed.
I do not dissent at all from the views of my fellow witnesses. Higher still has been seen as a significant advance in Scottish education in terms of social inclusion and recognising the worth of young people for whom the higher was not appropriate. However, if we are talking about principles dominating practicalities, we need look no further than the continuing failure to implement the new higher in English and communication. We all agree that there should be no bar between the academic and the vocational, but perhaps there should be two separate highers: higher English and higher communication. The profession has been saying that for six years, but it seems to go against the political mantra of equality of recognition. We have a political one-size-fits-all straitjacket, but one size does not fit all. However, when teachers have said that, we have been pooh-poohed.
There seems to be an idea that all courses ought to look the same, with the same number of boxes and units. What are your views on that? People have mentioned the SCOTVEC culture, with units being done sequentially. Will you comment on the shifting of the sands when the higher still development unit eventually recognised that concerns were being expressed and changed the times at which units could be done? We might have thought such developments okay if they had been done before, but shifting in the middle of the year can cause difficulties.
Substantial changes in the middle of a course would have caused considerable difficulty. In my school, colleagues in different subject departments had already adopted different models for delivering higher still courses. Although most departments were doing it sequentially, some—for sound professional and educational reasons—were doing it on a parallel basis. They believed that the timing of assessments was, to a considerable extent, under their control, but the immediate issue was the number of internal assessments, which was in many subjects far more than the number of units, as there could be several assessments within one unit. The pressures on young people to pass those assessments, the need to reassess where a young person failed to achieve a cut-off score, and the knock-on effects on teaching and learning caused substantial difficulties, which must be addressed in reviewing higher still. Even within the programme, teachers had some freedom, but we clearly would not want courses to be totally rewritten once a teaching session had begun.
I would like to pick up a number of points in the SSTA's submission; others will no doubt want to comment too. Mr Eaglesham, at one point in the submission, you refer to two specific meetings with HMI and ministers—one in January 1998 and one in May 1998. You say that, in January, HMI's reaction was "patronisingly dismissive" and that, in May, it was "aggressive and ill-judged". Will you expand on those comments? What points did you put at those meetings and what were the reactions?
At both meetings, we presented the same argument—that higher still was evidentially not ready. We pointed out episodes in which material should have been delivered and prepared but had not been. Our line did not change much between the two meetings, except that further evidence was available in May that had not been available in January.In January, the attitude of the advisers was that there was not really a problem. That was not the attitude of Brian Wilson, the minister at the time, and it was interesting to watch the body language and the reactions. The advisers' attitude was that, if we stood back long enough, the problem would resolve itself. They had the confident attitude that says, "There is no problem, and what you are reporting is insubstantial tittle-tattle that will ultimately have no effect on the process." By May, they came—I think that this was the expression that I used at the time—loaded for bear. We had scarcely got in the door when one of the inspectors was attacking my colleague and saying, "This is nonsense, this is ridiculous, this simply is not true." My colleague gave a very good account of himself, and I would like to pay tribute to him for being absolutely accurate. I am not currently a practising classroom teacher, and I confess that sometimes I have difficulty in knowing exactly what is happening. I bow to my colleagues here, who are practising teachers and therefore know what is happening on the ground on any day. My colleague at the May meeting was a practising teacher and knew exactly what was and was not happening. He had also been involved in all the higher still developments. He was directly controverted by HMI at that meeting, and one could see in the minister's face a genuine difficulty. My colleague came across plausibly and what he said seemed to be sound and accurate, yet the directly opposite view was being put by HMI. The minister had a real dilemma.
You say that the minister had a dilemma, and you say in your submission that he was receiving conflicting advice at that stage, from HMI on one hand and from professionals on the other. Whose advice was he following?
That question is difficult to answer. I suppose that the person to ask would be the then minister. My view, from informal conversations, is that the minister had become convinced that there was a genuine problem that had not been fully addressed and that steps might have to be taken. Of course, at that point, a new minister came into post, who would receive the same advice as had been given to the outgoing minister. There was a transition period, which was unfortunate—although not because of the two people involved, for both of whom I have the greatest respect. The timing of the change between ministers made things more difficult. Had Brian Wilson continued in the post for the rest of the year, I imagine that there would have had to be some kind of reaction. However, it is understandable that the new minister with a new brief would take advice from the appropriate advisers. It took a little time—I think until October—until we were back putting the same problems to Helen Liddell. By that stage, my colleagues will recall that we were into animated discussions over whether things would or would not go ahead, ballots on boycotts, potential lockouts from meetings and all sorts of other things. Everything was very public, and it was difficult to disentangle ourselves from those issues. There had been an opportunity in the summer, and it is unfortunate that that opportunity was lost. That happened not because of anyone's deliberate actions, but because of the reshuffle. I do not think that the reshuffle was connected in any way with these events; it was just a by-product.
The common factor during the period was HMI and the officials who, I presume, continued to give advice that was at variance with the experience of classroom practitioners.
I know of no evidence that suggests that HMI's advice changed over that period. We detected no change.
It is one thing to identify, and to be proved right on, defects in the implementation of higher still; it is another thing to prove any causative effect on what happened during the summer. Others may want to comment, but to what extent did the problems that you flagged up on the implementation of higher still contribute to the chaos of the exam results and issuing process this summer?
It would be nice to be able to draw the audit trails together and say exactly what happened at each stage—that is clearly what this committee is trying to do with the evidence that it is taking. To be frank, I cannot answer your question. All I can say is that, had the advice that we and other unions were giving at the time been listened to, the problem would not have occurred as it did occur, because the development of higher still would have taken place in a different time frame. For example, the merging of the two computer systems would have gone ahead, but not in hot pursuit of higher still. It is also possible that some of the assessment issues would have been resolved, which would have lessened the burden on our colleagues in the SQA.It is impossible to follow an audit trail directly and say that, if a specific something had happened in August 1998, the chaos of this summer would not have happened. As I said, we would have been culpable if we had had such knowledge but had failed to apply it to the benefit of young people in Scotland. Although people have been suggesting in the press that it was known all along that this was going to happen, I do not think that anyone in my association knew that. I would be very angry if people had known it but had not done anything about it.
With the benefit of hindsight, do you feel that, if some of your concerns had been listened to and acted on, it is at least likely that the crisis this summer might have been averted?
The impact of the crisis might have been less had some our concerns—and those of other colleagues—been listened to.
Nicola Sturgeon's last question was on a point that I wanted to explore. We may take evidence from the previous ministers at some stage, but I understand that the transitions between Government and Government, and between minister and minister, were almost seamless as regards the advice that people were receiving, and that that advice has contradicted the advice that you, as union representatives, were giving on the introduction and implementation of higher still. Now that we can look back, and given the consistency of your advice, do you feel that any of your warnings might have made a difference? We are aware that data processing at the SQA was a significant factor, and we are aware of problems—which have been consistently pointed out by you and others—with assessment, marking and the recruitment of markers. Would you like to take this opportunity to say that some of the points that you have raised during the past two or three years could have taken pressure off the SQA? What seems to come through—although we have still to establish this—is that all the contributory factors were adding to the core problem.Before I am criticised for asking a four-minute question, let me just ask this: could any of the points that you raised have made a difference, if you had been listened to?
If witnesses could keep their answers shorter than that question, we would be grateful.
We will try. As far as we understand them, the issues that arose this summer were largely operational. Like David Eaglesham, we could not, and did not, predict them. The EIS was aware of some continuing difficulties with data processing, but nothing more than that. I appreciate that I am speaking with the benefit of hindsight, but we would say that the clear difference between previous years—when the Scottish Examination Board, SCOTVEC and the successor body, the SQA, operated as efficiently and effectively as they had ever done—and this year was the additional volume of data that arose from internal assessment. Had some or all of our concerns about the burdens and pressures of internal assessment on young people, teachers, schools and, as it turned out, the SQA been listened to, we believe that that would have been a significant factor in preventing this year's difficulties.
You said that the SQA could not handle the volume of data that was generated by internal assessment—a point that has been made by many people. However, you say in your submission that teachers could handle it and that schools could handle it.
Teachers handled it, at some cost to themselves and possibly also to their on-going work. Senior staff in schools would input data in various formats, and teachers at all levels would have to input the data again, either for whole groups or individual youngsters. They would often then have to check with their colleagues and principal teachers, who then might have to check with class teachers on the exact status of the data.There was considerable repetition of input and correction of data, which were then not carried through the SQA systems. Pressure was put on administrative staff in schools, who had to be diverted from other tasks to input data to computer systems.
It would be wrong for me to speculate on the processes within SQA that caused the difficulties, but there is no doubt that they placed pressure on employees at all levels within schools.
Mr Shanks, as a principal teacher, will you tell us how the administrative burden and assessment procedures of higher still impacted on the rest of your teaching and on the other classes that you taught? How did it bulk in your work load?
It had a huge impact. I was one of the few enthusiasts who implemented higher still English and communication this year. I believed in it and thought that, as my colleagues have said, it was a good thing for the pupils of the school. It became clear very early on that the amount of assessment that was involved in internal unit assessments would be burdensome. Classes were large and we were dealing with a larger number of pupils because of the inclusion of intermediate 1 and intermediate 2. The marking took a great deal of time. The higher still assessments took a long time, sometimes an hour and a half, but critical listening took a great deal longer and individual presentation could take hours and hours. I had teachers who worked through their lunch hours and after 4 pm. I had pupils queuing up outside. First-year, second-year, third-year and fourth-year pupils were affected. Every aspect of teachers' work was affected by the burden of assessment for higher still. It has had an impact not just on the kids in fifth year and sixth year this year but on the quality and delivery of education by teachers throughout the education system.
That is precisely what I expected that you would say. If the results had been okay this year, the assessment procedures would have become part of the programme, which would have had a major impact. Is that right?
The burden of assessment needed to be examined. From early on we told the SQA that the amount of assessment was a burden. I think that all teachers, and in particular English teachers, would agree that the burden of assessment was almost too much and would have had to be reviewed to enable higher still to continue for another year. Many teachers were under a huge amount of pressure. Higher still could not have continued as it operated this year. There have been changes, which will reduce some of the burden of assessment. Certainly, there would have had to be a major review of assessment for higher still to run smoothly this year.
In response to Brian Monteith's question, I think that developments could have taken place outwith the hot pursuit of higher still. For example, the computer systems could have been brought together and there could have been trials of procedures. A number of changes occurred simultaneously, some of which could have taken place even with the previous diet of examinations. That would have allowed changes to take place and to be bedded in, which would make the introduction of higher still in the next year more straightforward. It was the overlap of a series of different things that led to some issues dropping through the gratings in the street.
David Eaglesham has largely covered the point that I had intended to make. The SQA got one thing wildly wrong. The introduction of higher still assessments at intermediate 1 and intermediate 2 resulted in far more candidates taking national exams, which required markers. I am not sanguine about what will happen in the current session when we get further into the implementation of higher still and more subjects come on stream.
I am anxious to move on. Mr Kelly has now raised the question of markers.
We are hearing a mixed message about higher still. To describe higher still as the millennium dome of Scottish education suggests that you thought that it was an expensive white elephant of the Scottish Executive rather than something to which you were committed. It was also described as a political mantra and a one-size-fits-all approach—presumably, the other side of the argument is that what we had before higher still was a one-size-fits-all approach from which many youngsters got no benefit.Given that, from what David Eaglesham said, there is a suggestion that some people were less committed than others to the idea of higher still, would it have been reasonable for ministers who were hearing conflicting advice to think that some anxieties arose from issues of principle rather than practicality? If that were the case, ministers would have been more likely to take advice other than that of the unions.
That is a complicated question, to which there are several layers.First—I hope that I will not misrepresent colleagues from other associations—I think that there is general support for the principles that underpin the higher still programme. Secondly, there are concerns about certain aspects of that programme, and in particular internal assessment. Thirdly, among all the teaching unions, considerable anxiety was caused by the perception that higher still was being imposed for political reasons and without adequate resourcing.Fourthly—this is perhaps the most important aspect of the imposition—there was a strong feeling among many teachers that they were excluded from any real debate, either about the principles or the practicalities of implementation. The reasons for that feeling relate to the culture within which we operated in Scottish education for much of the 1990s. The feeling led to the view that higher still was a political mantra to be pursued. That does not detract from the commitment of the EIS to the principles that underpin higher still. It might also lead to the situation that Johann Lamont described, in which mixed messages are sent to the Scottish Executive.
My association was committed to the principle of higher still from the very beginning, in 1994—indeed, there was a difference of opinion on higher still between it and other associations. Since then we have been committed to the principle that something like higher still should go ahead. It is important to articulate that the examination system at the end of secondary school should ensure that every pupil has equality of opportunity and access to every possible avenue for the development of their abilities.However, that does not commit us to any particular model of delivery. We have criticised the model of delivery that was chosen, although we have never resiled from the position that higher still was for the benefit of young people and should go ahead. Ironically, we are debating the higher aspect of higher still, although in many ways the most important aspects are intermediate 1, intermediate 2 and access. It is the articulation of that process as part of the upper end of secondary education that is crucial. We never moved from that view so there was never any mixed message to the Executive about the principle. We reported the practical difficulties that were involved. There seemed to be a political determination that the implementation of higher still should go ahead regardless. I likened it to the millennium dome because, regardless of the principle that celebrating the millennium was a good idea, there seemed to be a political determination that that celebration should take place in Greenwich, in a particular form. We endorsed the principle of higher still, but the practicalities were such that we had to be honest and report that it could not work in that form.
Is it reasonable to say that, despite the practical problems, the teaching unions and the teachers delivered? Of course, the fact that that was done at huge cost has to be examined with a view to modifying higher still. Despite the anxieties of the profession, it managed to deliver higher still. It was something that was not foreseeable that caused the problems in the end.
Mr Kelly, do you wish to respond?
I will not respond to that, but will develop the point that David Eaglesham made. I emphasise again that the teaching profession has been committed to higher still. Higher still in its current form has had a fair input from teachers. If we had not had higher still, we would have had Howie, which we all said was not the best way forward for Scottish youngsters, particularly at lower ability levels. When there was talk of delay, teachers said that they would do intermediate 1 and intermediate 2 first, because the higher suits the market at which it has always been aimed. That advice was not taken. I do not think that it can be said that the teachers were not committed to the principles of higher still.
Let us move on to marking. We have heard stories about timing issues and about unsolicited scripts being sent to teachers. I am interested in exploring the marking issues.
Cathy Peattie will know that as well as being president of the EIS, I am a member of the board of the SQA, although I do not appear in that capacity today. It is fair to say that marking would have been an issue regardless of all the other operational problems that arose this year, partly because higher intermediate 1 and intermediate 2 had to be marked externally. There is a growing need for markers. Should the winter diet be taken up, there will be an even greater need for them. Marking had been a problem that was raised outwith and within the SQA for some time. I think that the SQA recognised the problem. If the SQA wants markers and the Scottish Executive wants it to have markers, they must start to pay them properly.
There have been several statements about how much people were paid and about markers being paid less this year. Can you clarify the position?
I am a marker. My payment did not drop, but it increased only a little. The payment of marking is based on the time that it takes to mark a paper. There is disagreement about how the papers are paid for, and the system needs urgent review, particularly considering the stress under which markers were put.I was chairman of the subject panel, and in September 1999 it examined the reappointment of markers and setters as part of its remit. It became clear that more markers would be needed for the new diet, because of intermediate 1 and intermediate 2. As teachers, we knew through resubmission and through receiving inaccurate presentation lists that the SQA held inaccurate lists and that the markers were allocated on the basis of those lists. As far as I could see, the SQA did not speculate as to how many markers it might need. It planned only for how many it could prove it needed. In reality, there were far more papers in some subjects than markers had been allocated for.I think that the quality of the markers was good. However, the number of markers whom the SQA had contracted did not match the number of papers, so the normal rolling programme through which papers came into the board and went straight out to markers could not be followed. Papers had to be reallocated to markers who had already marked many scripts. If those markers could not take the number that they had been allocated, the papers had to be reallocated. Therefore, the time scale slipped. Markers were offered double the normal fee to take extra papers, and many markers took that up.
You talk about the quality of the markers. We have had mixed messages about that. You said that the quality was good; others have said that it was not good. Teachers and whole departments have stressed that candidates whom they had expected to achieve reasonable results did not achieve them. Does everyone agree that the quality was good, or is that simply your interpretation?
I know from my dealings with the SQA that the quality of the markers was high and equal to the normal standard. However, the time scale within which the markers had to mark the papers was significantly shorter than normal.
There have been many stories suggesting that the marking was not of such a high quality. We want those stories to be investigated. If it turns out that the marking was not of such a high quality as a result of time pressure, the way in which markers were recruited or the fact that quality assurance checks could not be performed—it would be speculation to talk about that issue—that will be a serious issue that requires to be addressed. It may be that, to some extent, some of the stories have been fed by media accounts that have not been entirely accurate. At this stage, we think that it needs further investigation.
Teachers that I have spoken to—I am sure that others say the same—say that they will never mark again. They talk about problems with timing and payment. How do we move forward? The current examinations diet started in June. How do we ensure that, this year and in future, there are markers to carry out the work?
To some extent, that is an issue for the Executive rather than the SQA. As Margaret Nicol has pointed out, the Executive must make a clear statement that funding will be available to ensure that markers will be adequately remunerated. The current pay levels are far too low to repay people for the stress and responsibility associated with the job. There must be a clear statement that the timing of the exam diet and the announcement of the results—matters within the control of the Scottish Executive rather than the SQA—will allow for marking and subsequent and accompanying quality assurance procedures to take place. That must include the more senior markers considering how markers have performed and the concordance procedures through computer programmes. Sufficient time for those checks and balances must be built into the time scale. That is a first and important step.As one investigates the events of last summer, it is crucial that the on-going work of the SQA is not diverted and is maintained properly. I am including the vocational qualifications in that work.
There are a couple of points that I would like to expand on. In our submission we were questioning the need for Scottish children's results to be out before those of English children—they all go through the same Universities and Colleges Admissions Service system. We are still dominated by the timetable for entry to universities. We have examinations for the whole of our cohort, regardless of ability. Just because, for the past 100 years, we have done something in time for the universities, there is no need to continue to do that. We need to consider how our system reports and the purpose of our assessment—that is a big question.On a minor practicality, it is obvious that people will not be attracted to giving up two and half weeks of their time—weeks of frenetic activity—for less than £8 per hour. That will be a big problem this year.
Several members want to ask questions.
I want to explore further the issue of quality. It is beyond doubt that, this year, there are question marks over pupil performance against predicted performance—there is a wealth of anecdotes about that. One of the explanations for that is quality of marking. Are there any other factors that might have contributed to that, such as the burden of assessment or the timing of the exams? Are there matters that are intrinsic to higher still that might explain in part why so many pupils appear to have underperformed in relation to their predicted results?
One of our great concerns has been the simplistic association of two elements—that performance is inevitably linked to poor marking standards. We refute that. We do not believe that to be the underlying cause. If there is a problem, its extent is not yet clear.One of the major factors to have impinged on pupil performance is the truncation of the examination diet. That presented young people with a series of examinations closely run together. Pupils might have sat two major examinations in one day and three examinations in four days. That was not the pattern in the past. The pattern tended to be a day of examinations followed by a day off. That is the practice that many of us have experienced in other contexts, such as further or higher education. There would have been a reasonable balance of time for pupils to prepare, sit exams, recover and then prepare again. That was not evident this year and has damaged the performance of some young people. They have been put under a huge amount of pressure and may have suffered because of that.The Education, Culture and Sport Committee must consider carefully the overall impact on pupils. Higher still has been a political football all the way through. Young people do not live in a vacuum—they live in the real world. I am sure that all of us have been lobbied by young people about higher still. Throughout the process, young people were acutely aware that there was something different and unusual about the situation. They knew that it was a problem because it was in the newspapers and on the television. As they saw it, they were the guinea pigs for higher still. They were not. The teachers and the professionals made sure that pupils were not treated in that way. However, the pupils' perception was that they were being treated in that way.The examination period was narrowed down and that put even more pressure on young people. It is to their credit that they have done as well as they have in the circumstances. The committee should bear that in mind, because its responsibility is to ensure that the best interests of young people are served.
If one looks at the results overall, a major factor has been the number of appeals. There has been a huge increase in the number of appeals, although the percentage of successful appeals is not so different. That can be explained by the fact that the SQA did not run its usual concurrence checks. In effect, in schools with a proven good track record of estimates, the SQA carries out a sort of appeals procedure before the results are issued, so that pupils are much more likely to have a grade that is consistent with their estimated grade. That was not done this year, which would have had a major influence on the difference between estimates and performances.
Margaret Nicol has touched on an important point: the holy grail of concordance. For many years, schools have been told that they must be accurate in their estimates, for the very good reason that if the child underperforms on the day, they can be accorded the performance of which they were thought to be capable. In the past few years, schools have been told only once how concordant they were. That is information that they need to get every year. The information that comes back from the SQA must be more transparent. If I, as the man in charge of a certain subject, am consistently wrong in my estimates without knowing it, I am doing a disservice to the young people in my charge. We need that information.
I have a slightly different point. One of the things that we must bear in mind is the fact that the procedures of the SQA and its predecessors have resulted in a highly sophisticated and ambitious system that requires only a few things to go wrong for the whole thing to crumble, because of the knock-on effects. The standards of the SQA—I am referring to the way in which it is organised rather than its exam standards—make it one of the most sophisticated examining bodies in Europe.There is an issue in the higher still programme about the relationship between internal and external assessment. Internal assessment is on a pass-fail basis—a candidate either achieves a unit or does not. External examination is designed to grade youngsters in terms of an A, B or C pass. That level of demand and sophistication needs further consideration.
I want to explore a different element of the marking issue. Every time I carried out marking I said that I would never do it again, so I am entirely sympathetic with people who feel that way. The pressures that markers were under this year were phenomenal.I am interested in the inability to react to the concerns that people were raising. That has been a feature of other evidence that we have taken. It is clear that your subject panels have been expressing concerns about practical issues, such as the need for more markers. As it became evident that there were problems with the markers, what efforts did the unions make to be heard on that matter? You have said that the board only meets four times a year. Were there efforts to push your concerns about this at board level? Or was the pattern repeated that we have heard elsewhere, which was that people raised questions, received reassurances and nothing changed?
Yes. The EIS raised those issues in several forums. First, we had meetings with the SQA. It was clear that we were raising the markers issue—there is a minute from 27 March. We contacted the SQA because of our serious concerns about estimates, grades, non-transmission of materials, marking and moderation, which was also a problem last year.We also raised those issues within the higher national liaison committee and the assessment focus group, which the SQA set up to address those issues. It is not fair to say that nothing happened. Questions were asked, solutions for this year were given and ways ahead for next year were identified. We accepted that some of the problems existed because this year there was a changeover to the awards processing system. Some of those problems existed because the system was new, and they would not continue when the process was bedded in.We also addressed problems that were not going to go away, such as the need for markers and more moderators. Those problems will not go away. Those were all raised and the SQA took away a commitment to address them. Questions about the need for markers and the transmission of information turned out not to have been addressed.
Would that minute have appeared before the board of the SQA? At what level were those alarm bells being rung? It seems to be a simple, practical point. There were three times more exams to be marked, but the appropriate number of markers are not being taken on and they are being taken on late. I can understand that different groups discussed it and received reassurances, but did somebody go to the SQA board and mention, or have in front of them, evidence of the unions' strong concerns that this was going to be undeliverable if there were not enough markers?
The alarm bells would have been rung through the assessment focus group. They were being rung in the national qualifications committee and they were being rung at the board. In the minute of the board meeting on 27 June there is reference to the difficulties that there had been in marking, the fact that new markers had been identified and the statement that the remaining papers that needed to be marked were to be marked shortly. The board was being given reassurances. That is specific; it is in the minute. It can be checked easily.
If those concerns were being raised over several months, were you, as a board member, raising those matters consistently at board meetings? The minutes do not reflect that.If you were being given reassurances such as those that appear in the minute of 22 June, were you reassured by what you were being told? It must have been at variance with the concerns that teaching colleagues expressed to you.
It was not at variance—the reasons were at variance. The information that we were getting from schools was that information was being submitted and not being recognised by the SQA. From the SQA's point of view, the problem was perceived to be that schools had not submitted the information. We were being reassured—as late as 22 June, which was the last time that the board met before all this took place—that the entries were coming in, steps had been taken to identify people who would be in touch with the SQA during the holidays and it was confident that the information would be in in time for the completion date of 10 August.
What I am trying to get at—it is the same point that I pursued with Ann Hill at the meeting of this committee on Monday—is how vigorously those concerns were being pursued at the level of the SQA board. We have heard that reassurances were being given, yet we have heard from your union and other unions that teachers were continuing to raise concerns. Were those reassurances being accepted at face value? Was there vigorous discussion of those matters at board meetings? It came out at our previous meeting that the minutes of the board meetings do not reflect that those issues were being pursued vigorously.
How vigorously can you pursue a question of what is happening to the results—
Issues such as that are not mentioned. Apart from marking being mentioned on 22 June, other concerns that were being raised do not feature in the board minutes.We were told by David Miller that that was simply a fault of the minutes. Do you agree with that?
I would have to agree with that. The reports that were given were often oral. The problems were often identified by the SQA officers. They said, "We have had this problem and this is how we are going to resolve it." Board members did not always have to ask the question. I am satisfied that no question that I could think of that could have been asked was not asked. I am also satisfied that no answer was not pursued that I could have pursued. I am confident that it was not just me—other members of the board felt the same way.
I will ask David Eaglesham this question, which is not on markers, but on a similar subject.In your submission, you say that the SSTA met the SQA—Margaret Nicol said that the EIS also met the SQA—and that there were follow-up meetings. The chapter in your submission on administrative problems says that the SSTA met the SQA in April to air concerns and that"At a subsequent meeting in June, we became aware that further problems had arisen".Did the SQA assure the SSTA that the concerns that it had voiced at the April meeting had been resolved by June? That seems to be the pattern. Were you happy with the reassurances? When the SSTA met the SQA in June, were those concerns repeated or was it accepted that the problem had been resolved?
There is a problem with the audit trail of each issue that members raise. When we met the SQA in April, we reported a series of SSTA members' concerns about data transmission. We met the SQA to explore the problems surrounding that. The SQA informed us of the procedures that it was following to deal with those problems. We were satisfied that those measures seemed to be right. We agreed separate releases timed to be at the same stage. People did not tell us subsequently that measures had been ineffective in specific ways.It would have been good to be able to follow each concern that was raised in early April to find out whether the problem had occurred by late April. However, it is not physically possible to do that and there was no climate of suspicion in which we would have been led to do that. In the normal course of events, matters are raised with the appropriate authority, which then takes the necessary steps. We did not repeat the same questions and we were not returning to the SQA to say that problems had not been sorted. Each time that a set of issues came up, we were able to address them. We were as reassured as we could have been that the matter was in hand and that concerns were being dealt with.In June we were aware that there was a problem, but that it was reducing in size and would be minimised. Although there was still a problem, it was not of such magnitude that it would lead us to require that anything further be done. The problem was with data entry and the matching of two bits of data. We cannot follow the audit trail for each individual member's inquiry.
David Eaglesham is saying that there were a series of different problems, which were difficult to trace.I will go back to questions to the EIS. Everybody agrees that the problem with markers was raised in September 1999—or earlier—and that it was raised continuously. It seems to have been a problem throughout the period. I understand that a series of individual problems were being resolved, but that the problem with markers was continuous. Why did you accept the reassurances that you were given every time that you raised the matter, when your members told you that the problem had not been resolved? I do not understand that.
Clearly, the recruitment of markers was an on-going issue, but it is important to bear in mind that that would not impact on many teachers in schools, because they would not be aware of difficulties. A person who had been recruited in the past as a marker, but who had not been recruited again, might have asked why that had happened in their case, but that would not necessarily impact on any other teacher in the school who was teaching a higher still course.From the viewpoint of most of our members, that issue was in a different category from the difficulty of transmission of data, which was rightly seen as a concern that was being raised by a number of individuals. When a trade union raises an issue with a body and is assured that the problem has been addressed and dealt with, it is difficult for the union to say, "You are totally wrong. It is necessary to go back and check data and figures."
You raised the issue in September and you raised it continuously until April, but did you accept the reassurances every time they were given?
One does not merely accept the reassurances—one assumes that people are doing their best to resolve an issue when they say that they are.
Our members' experience essentially backed up what we were hearing from the SQA, which was resolving the issue by asking our members to mark double the amount of papers. The members who got in touch with us were those who were displeased because they were not being asked to do that.
I want to move on.
I want to return to what has happened in the past 12 months. George MacBride said a few moments ago that the SQA examination system was so sophisticated that it required only a few things to go wrong for it to be in trouble. I am paraphrasing, but that is what was said. All three organisations' evidence identified the same problems: late vetting of papers during the year; difficulties with candidate entries in the autumn; difficulties with updating lists in January and February; inability to confirm entries; difficulties with unit results, which were asked for as many as four times without a response from the SQA; deadlines which took no account of holiday periods; a breakdown in communication between schools' and the SQA's computer systems; insufficient or wrong exam papers being provided; and all the difficulties with markers that we have discussed.That is a range of problems that were identified by individual teachers and schools. I presume that all the witnesses have experienced one or all of those problems. As individuals and as associations, do you think with hindsight that there were many signs of the impending crash—which we compared to the Titanic the other day—but that nobody joined up the signs sufficiently to realise what was happening in the SQA?
It is fair to observe that no one was there to put together the pieces of the jigsaw, but you must remember that we are teachers first and trade union associates second and that we do not have the time to get the bigger picture.
It was a genuine observation and I would like you to comment on it from your practical experience.
I am the SQA co-ordinator in my school and everything that Mike Russell enumerated is correct. It all goes back to last September and the failure of the SQA to capture properly the original data entries. The reason for that—whether it was a software problem or a management problem—did not concern me at the time, but all the subsequent problems came from it. It is a garbage-in-garbage-out situation. If one's initial information is not correct, one cannot judge properly what will happen further down the line.
Many of the examples that have been given are aspects of the same problem, which is not the setting of exams—we will leave that to one side because it is a different issue—but the input of data. As John Kelly said, we are not clear whether the problem was due to a failure of systems, of hardware or of management. However, many of the problems resulted from data not being input once they had left schools, data being input inaccurately and—most important—corrections not being made to incorrect data. That was extremely time-consuming for SQA co-ordinators such as John Kelly, but the knock-on effects—schools being given the wrong number of papers on the day of an exam, for example—could not have been reasonably predicted by people in schools.The problems with the recruitment of markers and the appropriate numbers of markers were, as Andrew Shanks said, a consequence of the data failure and could not be predicted in individual schools. I stick by what I said, which Mike Russell paraphrased; we have a sophisticated system, but when it began to break down and data entry began to break down, there was a series of major knock-on consequences.
In theory, one could put all the information together, decipher the pattern and produce a construct. In fairness, the unions operated together fairly closely on higher still, and we operated fairly closely with the SQA and other groups. No combination of any of those organisations was able to come up with such a construct. It would be difficult for that to happen, even if it was not for want of trying to find a construct. If, by a process of fusion of some sort, we had identified that we were on to something, we would have pursued it vigorously. There is no doubt about that. In theory, it would have been possible to produce the construct, but in practice—and with the best will in the world and the co-operation of colleagues in other unions and the SQA—that did not happen.
When I asked that question of head teachers and teachers in my area, I heard two interesting responses, which I will put to the witnesses. One response was that the SQA was in effect saying—albeit politely—that the problem was the schools' fault, because they did not know how the system worked and they did not use it properly. There was a tendency to blame the schools and teachers for weaknesses in the SQA system. I am not saying that it was done maliciously, but the SQA tended to think that it knew its systems so well that the problem could not possibly be with the SQA.The second interesting response was that it was felt that because the system had always worked, it would always work, no matter what difficulties were experienced. It was therefore considered impossible that a disaster such as that which took place would take place, because that was not what had happened before.There have been other explanations for the problems. One such explanation was that when people, especially in the Scottish Executive, learned of the situation, they were inclined to trust the SQA and not the teachers and the education profession. That has been a symptom for some time. Could you comment on those responses and suggest other reasons?
On the view that the SQA blamed teachers, head teachers or schools, although a few EIS members have said that such comments were made to members or to schools by representatives of the SQA, that is not our general perception. That attitude is not what we perceive from SQA management or SQA employees who were in communication with schools. Although there might be some evidence to support the view that Mike Russell expressed, there is little evidence to sustain the view that the SQA blamed schools and teachers for failures.The second view is one that carries considerable weight, because the Scottish exams systems operated efficiently and effectively for a considerable number of years—more than 100 years in the case of higher exams. People expected that efficiency, and it is possible that they were inclined to say, "Something has gone wrong in my school with input of data"—as the year went on, bigger and bigger things went wrong—"but those are blips that we hope will be adjusted." As we went through the session, it became clear that the blips were not going to be adjusted, but many people probably started out having faith in the system.Mike Russell's last point, that Scottish Executive senior civil servants, HMI, ministers and so on trusted the SQA more than they trusted teachers, is important. That view arises partly from the positive view that the SQA and its predecessor bodies had always worked effectively and that it was therefore reasonable to trust the SQA and have faith that it would continue to work effectively this year. However, as we have discussed, that led to the opinion that the views of teachers could be neglected and discounted as anecdotal—a word that has been used frequently—although such anecdotal evidence was sustained often by dozens, if not hundreds, of members. The fact that teachers' views were neglected is an important issue.
Like George MacBride, I have the greatest respect for colleagues at the SQA. They have been unfailingly helpful and courteous during the nine or 10 years that I have been doing this job. Even with the pressures that they were under—particularly from March to June when there were constant phone calls and faxes between us—they were tremendously supportive. We should put up our hands and say that there were only minor changes to the way that data were sent to SQA; the major difference was the volume of data. I pressed the same buttons, only a hell of a lot more often.As I have said, there seems to have been a failure to capture and implement data properly. The SEB and SCOTVEC always provided a fine service in the past and the problems at the SQA must be laid at the door of either the administration or the Executive, which lumbered the SQA with a dual system. When we were given a year's deferment for certain subjects, teachers' unions asked for the same deferment for all subjects, which would have allowed the awards processing system and some other processes to bed in properly. I do not know how much training SQA operatives had—that is a matter for the SQA—but it is little wonder that mistakes were made in a system in which a number 7 means a B pass for a Scottish Examination Board higher, a fail in a new higher and course completed in standard grade.For example, an operative in the morning might input data for a higher still subject where W means withdraw, and might in the afternoon input data for craft and design, where each youngster has had to compile a project in either wood or plastic. The "W" or "P" for wood and plastic that the school enters might be mistaken for "withdraw" and "pass". I know of a school where all the youngsters doing craft and design were reckoned to have failed because they had all worked in wood. Although that problem was very easily solved, it meant that the operative in the school had to go in to school during the summer holidays.
I have a final question for each of the union representatives. The word "reassurance" has been overused throughout the piece; indeed, Margaret Nicol used the word again today. Have people in Scottish education been inclined to be too easily reassured by the SQA and other bodies? Could the unions learn the lesson of being less easily reassured and asking harder questions?
That is an important lesson for us all to learn. However, those who run the system should also learn the important lesson that there must be considerably greater transparency, openness and opportunities for dialogue, which is the conclusion that we reach in our written evidence. We must investigate the culture of non-departmental public bodies and how they relate to the stakeholders with whom they work. Furthermore, we need to explore the culture in the Scottish Executive at Victoria Quay—not the Scottish ministers themselves—which has tended to be modelled on the Westminster culture and which decrees that "We know best because we are the senior people in the system."When people make statements about the system that are based on considerable evidence—as HMI does—that evidence must be up-front and subject to opportunities to question it. That would prevent over-easy reassurance or its reverse, which is a suspicion of every statement that is made by somebody in public life.
The lesson to learn is that there should be greater transparency.
Yes.
There has been much anecdotal evidence of concerns about marking. Now that appeals are under way, we are still receiving anecdotal evidence about dissatisfaction with the marking of appeals. Do your members have any experience or knowledge of whether appeals have worked well? Furthermore, do you support the idea that scripts from appeals—not all exams—should be returned to schools as a means of determining whether the initial marking of papers and appeals scripts, or the contributory factors of sitting higher still, have led to so many students' dissatisfaction with their marks?
Although I do not have a union role, I can say as a teacher that there is dissatisfaction with some of the results, and a sense that the results in English and communication, and other subjects this year are depressed—as are the pupils and teachers. We feel that the results do not represent the candidates' potential fairly. This year, most departments submitted far in excess of the usual number of appeals in the hope that, on re-examination of the papers, some justice would be done to candidates. Although we have not received the results of the bulk of the appeals, the emergency appeals that relate to university and college entrants have been returned, but to some disappointment. It remains to be seen whether the results of the bulk of appeals answer the prevalent pessimism.At the moment, teachers want scripts to be returned, because they are frustrated and angry and want some evidence to prove either that their expectations for candidates were not met in the exam, or that they were right. However, I can see the SQA's problem. If every teacher disputes every mark in every exam, we could end up with quite a debacle. Although returning scripts seems like a great idea, it might turn out to provide less clarity and transparency than we suppose. I would not choose that option.
Mr Eaglesham, in your submission, you say that the suggestion that papers be returned is unacceptable. Can you comment on Brian Monteith's point?
The return of papers poses technical and practical difficulties. When scripts are marked for the SQA, the principal assessor must be satisfied that they have been marked to the appropriate standard. That is an entirely different exercise from a teacher returning prelim scripts to their own class, when they need to explain to pupils where they have gone wrong and how to improve their performance. There is no point in sending back final examination scripts to candidates, saying, "If you do this or that next time, you will get a better mark."A requirement to return the scripts would also require a full explanation of why a particular mark was given to a particular section, to allow the pupil and teacher to understand the marking. That would increase considerably both the pay and the time that was required for marking, the former of which is scandalous. For example, I recall an occasion when I was marking in one room and the carpet fitter in the next room was being paid three times what I was being paid for the same period of time. The diet of marking would also need to be extended considerably to achieve that. In the majority of cases, it is probably not necessary for scripts to be returned and it would be unhelpful to prioritise a need to make marking of papers readable for candidates, parents, class teachers and lawyers. Some trust must be placed in that process.
You are free to give your opinion on the matter. However, my point is that there is a large degree of mistrust of the marking and the appeals process. However, it seems that in England, papers can be returned. Mr Kelly, given that the NASUWT crosses the border, do you have any anecdotal evidence on the experience of returning papers in England?
Although I have no observation to make on that matter—I have not had much to do with it—I would probably endorse David Eaglesham's views.We must consider the problem that returning the scripts to schools this year will create in relation to the time scale of the work that we are doing with the current crop of candidates. We must recognise that we are into the next generation of candidates and that the problem is historic.I do not think that it would be a good idea to return the scripts, for the reasons that David Eaglesham gave. We use youngsters' prelim scripts for formative purposes. As David said, we would say to them, "You have gone wrong on this aspect and you have not fully covered that one." It would serve no purpose to return the scripts to us.We must take the same approach that we have always taken: if we were dissatisfied with the results in for example, biology, we can ask for an examiner's report, for which a school would pay £100. That report gives us a full breakdown of where our candidates or our teaching went wrong—it gives us the source of the problem. That is the best way forward for our concerns. Because we have such concerns—especially in relation to this year's higher grade human biology—that is the approach that we will take.
I do not object to commenting on the situation south of the border, where scripts were returned in a planned way. The decision was taken to return exam scripts and the marking process took account of that decision.Although we have had a crisis, it is important to ensure that we do not arrive at knee-jerk solutions in response to it. The Scottish Executive is undertaking a full review of the first year of the implementation of higher still. That review was sought and granted through the trade unions and the national liaison committee and its impact will be increased by a much stronger review of internal assessment. There are major problems with internal assessment in particular, and with issues such as transparency and the return of candidates' papers.The reviews must address those issues—we expect them to be addressed properly. We also expect teachers and teachers' organisations to input into that work. We should seek major changes through the review process rather than through an immediate response to a crisis.
I understand the practical differences between prelim papers, exam papers and the English example of the planned introduction of returning candidates' scripts. You made those points well. I raised my point in order to establish what might be done to achieve credibility if exam scripts were to be returned. My point was to suggest that we limit the return of scripts to those candidates who are dissatisfied with their appeals. If that approach would not work, what more might be done—other than for every school to ask for examiners' reports—to try to re-establish the credibility of the SQA and the exam system?
Get it right next year.
That sounded like a good answer to me.
No—that was not a good answer.I do not necessarily agree with Brian Monteith's solutions. We should ask how we will restore credibility to the results of last year's cohort. That crucial question is part of the committee's remit.
That was a slightly different question.
Many of those students are still at school.
I stress Mr Monteith's point: many of those students are still at school and it is important that they realise that what will happen this year will be credible. To echo Margaret Nicol's point, we must get it right next year. That means that we must put in resources—not only money or more pay for markers, but resources of time, hardware and software at the SQA—to ensure that data processing and operational management systems work effectively this year. Young people and their parents must get that message, which must come from all levels of the Scottish education system.As far as the young people whose results were disappointing, their parents and their teachers are concerned, and regardless of whether they have left school, a number of processes are under way to try to ensure that a quality guarantee is built into their results. For example, the SQA is answering to a group of four members of the directorate who are seeking to pursue audit trails and to ensure that quality procedures are being used in the appeals process.The SQA has also committed itself to a detailed statistical examination of the appeals, subject by subject and geographical area by geographical area. That should help to identify any points of difficulty and restore credibility.
I am keen to move on. Are there any more questions?
I was not here when you discussed the line of questioning.
I would like to round up this part of the meeting.
People have looked for various reasons why it all happened. One argument that was put to us forcibly was that the difficulty lay in bringing together the SEB and SCOTVEC. There is a view that SCOTVEC's bad practice infected the new organisation. We were told that that view was commonly held in Scottish schools. Do you believe that view to be commonly held and do you give it any credence?
It would be fair to say that in schools, as opposed to colleges, the SEB was the major provider of the highers and standard grade systems. Among Scottish teachers, there is very strong loyalty to the SEB and a feeling of its quality, stretching back in time. The same feelings do not exist for SCOTVEC, because teachers had much more limited involvement in the more than 700 modules that SCOTVEC provided. There are some teachers who do not think as much of the quality of the SCOTVEC modules as they do of the new intermediate 1 and 2. However, to extrapolate that view, as Johann Lamont has done, is perhaps not true of the majority of Scottish schools.
I did not extrapolate. I presented a case and asked for your view.
In our evidence on the proposals to merge the SEB and SCOTVEC, we indicated that we hoped the prevailing culture of the new organisation would be that of the SEB, in which we had more confidence. The culture and mores of the SEB were much more conducive to teachers and schools. We would not be overtly critical of SCOTVEC or say that it was wrong, but the preferred culture was that of the SEB. There have been difficulties in bringing together those two cultures.
I understand that Johann Lamont was referring to someone else's view. However, whatever our views, we do not think that the use of emotive language is helpful to the debate.
A concern that has been around since the beginning of the higher still programme is that of parity of esteem. Many schools were not comfortable with the concept of internal assessment under the SCOTVEC model. Nationally, that was recognised from the beginning, which is why there is a hybrid system of internal unit assessment and the stamp of external examination. Like George MacBride, I would prefer not to use emotive language. The people at SCOTVEC were always fine and professional in their dealings with schools. We should not allow aspersions to be cast on them. However, schools have been more comfortable with the SEB tradition. Perhaps they will have to learn to change, too.
The verification procedures for SCOTVEC did not seem to be as rigorous as those for the SEB.
Please ask a question.
I was coming to that. There is the problem this year, and there is the review of higher still that Margaret Nicol mentioned. What can we do this year? Is there something that we should do in the short term and should we take a wee bit longer to do the things that Margaret Nicol talked about? I wonder whether we have time to make the changes that you are suggesting in the current year.
Although we believe that there is a need for radical review of and change in the assessment procedures—especially the internal assessment procedures—and for a drastic reduction in the amount of internal assessment, it would be totally inappropriate to start that on a major scale during a session that has already begun. At all levels of higher still, young people are now several weeks into their courses. Making radical changes this year would disadvantage them and cause confusion to them and their teachers. However, we must be ready to act quickly on the conclusions of inquiries such as this and of the Executive-sponsored review of the implementation of higher still, so that changes can be made not in the far distant future, but in the session after this one.
Would it be possible, even at this stage, to shift the exam timetable, or would that be a step too far?
Bringing the exam diet forward considerably would pose problems, because people are planning on the basis of 160 hours. John Kelly has already indicated the more flexible date—the date on which young people are told the results of exams.
When the examination diet is being composed, regard must be had for the pressure that young people are under at that point. That can be done without dealing with the problem of less teaching time.
Are you content with that, Mr Kelly?
Yes.
Good. I thank our witnesses for their attendance and for their answers to our questions. We will be taking evidence from other witnesses, and you are welcome to stay to hear that.
I thank members of the committee for their questions, which have been thought provoking.
There has been a delay, for which I apologise, in starting the next evidence-taking session. We are time constrained. I thank Pat Cairns and Alex Easton for their patience. We will proceed immediately with questions from members. I will stop taking questions at about a quarter past 12.
I intended to ask a general question, but I have decided to change tack.
Please make questions specific.
Section 2 of your written submission relates to concerns about marking; in it, you give a full explanation of one school's results and describe how they were"corrected to 13 band 1 passes and then further corrected".Presumably you are talking about the fact that results had been issued and people were phoning up or approaching the SQA for clarification because they were not happy. This was a period during which people were finding that marks were changing constantly, depending on whom they spoke to.
Those were marks that were sent to the school in response to inquiries. The school in question was one in which the marks would be expected to be high; it was in the independent sector and had a very good track record. When it received the results, it discovered that virtually the whole cohort had failed, when normally they would get A and B highers. That suggested that something was awry. The second return gave all the pupils a band 1 pass. Now the school has received final results, but it is still asking how certain it can be about those. I chose that school from a range of examples as one of two extreme cases, just to make my point.
In the last paragraph on page 1 of your submission, you say that:"The SQA's decision to allow centres to amend estimates at the time of appeals is to be regretted."Could you expand on why you think that?Your submission also says that:"‘Pushy, articulate parents' who have demanded that an appeal be submitted, even when their son/daughter performed in line with the estimate, have advantaged their children".Why do you think that?
That quotation was taken from SQA documentation that was sent out. I passionately believe in equity, and that has been one of the great things about the Scottish system. To put it bluntly, opportunistic parents have latched on to that. Parents' appeals were accompanied by a lawyer's letter giving a lawyer's interpretation of the statement and telling the school that, in an appeal, the school would have no choice.There were examples of youngsters who were predicted to fail by a narrow margin and who did fail by such a margin. However, in cases where there could be no appeal, parents wrote demanding a re-mark and enclosing a lawyer's letter. Phone calls to Dalkeith said, "Put them in through the normal appeals system." We are concerned about that, because a single parent on income support with four youngsters has not been able to use the system to the same extent as some of those other parents.
Your submission also mentions with regret"the decision to allow centres to amend estimates at the time of appeals".How much of a problem do you think that was?
Once the possibility that estimates could be amended got into the public domain, pressure was put on schools to do that. That was not helpful.
I do not quite understand that point. Were those instructions issued to centres by the SQA in August?
We were asked to resubmit estimates. On a few occasions, we were told that estimates could be revisited, as some estimates had been lost somewhere. That was interpreted by some parents as—
Was it done in response to some parents? Were parents pushing for that? Do you think that the instruction came from the SQA in response to parents pushing for it?
The SQA would have to speak for itself about that, but I suspect that the answer is almost certainly yes.
The paragraph of your submission that deals with the effects on pupils and parents says that one of the difficulties for pupils was that they were unable to seek reassurance from their teachers. That takes us back to the failure to provide schools with copies of the results. What was the effect of that and what action did you take in response to it, knowing that you would not be in a position to reassure pupils?
We all felt completely foolish, because there had been no communication to schools that we would not receive that information. I am sure that every school in the land had its board of management in place, together with secretarial support, to deal with the usual rush of phone calls. Also, because we were all interested in knowing what the results were, we were all there. People come back from holiday specifically for the exam results date. We were there, but there were no results. We were informed only on that day, when we started telephoning, that the results would not be made available to us.
When you were told that the results would not be made available, were you also told that, at the end of that week, the pupils would not get their results either?
On the day that the results were due to come out, we were told that neither we nor the students would get the information.
In the past, did you get the information prior to the day when the students got their results?
No. The students and the schools usually got the results on the same day. Occasionally, the schools got them the day before, but usually we got them for sure in the first post on the same day.
You would be in a good position to tell us what is happening now and what your fears are for the future. I have evidence from two places in Scotland that some students are still waiting for written confirmation of results today—10 higher biology students in Greenock and five history and geography students in Newton Stewart. Do you have evidence of students who are still waiting for final confirmation of their results?
Yes. I think that that is still the case. There has been talk of suggested action. The suggested short-term solution is to get the data-handling and administrative procedures really tight. I think that that can be done. The other suggested solution concerns quality assurance management. I am sure that that will be done. Because of the good reputation of the Scottish Examination Board, perhaps too much faith was put in the system.This year's youngsters are 25 per cent of the way into their courses, but already there is slippage in the procedures for the registration of candidates and notification of entries for SQA certification. I hope that what has been suggested will happen, but I would not want to bet my pension on it, when there is already slippage in the established time scales for the 2001 diet of exams.
You have gone a step ahead of me. I will stop you on that first point. You are telling us that you believe, from your experience or from what your members have told you, that some pupils are still waiting for final confirmation of results.
The final appeals are being mopped up.
Do you mean appeals or results?
I am talking about appeal results.
I asked about results. Is anybody awaiting final confirmation of results?
There are still people who have not received their standard grade results. I do not know of anybody who has not received their higher results, but there are still youngsters who have not received confirmation of their standard grade results.
Are you aware that that is contrary to the expectation that the minister expressed in his statement of 6 September?
Yes.
Let us move on to the second point, on the future. Your submission proposes several actions, including a review of the SQA administration, the removal of stress points by the higher still liaison unit, and the expansion of the role and responsibility of principal assessors. The suggested actions on which I will focus are, first, the proposals to second some school SQA co-ordinators to assist with data management and, secondly, to"have earlier exam diet, to finish by 1 June."
School SQA co-ordinators and, in particular, principal assessors know that side of the system inside out. It is important to involve such knowledgeable people in the review to ensure that the systems are in place.I think that the idea of having an earlier exam diet, which would finish by 1 May, is a non-starter for this session for the reasons that were given earlier. However, it may be worth considering in the longer-term review. The problems were made worse by the very short time that was available between the exam diet and the end of June and the start of the holidays.
It should be remembered that any reduction in the teaching year would have an impact on the course content, and would therefore require a review of courses. The time lines will have been set for this year, as youngsters have already embarked on courses.
But you are in favour of that proposal for future years.
It is worthy of consideration within a review of other matters, such as course content.
You say, rightly, in your paper that pupils have been the hardest hit and that many are disillusioned. Given that we know that some of our pupils have not yet received their standard grade results, how do we encourage young folks and give them confidence in the examination process? A senior teacher at a school in my area told me that some kids whom he was assessing for standard grade maths asked him whether they would receive marks for the assessments.
Probably the people whom pupils and parents trust most are—God bless them—head teachers. Parents know us, have worked with us and will accept reassurance from us that they would not accept from other people. They know that they can knock on our door and meet us in person.Our only strategy has been to tell pupils to assume the most optimistic standard grade result in choosing their courses. It is now too late to change to a higher if they discover that they have done better than they thought they had. It will be December before standard grade appeal data are available.
Will that strategy help to give pupils confidence? There is still a feeling among pupils that, if teachers are unhappy with the situation, pupils cannot be happy with it either.
I think that teachers are confident in their estimation of pupils' abilities and that pupils trust those estimations. We can give pupils only our view of how they are performing, and the reasons for that view, and tell them to carry on with what they are doing. It is a matter of trust.
In a previous answer, you expressed concern about slippage in processing pupils for entrance to the 2001 diet. Did such slippage occur for the 2000 diet? Are you saying that we are already repeating some of the mistakes that were made last year?
As far as I can judge, a different mistake is being made. Schools submitted data timeously last year, but the data were lost and not processed.
For entering?
For entering. We did not get the feedback that we had called for that the data had been entered.
Are you concerned that something similar is happening?
I hope that everything will be made as bomb-proof as it can be in the short term. That is what is being done at Dalkeith and through our internal quality assurance. There may well be sound reasons in an action plan—which I hope exists—to explain that it helps to delay the request for the initial data. The fact is that that request is later than we had anticipated.
It is important to note that a key change was made to the approach to the delivery of higher still. We had started on the premise that children would begin a unit, be assessed, and move on to the next unit. It emerged that too many youngsters were failing, so the critical change to testing pupils when they were ready was made. Of course, that led to a huge slippage in data input and pushed the bulk of data input to the end of the year.Earlier in the year, at meetings with parents, we had told them that they would receive three reports, term by term. It is critical that we ensure that information is entered earlier. We are concerned that, if it is not, information will again become backed up.
Obviously, if a strategic decision is taken to carry out all the unit assessments at the end, an enormous amount of data will arrive late in the year.
Is there any theory as to why so many pupils failed assessments after completing units?
You will appreciate that the difference between standard grade and higher is huge. In the traditional higher, youngsters had the opportunity to do assessments, be regraded and benefit from that, but this time youngsters had to pass the assessments. Youngsters are unable to produce a top-quality essay first time around, three or four weeks into their fifth year. The steps that were taken were sensible, but they meant that we were not operating under the original premise.
In paragraph 7 of your submission, on suggested action, you say:"We support the recent suggestions by Scottish Parent Teacher Council in particular in relation to the status of unit assessments."Could you explain exactly what that means?
Yes. The simplistic view had been that there was a mixture of internal and external assessment and that, therefore, the external assessment was shortened. As has been said, there is a lot of pain in administering the internal assessment. Assessments have served the purpose of motivating youngsters and teachers, but it has transpired that internal assessments have operated a bit like a class ticket. The mark that a pupil received was decided entirely by the final exam. As long as a pupil had completed the unit assessments, their final mark seems to have been determined entirely by whether they had passed or failed the exam. The question is raised of the balance between internal and external assessments.Questions are also raised about validity and reliability. It is obvious that, if the exams are shortened, one rogue question can have a bigger pendulum effect—a good or bad question that is worth 10 marks out of 100 will have a greater effect than one that is worth 10 out of 200. Although it is too late for this year, that needs careful consideration.
Can you clarify that the mark for a pupil who does three unit assessments in a term does not count toward the final exam?
The exam is a separate entity, but one must pass all the internal assessments as well as the external exam.
A pupil must pass the internal assessments, but the mark that they get for them does not count toward the final grade.
The final grade is decided entirely by the external exam. In some cases, there are practical components that count towards the final grade.
The final exam was the be-all and end-all. It had not been understood that the final grade was not really a balance of internal and external assessments.
So the system is not continuous assessment. The internal assessments are just a hurdle to clear.I am still unclear about the Scottish Parent Teacher Council suggestion about unit assessments, with which you agree.
I think that the SPTC is suggesting that internal assessments should not be mandatory this year. They may serve another purpose, which is to motivate and check, but we should step back from them this year.
Your submission makes no clear suggestion about the SQA. What are your thoughts about it? The EIS and other bodies from which we have heard have made suggestions, such as that the SQA should have closer links with the Executive.
I covered that in points a) and b) of paragraph 7 of my submission. The SQA was starting up a new system of administering exams with 100 years of excellent experience behind it. To start from scratch would be crazy. People made the mistake of thinking that there was a new computerised system. There was not—the SQA had faxes and bits of paper flying in from all over Scotland. As soon as possible we want to move to an e-mail system. I signed the papers for the current census in my school and took accountable responsibility for that. I am comfortable with that, as it is what I am paid for. Eventually we should reach a situation in which schools input data and they are e-mailed to the SQA. That would avoid the danger of wrong buttons being pressed, as has happened at the SQA. Not all schools are on e-mail yet, so what I am suggesting may be a year or two down the line. In the short term, there should be rigorous scrutiny of what the SQA is doing. I imagine that the Scottish Executive will adopt a much more inspectorial role regarding the quality assurance procedures.
Do you see no case for reconsidering how the board operates with the executive of SQA, for example?
To dismantle or to change radically the SQA at this time would not be helpful. That is a personal opinion.
In my view, a failure to manage and a failure to manage data were at the root of this problem. Apart from the schools, everybody failed to understand the huge complexity of the task and how much information was going to descend at the very last minute. The problem was flagged up regularly, but schools were not listened to as they should have been. Whether the SQA was able to implement in one year all that was being asked of it is a question that I could not possibly answer. We cannot allow these failures to happen again.
Do you think that the SQA was unable to deliver all that was being asked of it?
Probably. There were three changes. Two boards had been amalgamated, there was a totally different structure of course delivery and there were new information technology systems, which in the first instance rarely function as they should. The combination of those three things made it very difficult to have success first time round.
Does blame lie with the SQA's political masters?
There was pressure on schools to comply, which they did. The only success story is that schools delivered for young people. I am very happy about that.
There was an overambitious—I use that word rather than reckless—push on schools from the higher still development unit and HMI. We were perceived as conservative if we tried to suggest that we move forward at a reasonable pace, as happened with standard grade. If at public meetings we said that we ought to ca cannie and think things through, the response was often very sarcastic.
Three members have indicated that they wish to speak and there are only two minutes for questions, so I ask members to keep their questions brief.
I am interested in the impact of the way in which the SQA responded to the crisis. You say in your submission:"Several press statements were misleading."Was it obvious to you at the time that they were misleading, or do you think that people believed genuinely that what they were reporting was the case, although it turned out to be wrong? Either way, it is quite damaging.
We are referring to statements that were made very late on, when it was like Saigon in the last days of the Vietnam war. People were under pressure and close to panic, and it would be inappropriate to criticise them. Nevertheless, some of the statements that appeared in the press late on suggesting that schools had been contacted were misleading. These were mistakes made by individuals. I am not pillorying anybody, as people were under tremendous pressure. There was an element of panic at times.
I would like to put the same question in a different way. Would it be possible for you to comment on particular statements when submitting fresh written evidence to the committee? Could you identify statements that you thought were misleading and tell us why, so that we could consider that?
In this meeting
Everything on 4 Oct 2000 →- Scottish Parliament Education, Culture and Sport Committee Wednesday 4 October 2000 (Morning) Committee business · 2
- School Exams Committee business · 422
The type of each item (debate, statement, questions, members’ business and so on) and the times shown are worked out by this site from headings, the chair’s opening words and the Official Report’s running timestamps. They are not the Parliament’s own classification. Text: the Scottish Parliament Official Report.