Operating Procedures

The following pages define operating procedures for computer-assisted assessments. They may change over the next few years in the light of experience and we would value comment from any staff and students involved in computer-assisted assessments. Computer-assisted assessment in this context refers to the delivery of assessment on a computer screen, student entry of answers at a workstation, and marking of the answers by computer software. At present, the University supports Question Mark Perception as a vehicle for computer-assisted assessment across the University

The operating procedures are in five parts:

1. General principles
2. Operational considerations
3. Guidelines for staff
4. Guidelines for students
5. QMP Invigilator announcements Guidelines

Download the series of documents

General principles

1. Computer-assisted assessment is subject to the same policy and standards as all other assessments within the University.
2. Computer-assisted assessments are prepared and moderated to the standards and time-scales agreed by the host Department, including moderation by the relevant External Examiner where necessary.
3. Students submitting answers to a computer-assisted assessment are confirming that they are submitting their own work in accordance with the University’s regulations and policy on cheating and plagiarism.
4. Marking student answers using software complies with the University’s policy on anonymous marking of examination scripts.
5. All marks given to students by the software are subject to moderation at the relevant Subject Assessment Panel.

 

---

Operational considerations

1. Computer-assisted assessment involves the interaction of several parts of the University:
   • Technology & Information Services provide the networking and workstations;
   • Teaching & Enhanced Learning (TEL) the CAA software;
   • Academic staff provide the assessment tasks and marking schemes;

   In addition, for formal examinations,

   • Examinations Office provides invigilators and schedules the examinations.

   It is essential that good lines of communication are kept active for formal examinations, although the academic alone can often operate coursework and in-class tests.

2. For formal examinations, students should use a University-controlled account with appropriate security and in a lock down mode to reduce its functionality if necessary (for further information contact your local Learning Technologist).
3. For coursework and in-class tests, the level of security should be determined by the academic as appropriate to the assessment task, which may require students to use other software or access resources across the Internet.
4. For formal examinations, the module leader must be present and have access to Perception Enterprise Manager to solve problems. The support team should also include a member of Technology & Information Services and a member of the Learning Technology team: both should have experience of computer assisted assessment. The module leader and senior invigilator need to liaise closely. The assessment is ultimately the responsibility of the module leader

---

Guidelines for staff

These guidelines represent current good practice for staff designing and running formal computer assisted examinations using Question Mark Perception.

Two important things to remember when invigilating QMP assessments/examinations. Only use Internet Explorer for timed assessments and only start 10 students  at any one time. Once the first 10 have started, start the next 10 and so on.

Designing examinations

1. Question blocks should normally be used. This involves dividing the total number of questions into blocks which are delivered sequentially to students. In general each block should be designed to take about 15 minutes to complete. The advantages of this are:
   a) At least some answers are recorded by the server early in the exam rather than all answers being recorded at the end. This provides an early check that the system is working and means that should there be problems later during the exam, enough marks may be available to make a judgment.
   b) In conjunction with the use of topics, it provides a convenient mechanism for achieving a balance of topics in the questions extracted from a large database.
2. For examinations where students cannot all be accommodated simultaneously, the number of questions in the database from which exam questions are selected should be at least double the number of questions delivered in the exam. Compulsory questions can be used but the students should not be able to distinguish these from other randomly selected questions. In effect, this (combined with randomisation, see 4.) means that each student sits a different examination. Thus there is no advantage to students sitting the second of two exam sessions in a large class, even if they have spoken with students who sat the examination earlier in the day.
3. Where formal examinations cannot be accommodated in one room, the assessment should be password protected using a QMP Monitor obtainable from the University QMP administrator(s).
4. Randomisation of questions within blocks and of answers within questions should be used unless there is a good reason not to do this. This reduces problems of cheating during examinations.
5. The time scheduled for the computer-assisted examination will be at least 30 minutes longer than the time allowed for the examination. This will allow students to start when they are ready.
6. The timer within Perception Session Manager should be used to set the time for the exam so that students who take longer to log on to the server are not penalised.
7. Questions should be checked for accuracy by at least one other person with expertise in the subject being examined.
8. It is expected that approximately 10% of the number of questions in the exam will be added to the question bank each year.
9. Staff are encouraged to obtain assistance in typing and entering questions from Departmental or Faculty secretarial staff if possible.

Before the examination
All students should have experienced Perception and a test based on the same material used for the exam before sitting a computer-assisted exam. This may have been part of the formative assessment, coursework or may be available as an open access ‘taster’ before the examination.

1. Students should be informed of the examination assessment name by the Invigilator prior to the exam. Other assessments may be available to the student at the time of the exam, there should not be any confusion as to which exam the student should be taking.
2. Paper copies of a version of the examination should be available in case of problems with individual students’ during the examination. There should be copies available for 5% of the total group, with a minimum total number of 10 copies. These should be on coloured paper and be returned to the invigilators at the end of the session.

During the examination

1. The module leader must be present during the exam and have access to Perception Enterprise Manager to solve problems. The support team should also include a member of Technology & Information Services and a member of the Learning Technology team: both should have experience of computer assisted assessment.
2. If something goes wrong with an individual student:
   a) Check to see if any of the student’s answers have been logged on the server
   b) Change the number of attempts to 1 in Enterprise Manager and restart the student at the appropriate block (see 3)
3. If it is too late for the student to try the test again (i.e. more than half an hour has elapsed), then the student should do a paper copy of the exam and be allowed an appropriate amount of extra time at the end to complete this.
4. If something goes wrong with an individual student or with a group of students, as long as at least half the examination marks have been recorded, these will be used to form a judgement on student performance.
5. If there is a major problem (such as a power cut) affecting many students and none of the above actions are appropriate, the examination will be deferred until the alternative date on the examination timetable.

Students with extra time requirements

1. Students who require extra time should be scheduled in the last session, so that over-run does not affect a following exam.
2. If a student requiring extra time turns up at the wrong time, they should either be asked to come to the later session, or if they want to stay, not be given any extra time.

Drafted May 2000 by Dr Dan Charman
Amended May 2000 by Dr Chris Ricketts
Amended July 2002 by Dr Chris Ricketts
Amended Dec 2013 by Sid White (Learning Technologist)

---

Guidelines for Students

During an Invigilated Computer Assisted Assessment.

LOGON to the Student Portal.
DO NOT start your assessment until told to do so.
The test may not be available until the exact start time given on the examination
time-table, so logging in to the Perception Server too soon will not give you access to the test.

1. When you attend a computer-assisted examination you will be logging on to your own network account. When told to log on go to the Student Portal (http://student.plymouth.ac.uk/) enter your University Username and Password, go to MyEdesk > Computer aided assessments and stop there until told to proceed. In some cases the assessment will be password protected.
2. When you attend a computer-assisted examination your exam assessment may be Password protected. If this is the case an Invigilator will give you the Username and Password which you will need to begin your exam assessment.IMPORTANT
   Click once on the button to start your assessment.If you double-click the assessment, it may take you out of the assessment and prevent you from starting.The time available will be displayed on the screen, and you will be presented with either the whole assessment or the first block of questions as appropriate.
3. If the assessment is divided into blocks, you must complete each block of questions and click on the ‘Submit’ button before proceeding to the next block. You cannot go back to a block once you have submitted your answers to that block. You can go back to any question within a block.
4. At the end of the assessment, make sure you click on the ‘Submit’ button to get your mark. You may want to record your mark.
5. If you have not been given a mark, do not close down Perception or Internet Explorer, but contact one of the invigilators.
6. Once you have your mark, click on the ‘Home’ button to end your session.
7. If you have been using your own network account, don’t forget to close Internet Explorer and Exit from the network.

If you have problems.

If you have problems with the computer system, you should attract the invigilator’s attention. Depending upon the time left in the assessment, one of the following actions will be taken.

1. The session will be restarted manually at the point at which the problem occurred.
2. The session will be rescheduled to start at the beginning of the block.
3. A paper version of the test will be provided for completion.
4. No action will be taken and your performance will be assessed on the basis of the questions you have completed so far.

If there is a major problem (such as a power cut) affecting many students and none of the above actions are appropriate, the assessment will be deferred to an alternative date.

---

Guidelines for invigilators

These guidelines represent current good practice for staff invigilating formal computer assisted examinations using Question Mark Perception.
Most of the procedures that apply to paper-based examinations also apply to computer-assisted examinations. The main differences are:

• Because of limited availability of computers, many computer-assisted exams will divide the student group between two or more successive sessions;
• To ensure that loading on the network and assessment server do not cause problems, students will not all start at the same time, but will be staggered in groups of about 10 at a time. Once the first group of students have logged on, it will be clear to logon the next group, and so on;
• Some examiners may choose to have students entering and leaving on a rolling basis, rather than all starting at the same time;
• The time allocated for the examination will be determined by the computer, not the invigilator;
• The member of academic staff who set the examination will be present, and will usually be familiar with computer-assisted assessment.

You should be familiar with the following operational considerations

1. Computer-assisted examinations involve the interaction of several parts of the University:- Technology & Information Services provide the networking and workstations;
   – Technology Enhanced Learning support the CAA software;
   – Academic staff provide the assessment tasks and schedule the assessment on the CAA system;
   -Examinations Office provide invigilators and schedule the examination rooms.
   It is essential that good lines of communication are kept active for formal examinations.
2. For formal examinations, students should use a University-controlled Network account with appropriate security. If available, Internet Explorer in ‘kiosk’ mode should be used to reduce its functionality.
3. For formal examinations, the module leader must be present and have access to Perception Enterprise Manager to solve problems. The support team should also include a member of Technology & Information Services and a member of the Learning Technology team: both should have experience of computer assisted assessment. The module leader and senior invigilator need to liaise closely. The assessment is ultimately the responsibility of the module leader.
4. You should be familiar with the Guidelines for students. A copy of the ‘Guidelines for students’ should be provided at each workstation
5. The briefing of the students should include the following:
   -mobile phones to be switched off
   – do not start the exam assessment until told to do so
   – fill out attendance slip, which will be collected by an invigilator
   – ensure mark received before leaving
   – Log off the computer after you have finished
6. The University undertakes regular evaluations of computer-assisted examinations. Students may be asked to complete an evaluation form before leaving the exam.

QMP examinations – QMP Invigilator Announcements Guidelines

---

---

Version 1
July 2002
Dr Chris Ricketts

Version 2
May 2011
Sid White (Learning Technologist)

Version 3
Dec 2012
Sid White (Learning Technologist)

Version 4
Dec 2013
Sid White (Learning Technologist)

Version 5
May 2014
Sid White (Learning Technologist)