A Statistics Problem: How to use pre-board scores for Class 10 ICSE Papers if written examinations are cancelled

posted Jun 9, 2020, 2:11 AM by Prashant Bhattacharji   [ updated Jun 9, 2020, 2:12 AM ]

Let's say, there were 10 scheduled papers. 3 or 4 are left. How do you follow a fair process to award marks on the basis of the internal exams assessed by schools?
For class 10 (which is not an end point) it is very simple to use internal scores from schools and to adjust them. 

However, there are many students who might want to take the examinations, and many cities where there's no serious CoronaVirus spread right now. 
So the right thing to do, will be to give the students a choice. They may either take the written examination conducted by the board - or accept an adjusted score
on the basis of internal examinations. 

This is similar to a process followed in Australia as well and it is a statistically sound method.

The idea is 
- the overall nature of the scores should be in-line with the kind of scores students get in a board exam BUT 
- the ranking should be in line with what the school awarded for that particular subject, to a student. So let's say the preboard math topper scored 95, next one scored 94, next one scored 92. Their marks will be normalized but ultimately in the final board exam result; the student who scored 95 will get the highest, next highest will go to the one who scored 94, next highest will go to the one who scored 92 in the preboard.
- so the ordering of the performance of the students in the board-awarded mark for a subject needs to be the same as where the student ranked in class during preboards; but it doesn't matter that the school's preboard marking or paper
was strict or lenient 

One (WRONG) way which some people have suggested 
- look at the overall improvement or change between preboard to board, across different subjects, and then add that much of a "jump" percentage to the papers which weren't held. 
- this one is wrong because it's possible that in the pre-board, one paper (say, English) was marked very liberally relative to the board; whereas another paper (say: Bio) was marked very strictly compared to board standards. 

So, don't use marks of papers where the board exam has been conducted though for safety, it is ok to collect that data from schools. 

The concrete steps are: 

1. Ask for the preboard score for a specific subject: 


     A - 80
     B - 85
     C - 82 
     D - 79
     E - 92 

     Ignore the scores but convert them to ranks:

     A - 4
     B - 2
     C - 3
     D - 5
     E - 1

2. Now you need to assign marks to these students, based on these ranks, while factoring in the school's performance in 
    exams which have already been conducted. 

    This is where it starts to become a bit tricky. 

    The overall assumption is, that if a school's history marks have an average of 70; then the geography scores should be more or less the same.
    It is perfectly possible that an individual student who is the class topper in Geography,  ranks only 10 (out of 45) in history; but the overall distribution
    scores for the school should look the same. 
    One way 
    - completely ignore the project marks for now, just focus on the exam papers out of 80 
    - In the Australian Moderation process   just use External = History (already conducted) and Internal = Geography (not been conducted) and then upgrade the "Internal" marks. 
    - Similarly for science use External = (Physics + Chemistry)/2  and Internal = Biology (not been conducted) and just upgrade the internal (pre-board) bio marks using this algorithm   
    - schools which have performed much better than others in board exams already held, the upgrade will naturally ensure the trend is consistent in the papers where marks are awarded on the basis of pre-board
   - For papers where no-subpart has been held (eg: Computers) this will require some thought. One option might be to compute "external" as the average of all scores in compulsory papers which have been held.

The last part (deciding what to use as the correct external data to properly normalize scores from pre-board) - this will require some thought,
maybe pull in some experts from the Indian Statistical Institute: However, the overall way to normalize pre-board scores meaningfully, is this Australian Moderation Process. 
it is nothing too complicated, very much do-able -- maybe not a good idea for class 12 (elective papers have just a few students sitting in the exam hall) 

Maybe just give students an option. Those who want to take the exam in July can do it, else they get the upgraded pre-board marks. 
We need to be careful - this is a process which may easily go wrong like this and this. The process has to be approved and ratified by qualified statisticians or professionals
in the assessments space, otherwise top schools maybe unfairly penalized for strict marking in their preboard examinations.