HOME PAGE | COMMUNICATION | LINKS | DISCLAIMER | NEWS AND EVENTS

 
        Greek
ENL   RCeL
 
 

Research and publications

 
 
  Research |

Publications

|

RCeL editions

|

Presentations

 
 
 

ORAL EXAMINERS

Oral examiners’ attitudes towards the exam and specific activities are investigated through the use of questionnaires distributed to examiners in all exam centers on the day of the oral exams. These feedback forms include questions about the oral test procedure, the examiner conduct, the format and the content of the oral test papers and communication strategies.

 

SCRIPT RATERS

Script raters’ views about the rating procedure, the script rating training, the script rater tools and their marking experience are also explored through the use of questionnaires completed at the end of the script rating procedure. In order to assess the quality of script evaluation, the ways in which script raters use evaluation criteria are systematically observed, inter-relater reliability is conducted and the characteristics of the scripts which cause serious problems in inter-relater reliability are thoroughly examined.  

 

CANDIDATES

Through specially designed questionnaires distributed in each examination period, KPG candidates’ views and attitudes towards test papers are also being explored with a view to discovering what types of tasks pose difficulties to candidates, what factors they think affect their performance, what strategies they use in order to perform well or in what themes they are particularly interested. Statistical analysis is then conducted (through SPSS or other tools), the results of which are exploited for the development of reports referring to the main research results.

 

THE ITEM ANALYSIS PROJECT

This project aims at determining the degree of item and task difficulty of the reading and listening comprehension test components (Modules 1 and 3). Using examination data the Ministry of Education provides us with after each exam administration, we calculate through specialized software the following statistical measures: a) internal consistency or reliability of the exam, b) index of difficulty (i.e., a value showing the proportion of candidates answering an item correctly), c) discrimination indices (i.e., values showing how well an item succeeds in distinguishing highly competent from less competent candidates) and d) distractor analysis (i.e., the frequency with which each option of a particular test question is chosen). This information combined with more sophisticated statistical measures enable test developers and item writers to evaluate item and task performance ensuring overall test validity and reliability. See also: ELT News KPG corner

 

THE INTER-RATER RELIABILITY PROJECT

The on-going study on inter-rater reliability (i.e., agreement between raters on scores assigned to papers) in the KPG script rating process is carried out as a means of investigating the effectiveness of the instruments subservient to the process employed by the KPG test developers. These instruments are: a) the rating grid together with the assessment criteria, b) the script raters training material and training seminars and c) the on-the-spot consultancy to the raters by KPG experts and test developers. The aim of the particular project is to look into the factors affecting the rating process in the specific situation and examine the extent to which these factors contribute to higher inter-rater reliability. The data consists of the marks given by different raters to A1+A2, B1, B2 and C1 level scripts of candidates who have taken the KPG exam in English and is being obtained at the Athens Script Rating Centre.

 

THE TEST-TAKING STRATEGIES PROJECT

The KPG test-taking strategies project aims at investigating the test-taking strategies, i.e. the techniques or ‘tricks’ consciously used by KPG test takers so that they can respond to KPG test requirements and whether these can be successfully taught. Questionnaires are the basic tools designed especially for this project, and their development was a complicated procedure involving different groups of research participants at various stages of the research process, including the research subjects. Research results have already been published and presented in numerous conferences.

See also: ELT News KPG corner

 
 
 
 

The KPG multilingual examination suite

 

Test papers and evaluator training

 

Research and publications

 

Funded KPG Projects

 
 

Partnership with the Aristotle University of Thessaloniki

 
 

 

Web Developer: A. Sarafantoni / Web Designer Ch. Frantzeskaki