Programme

The Conference

The Hisar conference will follow a similar structure as previous AEA-Europe conferences, with keynote presentations, parallel discussion group sessions, parallel paper presentations and a poster presentation session. 

Click on this link to view the preliminary programme for 9th annual AEA-Europe conference

Pre-conference workshops

Pre-conference workshops, organised by the Professional Development committee, will be held the day before the conference, November 5, at the same venue as the main conference.

1. "Assessment for Learning" led by Gordon Stobart
2. "Developing Questions and Mark Schemes to ensure construct validity", led by Alastair Pollitt and Ayesha Ahmed
3. "Assessment for Policymakers" led by Steven Bakker and Andrew Watts
4. "Item Response Theory and Predictive Systems" led by Frans Kleintjes and Eduardo Cascallar

Click on this link to download the abstracts for the pre-conference workshops

Pre-conference workshop abstracts

1: Assessment for Learning

This workshop will be led by Gordon Stobart, who is a member of the Assessment Reform Group, a group which has helped to shape formative assessment practices in the UK

This session will look at current international understandings of formative assessment and how it is being used in the classroom. We will consider how practices may vary across different cultural contexts, particularly in relation to summative assessment.

At a practical level we will look at classroom developments in making clear what is being learned, of finding out where learners are in their learning, and in giving effective feedback. We will also consider ways in which formative assessment can best be presented to classroom teachers.

2: Developing Questions and Mark Schemes to ensure construct validity

This workshop will be led by Alastair Pollitt and Ayesha Ahmed from Cambridge Exam Research

Evidence lies at the heart of educational assessment: the job of an examination is to obtain and evaluate valid evidence of students’ achievement. Within this, the job of the question is to elicit suitable evidence of the trait we are assessing; while the job of the mark scheme is to evaluate it appropriately. Of course, the evidence comes from the students’ minds.

What is happening in students’ minds when they are being assessed?

What do we mean by evidence?

How do we decide the best way to evaluate the evidence? 

Our definition of validity is that a question can only contribute to valid assessment if the students’ minds are doing the things we want them to show us they can do, and if we give them credit for, and only for, evidence that they can do it. We will present our system for developing questions and mark schemes, which focuses on obtaining and evaluating evidence to discriminate between those at different levels on the trait. The system includes four essential elements:

• Expressing the idea and the intended demands of a task
• Describing the evidence we would like to elicit from students’ minds
• Deciding on a marking strategy  to evaluate that evidence
• Writing a question to elicit the appropriate evidence

The workshop will start with a presentation of these ideas, using examples from many exam contexts. In the second part of the workshop participants will be invited to develop assessment materials for their own contexts, working through the four elements systematically. This workshop is suitable for anyone with a general knowledge of assessment principles who wishes to develop their skills and understanding in the area of test and examination construction. It would be especially useful for anyone involved in the training of question writers.

3: Assessment for Policymakers

This workshop will be led by Steven Bakker and Andrew Watts. The workshop will be divided into two halves.

During the morning Steven will present "National Assessment Debates". Policy makers need to decide what kind of assessment efforts they would like to see implemented to support their policy and decision making. Two presentations, one on admissions testing and one on international surveys will fuel discussions on these issues. Participants will be encouraged to bring perspectives from their home countries.

In the afternoon Andrew will present "Maintaining trust in a national assessment system". This will begin with a brief review of the context and purpose of assessments. The workshop will then move on to the key topic of validity. The currently favoured ‘unitary view’ of validity will be described. We will discuss the implications of this and the place of validation activity in an assessment system.

Within the discussion on validity the workshop will also focus on the issue of reliability. We will reflect on what makes for reliable assessments, both in terms of examination and test development, as well as of marking and grading. Finally, we will consider more broadly the levels of public trust in national assessment systems and the issues faced by assessors in explaining their work publicly.

4: Item Response Theory and Predictive Systems (with applications in creating and evaluating standards and applications of predictive systems)

This workshop will be led by Frans Kleintjes (Cito, Arnhem, The ) and Eduardo Cascallar (Assessment Group International, Belgium)

Item Response Theory (IRT) is used to analyse test data on item level, IRT provides a statistical process for estimating characteristics of items and examinees and defining how these characteristics interact in describing item and test performance. When used properly IRT can increase the efficiency of the testing process, enhance the information provided by that process and make detailed predictions about unobserved testing situations.

IRT is used for many measurement applications including item banking, test construction, adaptive test administration, scaling, linking and equating, standard setting, test scoring and score reporting. In the workshop, the emphasis will be on using IRT in examinations.
The workshop starts with some theory on IRT, including a comparison with classical test theory. Major properties of IRT will be highlighted using illustrative examples. IRT output will be explained, discussed and interpreted based on materials that will be provided. Several concepts used in IRT will be explained using examples from the test construction experience of the presenters and if available from participants. 

Part of the workshop will focus on the use of predictive systems in predicting educational outcomes. After a short introduction in predictive systems, the relationship with IRT and its use in evaluating educational outcomes will be presented. Emphasis will be on methods of pruning input data to maximize the accuracy and effectiveness of automated intelligent systems in educational applications which involve classification tasks. We intend to deal with application of IRT in:

• Content related interpretation of the ability scale.
• Standard-setting and maintaining standards in examinations.
• Predictive systems and the prediction of educational outcomes.