Newsletter – Spring 2015

President’s Report

Guri Nortvedt

This is my first newsletter update as President of the Association following my inauguration at the general assembly in Tallinn. I am very thankful to all of you for your trust in me and for this opportunity to follow in the footsteps of previous presidents of AEA-Europe. Our association is thriving and growing and it has been very fulfilling for me to contribute to this activity and to give my time alongside everybody else who has volunteered their time to a committee or to the council.

Since Tallinn, we have welcomed Alex Scharaschkin (AQA, UK) onto the Council as Executive Secretary. Alex joined the Council in January, but in accordance with our constitution, he will be formally inaugurated at the general assembly in Glasgow this November. The Council would like to thank the Nominations Committee (Christina Wikstrom, Frans Kleintjes and Gordon Stobart) for providing such a great candidate. Alex will be a great asset to us.

The two committees contribute significantly to the work of our association. They are mainly operated by members, offering their time for the good of all. I am delighted to welcome one new member to each of our two committees: Bas Hemker (Cito, the Netherlands) to the Professional Development Committee and Daniel Xerri (University of Malta) to the Publishing Committee.

This summer we will prepare for two new elections to take place in early autumn. Our Treasurer, Henk Moelands (Cito, the Netherlands), will retire from his position after having served on the Council for nine years. We will need a new, dedicated Treasurer to follow in Henk’s footsteps. We will also need a new and just as dedicated Council member to replace Sandra Johnson (Assessment Europe, France). Sandra has served four years on the Council and was a key player in the organisation of our very successful Paris conference. Our Association owes a great deal to Henk and Sandra, for all their hard work and efforts over many years.

I think it is safe to say that our annual conference is considered to be the main activity of our organisation. This conference is where we all get to see each other and to listen, present, discuss, network and plan future cooperation, laugh, learn and share. We had a successful conference in Tallinn before Christmas much because we had such a dedicated local host in Foundation Innove, and I believe that all of you who heard the presentation of the upcoming Glasgow conference recognised another dedicated host who will welcome us this November. Hosting a conference is a good opportunity to give back and to get involved in the life of AEA-Europe. Last autumn we sent out a call to all of you, the member, asking for proposals and bids to host future conferences. I am very happy to tell you that the Council is currently reviewing the offers we got.

We are all looking forward to the Glasgow conference. Our local hosts, the Glasgow Organising Committee and the Glasgow Scientific Programme Committee, chaired by Louise Hayward, Jo-Anne Baird and Sarah Maughan, are all working hard to prepare this year’s conference. We are very grateful to all involved who are very willingly giving their time to ensure we all have a good experience and a fruitful conference. As I write this, the submission process is still ongoing. We are convinced that this year’s theme of “Assessment and Social Justice” will attract just as many submissions as past years, and that you are all busy planning your proposals.

As you can see from the conference webpage, I can confidently promise you that this year’s conference will have an excellent set of keynote presentations. Speaking of keynote presentations, I would like to mention that there is a way to get involved in the Glasgow programme that I encourage you all to think about. Each year the winner of the Kathleen Tattersall New Researcher Award gives a keynote speech as part of their prize. If you know someone early in their ‘assessment career’ who contributes significantly to the field of assessment, please tell them about the award and encourage them to apply. If you yourself contribute to this field, you should also consider applying to be this year’s award winner. Please visit our webpage to read the eligibility criteria. Do not wait too long: the submission deadline is rapidly approaching.

Introducing the Glasgow conference

AEA-Europe holds its 16th annual conference in Glasgow, Scotland at the Radisson Blu Hotel from November 5th to the 7th November 2015.  The local host is the School of Education, University of Glasgow.

The theme of the conference is “Assessment and Social Justice”. One of the major challenges facing governments and education systems across Europe is how to ensure equitable opportunities in education for every citizen. Assessment can contribute to social injustice, even if it is not of itself its primary cause. All three of the main purposes of assessment have an impact on social justice: assessment for formative purposes, for summative purposes (including certification) and for purposes of accountability. The conference will explore each of these purposes and how they impact on social justice.

Keynote speakers are Louise Hayward from University of Glasgow, Florian Waldow from Humbolt University and Bruno Zumbo from University of British Colombia.  More information about the keynote speakers and the programme and theme in general can be found on the AEA Europe conference pages.

Glasgow, Scotland’s largest city, is an exciting destination, combining the history, energy and sophistication of a great international city with easy access to some of Scotland’s most spectacular scenery. Glasgow was named in 2014 as one of the world’s top ten must see cities by Rough Guide and Fodors. From Glasgow you can easily explore Scotland. Just beyond the city of Glasgow lies some of Scotland’s most beautiful scenery. Ancient castles, atmospheric distilleries, tranquil lochs, outstanding golf courses and miles of unspoilt coastline are all just a short journey from the city centre.

We are expecting this conference to build on the ever-increasing success of the Association and look forward to seeing you in Glasgow in November.

Sarah Maughan and Louise Hayward

Co-Chairs of the Scientific Programme Committee.

Work in progress

Creating European Standards for Open Education and Open Learning Resources (EU-StORe project*)

The context

Educational concepts experts have traditionally been responsible for designing curricula, courses and learning materials. These days, with unstructured and widely varying content creation, we have to create standards and guidelines to ensure quality in general and also to foster quality to fight social disadvantage, which can arise because of wrong, unstructured or misleading information and knowledge transfer. In particular, disadvantaged and marginalized groups in European society can suffer from misleading information because they face many challenges.

Open education and open learning resources (OERs), the majority of which include assessment, are actually among the main educational topics in the EU. Given that most OERs are thought to be used in learner-machine interaction environments rather than within human- tutoring systems, assessment represents in itself the main, if not the exclusive, form of feedback to learners to help orient their future learning. Establishing quality and possible standards, not only for content presentation and delivery, but also for tests and other forms of assessment offered in the OERs represents a crucial step for leveraging best practices in open education.

One goal is that, as in a giant supermarket, online and open learning resources should be available all over Europe at no cost to learners. Although making education available to a broad European group of interested learners, teachers, and trainers is an opportunity as well as a challenge, there is currently no mechanism available for quality assuring courses, learning resources, and open measures.

What is happening

The project “EU-StOre” – European Standards for Open Education and Open Learning Resources – is an international consortium situated within the ERASMUS+ Project, with a focus on Vocational Education and Training (VET). The duration of the project is 24 months. The seven partners within the project consortium are drawn from six different countries – Germany, Great Britain, Ireland, Italy, Romania and Malta – and include (a) four universities, (b) an IT enterprise for the technical design, (c) an enterprise in the field of educational design, adult and further education, and (d) an official institution for rural development and education.

The main aim of the project EU-StORe is to conceptualise and implement a European inventory of open learning resources, to analyse open learning scenarios and open learning resources, and to create shared common European standards and guidelines for open learning, including assessment methods and procedures.

Why inform members

AEA-Europe wishes to promote projects that link researchers in different countries; this project is an example of such collaboration, linking universities in Germany, Italy, Romania and Malta and promoting cooperation with companies in the field of education in Great Britain and Ireland.

Finally, the idea of setting standards for the development of Open Educational resources, with a bottom-up approach – i.e. collecting and cataloguing available OERs – has more specific repercussions on the possibility to assess quality in e-learning for self-regulated, life-long learners.

Further information

Prof. Carmen Duse

carmen.duse@ulbsibiu.ro

Prof. Marc Beutner

marc.beutner@uni-paderborn.de

Prof. Gabriella Agrusti

g.agrusti@lumsa.it

For additional information, see also the EU-StORe Website:

http://eustore.eduproject.eu/


Announcement from the International Association for the Evaluation of Education Achievement (IEA Secretariat)

Preparing for Life in a Digital Age: The IEA International Computer and Information Literacy Study International Report

Summary

Ability to use information and communication technologies (ICT) is an imperative for effective participation in today’s digital age. Schools worldwide are responding to the need to provide young people with that ability. But how effective are they? The IEA International Computer and Information Literacy Study (ICILS) responds to this question by studying the extent to which young people have developed computer and information literacy (CIL), which is defined as the ability to use computers to investigate, create, and communicate with others at home, at school, in the workplace, and in society.

The context

ICILS 2013—the first in international research to investigate students’ acquisition of CIL—was conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA). It builds on a series of earlier IEA studies focusing on ICT in education.

In 2013, researchers assessed almost 60,000 eighth grade students in more than 3,300 schools from 21 education systems. Data were also collected from almost 35,000 teachers in those schools and from school ICT-coordinators, principals, and national research centres.

The ICILS 2013 international report, Preparing for Life in a Digital Age, presents the major findings from this study. It provides an overarching comparative perspective on student achievement in CIL, the contexts for CIL education, and how achievement relates to student characteristics and school contexts.

In general, the study findings challenge the notion of young people as “digital natives” with a self-developed capacity to use digital technology. The large variations in CIL proficiency within and across the ICILS countries suggest it is naive to expect young people to develop CIL in the absence of coherent learning programmes. Findings also indicate that system- and school-level planning needs to focus on increasing teacher expertise in using ICT for pedagogical purposes if such programmes are to have the desired effect.

What is happening?

The ICILS 2013 international report is freely available online at: http://link.springer.com/book/10.1007%2F978-3-319-14222-7.

What is new?

The ICILS 2013 international database will be made available for download in March 2015 from the IEA Study Data Repository: http://rms.iea-dpc.org/. There are a number of resources for researchers interested in working with the datasets from ICILS or other IEA assessments—to learn more, visit: http://www.iea.nl/data.html.

Why inform members

A second cycle of ICILS is planned to be carried out in 2018. Development work on the framework and instruments will commence in early 2015. For more information, see: http://www.iea.nl/fileadmin/user_upload/Studies/ICILS_2018/IEA_ICILS_2018_Leaflet.pdf.


Doctoral Work in Progress

Strategic entry to GCSE mathematics exams – challenges for maintaining standards


Rachel Taylor

Supervisors – Professor Steve Strand and Professor Jo-Anne Baird

Context

Within the English education system, public examinations are expected to be taken at specific points in a student’s education, with one of the main qualifications being the GCSE taken at age 16. In recent years, however, there has been a rise in the number of students sitting GCSE assessments early and on multiple occasions, leading to concerns that this behaviour is not in the best interests of students (e.g. ACME, 2011; DfE, 2011). Rather, these practices are considered to be driven by the pressure on schools to achieve results for accountability purposes: in particular, the pressure for students to achieve a grade C, an important threshold in performance measures. These concerns led to a policy change in September 2013 when it was announced that, thereafter, only a student’s first entry attempt would count towards school performance indicators (DfE, 2013), prompting a significant decrease in early and multiple entry the following summer (Ofqual, 2014).

There is little published evidence documenting the rise and fall of early and multiple entry, despite there being a range of likely consequences for schools, students, awarding bodies, and the wider assessment system. For students, early and multiple entry could impact on attainment and future progression, whilst for awarding bodies, shifts in entries may have implications for the statistical guidance used to set and maintain standards. As such, this thesis aims to add to this literature by exploring the trends, drivers, and potential consequences, of this behaviour. The research focuses on GCSE Mathematics, since this subject has experienced considerable flux in entries over recent years and is a key subject for performance measures.

Research methodology

This research adopts a mixed-methods approach and contains three main phases; a quantitative analysis of national trends in early entry between 2007 and 2011; a qualitative study exploring the drivers and potential consequences of this behaviour; and a quantitative study considering the impact of early and multiple entry on national outcomes.

Phase 1 charted the rise in early entry between 2007 and 2011, particularly in GCSE English and Mathematics, and identified how early entry was increasingly being used for lower ability students. Furthermore, early entry was particularly prevalent in lower performing schools, suggesting that this strategy might not be in the best interests of students.

Following this, fourteen semi-structured interviews were conducted with GCSE Mathematics teachers in schools in England, to explore the drivers behind these trends. The main findings described how the pressure of performance measures had been driving behaviour in schools, prompting the use of early and multiple entry to maximise results. This led to a number of teaching behaviours that were not necessarily in the interests of students and their learning; for example, teaching to the test, a focus on borderline students, and continuous cycles of revision and examination entry.

The final phase is currently underway and will explore the extent to which early and multiple entry might impact on national outcomes, using data from the National Pupil Database.

Why is this research of interest?

During the course of my doctorate, there have been several policy changes that have impact upon schools’ behaviour and the focus of my research, something that is likely to continue. Nonetheless, the behaviours described here, early and multiple entry, can be considered as examples of strategic behaviour adopted by schools to maximise results when faced with accountability pressures. Thus, they provide evidence of the way in which accountability can influence behaviour within schools that is relevant beyond the current context. Strategic entry behaviours are likely to have a range of consequences for students, schools and the wider assessment system. Consequently, generating evidence surrounding such behaviour is considered important to aid ‘policy memory’ (Higham & Yeomans, 2007) and inform debate during future periods of reform.

References

Advisory Committee on Mathematics Education (2011) Early and Multiple entry to GCSE Mathematics. Retrieved from http://www.acme-uk.org/news/news-items-repository/2011/5/position-paper-on-early-and-mutiple-entry-to-gcse-mathematics

Department for Education (2011) Early entry to GCSE examinations. London, DfE.

Department for Education (2013) Changes to early entry at GCSE. Retrieved from https://www.gov.uk/government/news/changes-to-early-entry-at-gcse

Ofqual (2014) Summer GCSE and IGCSE entries for England: Provisional Figures April 2014. Coventry, Ofqual.

Higham, J., & Yeomans, D. (2007) Policy Memory and Policy Amnesia in 14-19 Education: learning from the past?, in D. Raffe & K. Spours (Eds) Policy Making and Policy Learning in 14-19 Education. London: Institute of Education, 33-60.

External Conferences

ECER 2015
European Educational Research Association

2015 European Conference on Educational Research
Budapest, 8-11 September
(Emerging Researchers’ Conference 7-8 September)

Network 9

Assessment, Evaluation, Testing and Measurement

Network 9 offers a full programme of symposia and individual presentations throughout the main conference, covering a range of topics of interest to professionals working in education in general and in the fields of assessment and evaluation in particular. For further information on current activities and past Network 9 conference programmes see http://www.eera-ecer.de/networks/network9/”

Early bird registration is available until May 31st.
http://www.eera-ecer.de/ecer-2015-budapest/



Association pour le développement des méthodologies d’évaluation en éducation

(The Association for the Development of Educational Evaluation Methodologies)

28th Annual Conference
Lisbon, 13-15 January, 2016
Theme: Assessment and learning

ADMEE is a French language international association with national sections in Belgium, France, Lebanon, Luxemburg, Morocco, Portugal and Switzerland. It brings together researchers, teachers, trainers, educational managers and others interested in issues concerning assessment and evaluation in education and training.

http://www.admee-europe.org


The 2nd Asia-Pacific Educational Assessment Conference (APEAC) is organised by the Academy of Principals (Singapore) from 10 to 11 September 2015 for school leaders, teachers, academics and researchers from all levels in Singapore and the region. The aim of APEAC is to raise the assessment competencies in our educators and school leaders so that quality teaching and learning can be achieved at every level. Educational assessment topics have been specially chosen to broaden one’s view of education assessment as well as to explore current trends and global best practices in education assessment. Whether one is new to educational assessment or a seasoned expert, the participant will find ample opportunities to gather new knowledge and insights at APEAC.

Participants will also discover how renowned institutions and educators have implemented their own successful programmes and find out how to implement them at one’s institution. International speakers who are authorities in educational assessment have been invited to share their knowledge through a series of keynote and spotlight sessions. As an event both diverse and informative, APEAC will serve as an opportunity for educators to network with peers, connect with experienced practitioners and exchange ideas with professionals from around the world.

Theme
Assessment with Purpose

 Objectives
· Sharing updates on latest and upcoming trends in assessment development
· Learning more about a learner-centred and balanced assessment
· Providing an opportunity to advance communication, collaboration and exchange among academics, researchers and practitioners.

Who should attend?
· School Leaders
· Teachers
· Academics
· Researchers
· Education Policy Makers
· Ministry of Education Officers

www.aps.sg



6th IEA International Research Conference (IRC)

International Association for Evaluation of the Educational Achievement (IEA) and the University of Pretoria in South Africa invite you to participate in the 6th IEA International Research Conference (IEA IRC-2015). The conference will be held on 24–26 June 2015 in Cape Town, South Africa, and will be preceded by training workshops on secondary data analysis on 22–23 June 2015. For details and registration, please visit: http://www.iea.nl/irc-2015.html. We look forward to seeing you in Cape Town!

June 22 – 26, 2015

Cape Town, South Africa

www.iea.nl

My visit to Cito

As the sponsor of the award for best poster presentation at the AEA-Europe annual conference, Cito invites the winner for a visit to their offices in the Netherlands. And as the winner of the poster award in Tallinn, 2014, I had the privilege of attending a custom-designed day at their institute. On March 12th, an eager PhD student, with an equally eager supervisor, appeared at the new Cito building in Arnhem.

We were greeted by Diederik Schonau and Henk Moelands, who talked about the role of Cito in Dutch education, and looked after us throughout the visit. Three presentations from Cito experts were arranged during the day, all related to the topic of my PhD project. First, Erik Roelofs presented his work on the development of cognitively based diagnostic assessments. We also discussed the potential and challenges of diagnostic assessments regarding the reporting and interpretation of information, and item development. In a second presentation Angela Verschoor and Frans Kleintjes described the Cito Student Monitoring System, and we discussed issues and topics related to my project (e.g. validity issues). For the third presentation, Daniel van der Palm introduced his work with Latent Class Models as a density estimation tool for handling missing data and for estimating test-score reliability.

In addition, I was given the opportunity to present my project to Cito staff. Many people within psychometrics and mathematics assessment showed up, and several constructive questions and comments were offered. During the visit, we were also given a tour of the impressive building and enjoyed a nice lunch in the cafeteria. The day ended with a fresh and tasty dinner in a nice and relaxing restaurant in company with Henk Moelands and Cor Sluijter.

I am very grateful to everyone involved in my visit to Cito and who gave me the rare opportunity for having a day at the Cito institute, especially arranged for me and my PhD project.

Best regards,

Andreas Petterson

Department for Teacher Education and School Research, University of Oslo

Professional Development Committee: Update

The PDC in its current form comprises Antonella Poce  (University Roma 3) – a Council member and Committee Chair; Yasmine El Masri (researcher at the Oxford University Centre for Educational Assessment); Ingrid Radtke (Norwegian Centre for ICT in Education); Bas Hemker (Cito, Dutch Institute for Educational Measurement); and Stuart Shaw (Cambridge International Examinations, Cambridge Assessment).

The PDC meeting at Heathrow Airport, January 2014.The PDC thanks Frans Kleintjes who stood down from the committee at the general assembly in Tallinn.

The PDC is organized around a number of strands of activity:

  1. AEA-Europe Accreditation Scheme 
  2. New Researcher Award combined with the Doctoral Network 
  3. Poster Award 
  4. Webinars and Seminars. 

Here we briefly summarise these activities and provide an accompanying explanation of how we intend to take each one forward.

1.       AEA-Europe Accreditation Scheme 

AEA-Europe’s professional accreditation scheme was first launched in 2007. The scheme provides members of the Association with the opportunity to have their assessment knowledge, skills and expertise recognised through the award of the status of Practitioner and Fellow of the Association. The PDC is actively encouraging Association members who have not yet applied for accreditation as Fellow or Practitioner to do so.  Association members who are not yet ready to apply for accreditation as a Practitioner can nevertheless declare an intention to do so at some future point and be awarded status of ‘Associate’.

The issue of Accreditation represents one of the most important remits of the Committee. The PDC has been working hard in order to increase participation in the accreditation procedures and in addition to contributing to the revision process (which is now administered by the Committee), the PDC has decided to carry out a series of actions to support the Accreditation Scheme.  

First, the PDC has agreed to administer a questionnaire to Fellows and Practitioners in order to elicit feedback on how to raise awareness within the educational assessment community of the value and currency of accreditation. The outcomes of the questionnaire will be communicated via the Newsletter and a poster to be presented at the next annual conference in Glasgow (2015).

The PDC is also looking for Fellows and Practitioners who would be willing to describe the benefits and entitlements their accreditation status affords. For example, Fellowship status allows opportunities to review conference proposals; participate in the election of other Fellows; and create openings for engagement with the New Researcher Award.  If you are interested in finding out more about the accreditation process, please contact the Secretariat of the AEA-Europe Association (admin@aea-europe.net) or any of the PDC membership

2. New Researcher Award combined with the Doctoral Network

AEA-Europe is keen to provide support for new researchers and doctoral students who have an interest in assessment. The Association has built a network of doctoral students from various European academic institutions. The Doctoral Network has created a LinkedIn page to provide a platform where doctoral students can share their experiences and struggles. It has also organized several successful events over the past years including pre-conference workshops. The PDC supported the Meet and Greet event, which was very successful in Tallinn. A similar event will be held in Glasgow this year. The social event will be prefaced by a more content-oriented meeting, where PhD candidates will be able to exchange views and opinions about their current research.  

AEA-Europe recognizes new talent in research by granting the Kathleen Tattersall New Researcher Award every year to the best new researcher applicant. 

In September 2014, the Council reviewed the procedure for selecting the New Assessment Researcher Award winner. Many of the amendments were based on suggestions made by the PDC. The amendments included:

  • Changing the time criterion of eligibility to the first three years after being awarded the doctoral degree or within the first seven years of being involved in assessment research,
  • Only one of the referees now needs to be an AEA-Europe member
  • The reviewing panel should be announced in the conference programme

The deadline for potential candidates for the New Assessment Researcher Award for 2015  is May 15th. The website provides further information about the process.

3.       Poster Award

AEA-Europe encourages its membership to submit poster proposals for its Annual Conference each year. The PDC has made a number of changes to the procedure for determining the poster award winner. The new procedure consists of a straightforward numerical count of votes cast. In case of a tie it has been agreed that the PDC (excluding members already participating in the poster competition) should decide on the eventual winner. The winner will be announced by the President of the Association (Guri Nortvedt).

4.       Webinars and Seminars

In the past, the PDC has been responsible for organizing webinars for AEA-Europe members. In view of offering a more efficient service to the membership, the PDC sent out a questionnaire to all members of the Association in November 2014 to determine the most popular and useful topics as potential foci for future presentations. A significant majority of respondents (72 %) agreed that AEA-Europe should organise webinars in 2015 with around half suggesting that they should be organised around selected Tallinn conference presentations.  Overall, 68 % of the respondents would consider participating in webinars.

Suggestions for future topics focused on topics presented in the keynote sessions or specific pre-conference workshop presentations Some respondents suggested specific topics from the paper sessions and other topics not covered during the conference (21st Century Learning; Assessment across Cultures, Assessment of Literacy Across Cultures/Migrant Students ; Technology Enhanced Assessment – TEA; Rethinking Assessment for a Digital Age Problem Solving Abilities, Scientific Reasoning).

The preferred time of day for presentations was early afternoon, late afternoon or early evening.  Respondents suggested that presentations should occur either every three months or twice a year.

The PDC is committed to making AEA-Europe the foremost association for all assessment professionals throughout Europe, particularly, it is hoped, with the active involvement of its academic, professional and vocational members.

Accreditations, honours and awards

At the 2014 annual conference in Tallinn the following individuals were accredited by the Association for their contributions to educational assessment:

Fellow
Gordon Stobart (Institute of Education, University of London) receiving his certificate from Jo-Anne Baird, AEA-Europe President.

Fellow
Cor Sluijter (Cito) receiving his certificate from Jo-Anne Baird, AEA-Europe President.

Sarah Hughes (Cambridge International Examinations) and Nico Dieteren (Cito) were awarded Practitioner status, while Gareth Hopkins (City and Guilds), Zak Horrocks (AlphaPlus Consultancy) and Hayo (Hendrikus) Vink (Kenniscentrum Beroepsonderwijs Bedrijfsleven Curaçao) received Associate certificates.

Yasmine El Masri (University of Oxford) received  the 2014 Kathleen Tattersall New Assessment Researcher Award, and Andreas Pettersen [University of Oslo] received the award for the best poster presentation at the Tallinn conference.

Jo-Anne Baird became an honorary lifelong member of the Association when she handed over the Presidency to Guri Nortvedt.