Newsletter – Summer 2018
Sunny greetings to you all! Yet another academic year is coming to an end but before you leave for your holidays, let me share some AEA-Europe news in the Summer 2018 edition of our newsletter.
In this issue, our AEA-Europe President, Dr. Thierry Rocher reviews the progress achieved by the association over the last two years thanks to the tremendous effort invested by its active members and he shares with us the planning of future conferences.
By now, you have all marked your diary for our annual conference in Arnhem-Nijmegen in November 2018. Our hosts this year, the CITO team, are especially excited to include this event as part of Cito’s 50th anniversary celebrations and have prepared a packed academic and the social program to mark the occasion. For those of you whose submitted proposals have been accepted, you would have received confirmation by now. Don’t forget that as a presenter, you need to register and pay by Friday 31st August 2018, failing which your paper will be removed from the conference programme. Check out the conference page to discover the agenda and our keynote speakers. You will note that this year, the AEA-Europe SIG for e-assessment will be holding a pre-conference workshop on “Innovative on-screen assessment”. Catch up on more of the SIG’s e-assessment news in their article and fill out their survey to guide the SIG on where you feel the group should place its focus and activities. As this topic continues to top the charts of educational assessment strategies, policy-makers have started real conversations on the challenges and solutions to implement these e-assessments in their own digital context. Read more to find out about the FLIP initiative where France, Italy, Luxembourg and Portugal are taking the lead to set up an international collaborative partnership to share very large-scale e-assessment experiences, IT costs and digital content in an open-source environment. Discover MathemaTIC, another national example of an ongoing international collaborative partnership with the aim of building a digital language-free solution to learning mathematics in Luxembourg schools. Three years after its modest start, MathemaTIC has already won an international e-assessment award. As this wind of e-assessment blows across Europe and overseas, more needs to be documented and shared. To this effect, the European Schoolnet has made available, a series of articles on assessment in school education on its School Education Gateway platform. In the world of higher education, “The Learning Portfolios in Higher Education” report provides readers with a review of how far learning portfolios actually impact learning outcomes. Policy-makers are craving to measure 21st century competences and skills of learners, which is why researchers flock to the field of educational measurement. In this context, check out the PhD and postdoc vacancies advertised by the University of Oslo. As you know, the AEA-Europe also values the work of researchers in this field. This year, the Kathleen Tattersall New Assessment Researcher Award 2018 has been won by Dr. Maria Bolsinova, research scientist at ACTNext, the Netherlands. Stick around for her keynote presentation at the conference in Arnhem-Nijmegen in November.
Finally, thank you to our contributing authors for reaching out and keeping us posted with the latest news on educational assessment. To you readers, please make this your space and share your work related to research, educational assessment and policy programs. Whether you are piloting or implementing assessment initiatives or you have discovered education technology tools that add a new dimension to student assessment, or you have set up collaborative initiatives between researchers and practitioners – write to us and let the networking begin!
So should you wish to share your story in the next issue of our newsletter in October, please feel free to reach out to me at firstname.lastname@example.org.
AEA-Europe Newsletter Editor
A word from the President
Almost two years after taking up my role as the president of the association, I look back and realize the significant progress that AEA-Europe has made thanks to the engagement and commitment of its Council and committee members. When I presided my first general assembly meeting in 2016, I recall the suggestions from members, to use various tools such as a new website and social media to extend the outreach of AEA-Europe. Today we have a brand new website, over 260 followers on our Facebook page, connections on Twitter and LinkedIn, three newsletters per year and a mobile App for our conference. Our first SIG on e-assessment was only launched during the conference last year but it already has its own Facebook page with over 80 members and it will be holding a pre-conference workshop in November in Arnhem-Nijmegen. Our internal organisation is running very smoothly (regular council meetings, active committees, reflection on membership strategies, running the elections, planning of future conferences). Simply impressive output that our active members achieve on top of their already hectic, full-time work and family schedules! Thank you everyone!
Following the call for submissions to our upcoming 19th annual conference in Arnhem-Nijmegen, a great number of proposals was received, again confirming renewed interest in AEA-Europe. Registration for the conference is open since July and I strongly advise all the presenters to register as early as possible to facilitate the organisation of the conference programme. “Building bridges to future educational assessment” is a theme that challenges us all in our work, so we count on your participation to make the conference yet another success. Besides, the AEA-Europe council already had the opportunity to visit the conference venue during its second meeting in May this year in Arnhem-Nijmegen. I can only confirm the CITO team’s enthusiasm that all is set for a great conference.
Looking ahead to future AEA-Europe conferences, our Vice-President, Jannette Elwood and I, visited the future hosts from IAVE Portugal. A preliminary agreement was signed to formalise IAVE’s intention to host the conference in Lisbon in November 2019. We also visited the Trinity College and Educational Research Centre in Dublin to meet the organisers of the 2020 annual conference. We have no doubt in the capacity of the hosts to successfully organise our future conferences. Thinking over a longer term, a mail was also sent out to invite potential candidates interested in hosting the 2021 and 2022 AEA-Europe annual conferences, to submit your expression of interest us by 30th September 2018. The objective is to meet them in November in Arnhem-Nijmegen.
Before ending, I wish to reiterate my gratitude to each and every one of you, who give your time and expertise to support the work of AEA-Europe. It is a real personal commitment to be actively involved in the association, one that sometimes comes at the expense of spending time with the family. Yet, it is because of your willingness to volunteer that AEA-Europe can pursue its mission over the years. Thank you very much and I look forward to your continued support. May we continue working and networking in the same team spirit while continuing to add knowledge to the field of educational assessment.
AEA-Europe 2018 conference in Arnhem-Nijmegen
The second half of the year is fast approaching and this means ‘Conference time’ for all experts and professionals that are involved in educational assessment. Our team at Cito is proud to be the organizers of the 19th annual AEA-Europe conference in Arnhem-Nijmegen, the Netherlands on 7-10th November 2018. Registrations may be done online through Easy Conferences by clicking on http://easyconferences.eu/aea2018/.
The theme of the conference will be “Building bridges to future educational assessment”, a theme which is both a showcase of the landmarks of the Arnhem-Nijmegen region, and a very exciting hot topic for our assessment community. The submission period for abstracts and proposals is now closed and a high number of submissions have been accepted. The conference venue is the Van der Valk Hotel Nijmegen-Lent where a rich and diverse academic program awaits the participants. This includes parallel sessions, with open papers, discussion groups, symposia, poster presentations and ignite sessions (new this year!). We are also thrilled to confirm the participation of amazing keynote speakers – click here for details. The pre-conference workshops on Wednesday 7th November are shaping up to be an amazing start to the conference.
In addition, Cito Institute for Educational Measurement is celebrating its 50th anniversary this year. We have therefore taken this opportunity to proudly host and sponsor the welcome reception on Wednesday evening in the CITO Office Building close to Arnhem Central Station.
The local committee has put together an attractive social program that will have its highlight on Friday evening, when we all will embark on the River Dream boat for a very special ‘floating gala dinner’. We are really looking forward to welcoming you all in the Netherlands in November!
AEA-Europe SIG e-assessment news
The SIG is delighted to announce that during the AEA-Europe 2018 conference in Arnhem, we will be holding a full day pre-conference workshop on “Innovative on-screen assessment – confronting barriers, sharing ideas and moving forward”. In this workshop, presenters from CITO, Durham University and the IB will share implementation experiences and participants will be offered the opportunity to discuss and contribute to a proposed framework. This will guide development and future implementation of more technologically advanced on-screen assessment items leveraging the digital opportunities more effectively.
The second e-assessment SIG webinar took place on 25th April 2018 with Gareth Hegarty, Senior Manager, eAssessment Transition at International Baccalaureate Organization. In 2016, the International Baccalaureate (IB) introduced non-compulsory eAssessment in its Middle Years Programme (MYP, examination at age 16) which recently won the International e-Assessment Association award for Best Use of Summative Assessment. The webinar focused on the MYP on-screen examinations using multimedia and interactive tools to provide an immersive assessment experience, challenging its students to demonstrate knowledge and conceptual understanding as well as enquiry and communication skills. Gareth presented a fascinating discussion and demonstration of the MYP to an online ‘live’ audience of 42 international participants. In the Q&A session, participants raised a range of issues including validity of items, school resourcing and user competency. We ran out of time to answer all of them, so these will be discussed in posts on the SIG Facebook group soon. If you were one of the 93 people who signed up for this webinar, you will have received the link to watch a recording. More information on the MYP eAssessment can be found on the IB website here. If you’d like to join the eAssessment SIG, please join the SIG AEA Europe E-Assessment Facebook group or email LWye@aqa.org.uk.
Currently the response to the SIG survey stands at 32, of which many are AEA members and 18 SIG members. Respondents are requested to select up to five themes that most interest them. Following up on the initial survey results, the SIG pre-conference workshop addresses an issue high on this list of interests: computer-based assessment delivery models, data and operational implementation issues. We would like to call for more of our 80+ members and other AEA members to share their views which will inform the SIG’s focus and activities. Our five-question survey will remain live and can be found here. More and hopefully updated results will be shared at the AEA 2018 conference.
The FLIP initiative: an international collaborative partnership for e-assessment in education
FLIP is the acronym for the four initial countries (France-Luxembourg-Italy-Portugal) that set up a joint initiative in 2017 to share knowledge and experiences, IT development costs and digital content within the context of e-assessment. The idea emerged from the growing number of countries facing similar interests to develop technology-enhanced items which offer better ways to assess traditional competences, to address 21st century skills and to link assessment feedback closer to learning. Moreover, countries share the common need to resort to a robust test delivery platform which supports “very-large-scale assessments”.
The underlying objectives of FLIP are three-fold. Countries or institutions joining FLIP would firstly share knowledge and experiences in e-assessment in order to move towards robust solutions and to avoid technical setbacks. Secondly, countries would share IT development costs by mutualizing resources to fund solutions that could resolve common needs. Thirdly, countries would share technology-enhanced item content according to specific copyright, sharing and confidentiality regulations. In addition, two important principles (open-source and interoperability) underlie the FLIP initiative. The open-source solution is favored (open source rather than proprietary code) and is driven by the spirit of collaboration in order to constantly improve the digital tools as a result of the activities of the community participants. Interoperability implies that the digital content should not depend on any specific technology that is not exportable to different platforms. This would hence enable countries to develop content items that may be delivered and used on different platforms without the need to depend on a particular provider. In this way, the content itself will remain the property of the countries that created them, who may still decide to change the underlying platform if they wish, without additional costs.
Some digital tools have also been identified and planned for development in the short and medium term, according to the priorities of different countries. Brazil has since then also joined FLIP and there has been growing interest from institutions in many other countries.
By popular demand, the FLIP initiative was also presented in the framework of AEA-Europe’s first SIG e-assessment webinar in January 2018, following which the first FLIP event was held in Paris in June 2018. Organized by the DEPP of the French ministry of education, this 2-day meeting was attended by representatives of at least 13 countries and corporate bodies. Participants benefited from rich inputs and exchanges related to: country experiences in e-assessment, open standards, product demos, platform functionalities, broad issues related to security, confidentiality, usability, data analytics, reporting etc. and a hands-on session with respect to the development of technology-enhanced items. A slot was also dedicated to discussions on how the FLIP initiative could grow as a partnership organization to serve its members, everyone leaving the meeting convinced that this was a promising start to a fruitful and long-lasting community.
At this stage, an official framework for FLIP still needs to be drawn up and officialized. The intention is to set out FLIP’s governance structure and mode of functioning as well as detailing how costs would be shared. Meanwhile, should you wish to learn more about FLIP, you may contact Dr. Thierry Rocher, DEPP-France at email@example.com.
International e-assessment award 2018 won by Luxembourg’s MathemaTIC digital environment
At the 2nd international e-assessment award ceremony in London in April 2018, MathemaTIC won the “Best Use of Formative Assessment Award” for its innovative approach to large-scale formative assessments. MathemaTIC is a top innovation project of the Luxembourg’s Ministry of National education, Children and Youth, which capitalizes on technology to adapt the learning of mathematics to individual student needs using a “language-free” solution. It is a product of an ongoing international collaborative partnership between schoolteachers, researchers, curriculum specialists in Luxembourg and France who have been working together since 2015 with a technology partner, Vretta, from Canada. The product is a digital personalised multilingual learning environment for learning mathematics.
Tailor-made to Luxembourg’s national mathematics curriculum for Grades 3-8 students, the learning content is available in German, French, Portuguese and English. This brings a high value-added for students who have a lower command of the language of instruction (which is German) and whose achievement in mathematics would be hindered when the language becomes a barrier to understanding the mathematical text. This aspect is particularly significant for Luxembourg which is the European country with the highest proportion of students who do not speak the language of instruction at home, with the highest number of foreign languages taught at school and the highest number of hours dedicated to the teaching of these foreign languages (Eurydice and Eurostat, 2012). To alleviate this multilingual challenge and foster a highly interactive learning experience, MathemaTIC diagnoses the skills of the learners and formatively determines their sequence of learning, providing a truly personalised learning experience. It enables dynamic switching between languages, supporting both numeracy and language growth and enabling effective teacher and parent intervention. It also enables effectively monitoring and supporting through real-time data visualizations. The successful implementation of MathemaTIC in Luxembourg has enabled the expansion of the use of the platform and interactive assessment items in Portugal and France. Luxembourg is now open to broadening this collaborative model with other interested countries who may wish to review and customize the platform for their use.
To learn more and participate in the MathemaTIC project, please contact the project coordinator, Amina Afif firstname.lastname@example.org or email@example.com. Alternatively, feel free to visit the MathemaTIC website www.mathematic.lu.
Europe’s online platform for school education focused on assessment for learning
The School Education Gateway – Europe’s online platform for school education operated by European Schoolnet on behalf of the European Commission – recently published a series of articles on assessment in school education. Opening the theme, Dr Yiasemina Karagiorgi of the Cyprus Centre for Educational Research and Evaluation argues in her article that assessment of learning is the first step, in order to identify and monitor students “at risk”, and that assessment for learning is a necessary addition in the context of student literacy.
A recent School Education Gateway survey also focused on assessment for learning. The results indicate that summative assessment indeed is used more than formative assessment (N=534). While the respondents seem to understand the benefits of formative assessment, the vast majority of respondents (86%) indicate they need more training to use formative assessment effectively. Furthermore, more than half of the respondents (57%) agree or strongly agree that there is a lack of useful digital resources available for formative assessment.
The monthly Practice article highlights three inspirational projects that address day-to-day challenges of teachers regarding learner assessment helping teachers develop effective, ongoing assessment practices that support students’ learning.
Other articles published by the School Education Gateway on the topic are the following:
- Publication: A literature review on assessment and learning shows the important relation between assessment and learning and its effect on quality in education.
- Publication: Ten notes on learning assessment systems
- News: The ATS2020 project created a comprehensive learning model supporting the development and assessment of students’ transversal skills.
- Teaching Material: The Irish National Council for Curriculum and Assessment released a set of booklets and associated materials supporting a whole-school approach to professional development and capacity building in the area of student assessment.
- Teaching Material: Inclusive student assessment: training tools for curriculum development
Also available: a full list of articles together with a brief introductory text for each item.
Presented in 23 European languages, the School Education Gateway is a single point of entry for teachers, school leaders, policy makers, experts and other professionals in the school education field. The platform aims to support quality development of school education and to promote European policy and practice for school education on the EU’s nine key priorities in school education.
Firstly, the platform publishes monthly articles on a specific topic in school education, including expert interviews, practice articles, news items and teaching materials – of which the abovementioned items are great examples.
Secondly, the platform supports the professional development of teachers, by hosting free online courses and webinars on a wide range of subjects. Additionally, the website has services for school interested in Erasmus+ funding opportunities: a catalogue of on-site courses for teachers and staff, a directory of mobility opportunities, and a strategic partnership search.
For further information, please contact Koen Glotzbach Koen Glotzbach, firstname.lastname@example.org at School Education Gateway, European Schoolnet.
Kathleen Tattersall New Assessment Researcher Award 2018
AEA-Europe is pleased to announce that the winner of the Kathleen Tattersall New Assessment Researcher Award for 2018 is Dr. Maria Bolsinova, research scientist at ACTNext, the Netherlands.
Dr. Bolsinova’s work was considered by the jury as truly impressive. With an exceptional CV, she has already contributed to a high number of publications in top-ranked journals. Her work on the combined use of response accuracy and response time in educational assessment has been and continues to be a significant issue in terms of the design of assessments, and their claims to reliable and valid outcomes. This work, although complex, casts some clear light on how response times can be usefully incorporated into the tool kit of assessment design, evaluation and modelling – and not least, her work already includes contributions to improved understanding on how models may be specified without relying on some restrictive and in many cases, unrealistic assumptions.
We have high expectations that Dr. Bolsanova will make a huge impact over the next years, both in terms of producing work in the frontier of psychometrics, and in terms of applications of these ideas into the practice of assessment. Don’t miss her keynote talk at the conference in Arnhem-Nijmegen in November 2018, entitled “Response times in educational assessment: Moving beyond traditional assumptions”.
For more information about the KTNA award, please contact: Rolf Vegar Olsen of AEA-Europe’s Professional Development Committee, at email@example.com.
The Learning Portfolios in Higher Education
The growth in the use of Learning Portfolios has been especially notable in higher education (Eynon & Gambino, 2017; Kunnari & Laurinkainen, 2017). In this context, they are used predominantly to enhance learning, as universities face growing demands to produce so-called “T-shaped graduates” (i.e. those equipped not only with disciplinary specialization, but also with generic, cross-curricular skills), however, the secondary “showcase” value of the tool is also keenly recognized. Indeed, it seems that, in higher education, Learning Portfolios are perceived primarily as a catalyst in the learning process, but also as an effective platform through which learners can demonstrate notoriously hard-to-measure skills such as critical thinking and creativity.
The report “The Learning Portfolios in Higher Education” provides an integrative review of the recent literature on the use of Learning Portfolios in higher education, with the overall aim of discerning whether or not they can be considered to be ‘effective’ in terms of their impact on learning outcomes. It is acknowledged that potentially relevant literature dates back to the early 1990s, although, as Klenowski, Askew and Carnell (2006) pointed out, much of this focused solely on the use of portfolios for assessment purposes. Furthermore, there has been a noticeable surge in the number of papers published on the topic in recent years, reflecting factors such as the facilitating influence of interactive Web 2.0 technology in the development of various platforms, and the launch of the International Journal of ePortfolio in 2011. In light of this cumulative pattern, and to ensure that the most recent developments are taken into account, this paper focuses specifically on research conducted since 2010.
Despite the progressive growth of literature on learning portfolios in recent years, our knowledge of these tools remains rather limited (e.g. Bryant & Chittum, 2013; Rhodes, Chen, Watson, & Garrison 2014). Although there is a strong theoretical foundation for their use, a consultation of the research reveals insufficient empirical support for their effectiveness, and a clear need for more sophisticated evaluations. There is also a need for more methodologically robust studies that triangulate outcomes (as measured by achievement data and demonstrable competencies) with the self-reported attitudes and perceptions of key stakeholders.
It is evident that portfolio implementation can be fraught with difficulties, due to insufficient understanding of processes, and tensions between the developmental and evaluative aspects. Difficulties with technology have also been cited, but these are somewhat superficial, and typically mask deeper pedagogical deficiencies. It cannot be taken for granted that instructors – let alone learners – comprehend the key processes involved in their creation. Moreover, it seems that learners will need to be granted greater autonomy in selecting the nature of the portfolio artefacts as well as the platform used to create them. Successful and sustainable use of learning portfolios in higher education requires considerable planning and preparation, and a substantial commitment from all stakeholders. Without these, the experience is likely to be as Joyes et al. (2010, p.493) described, “like a game of snakes and ladders, where initial rapid progress can suffer major setbacks.”
For further information, please contact Prof. Michael O’Leary, firstname.lastname@example.org at the Centre for Assessment Research, Policy and Practice in Education, Institute of Education, Dublin City University.
Vacancies for PhD and postdoc positions: Centre for Educational Measurement in Oslo
The Centre for Educational Measurement (CEMO) at the University of Oslo is presently advertising three full-time PhD positions (4 years) and two full-time Postdoc positions (3-4 years) in order to increase its range of activities. CEMO currently specializes in educational measurement and has nearly 30 employees with more than 10 nationalities. It has a large portfolio of projects and research opportunities.
If you wish to apply for one of these positions and are unsure about moving to Norway, or you would like more information about what these positions would imply in the CEMO context, please refer to https://www.uv.uio.no/cemo/english/about/vacancies/.
For more information, contact Rolf Olsen, Professor and acting director CEMO by email: email@example.com.