Evaluation

iTEC classrooom pilots: final evaluation results

September 2014

Evaluating the impact of iTEC Learning Activities in schools

Over the four years, the evaluators gathered the views of teachers and students (some 1,488 were surveyed), national coordinators and policy-makers through surveys, interviews, focus groups, case studies and observations. The results were collated under three headings in the final evaluation report.

How did the iTEC approach impact on learners and learning

• Key finding 1: Teachers perceived that the iTEC approach developed students’ 21st century skills, notably independent learning; critical thinking, real world problem solving and reflection; communication and collaboration; creativity; and digital literacy. Their students had similar views.
• Key finding 2: Student roles in the classroom changed; they became peer assessors and tutors, teacher trainers, co-designers of their learning and designers/producers.
• Key finding 3: Participation in classroom activities underpinned by the iTEC approach impacted positively on students’ motivation.
• Key finding 4: The iTEC approach improved students’ levels of attainment, as perceived by both teachers (on the basis of their assessment data) and students.

How did the iTEC approach impact on teachers and teaching?

• Key finding 5: The Future Classroom Scenario development process was viewed as innovative by policy makers, teachers and stakeholders, but further work is needed.
• Key finding 6: Teachers and coordinators perceived that the Learning Activity development process has potential to develop innovative digital pedagogies in the classroom, but further work is needed.
• Key finding 7: Teachers perceived that the iTEC approach enhanced their pedagogy and digital competence.
• Key finding 8: Teachers became more enthusiastic about their pedagogical practices.
• Key finding 9: Teachers stated that they used technology more frequently; it was systematically integrated throughout the learning process rather than reserved for research or presentations.
• Key finding 10: Teachers were introduced to digital tools they had not used before; some were more favourably received than others.
• Key finding 11: Teachers collaborated more, both within and beyond their schools, a process facilitated through the online communities.

What is the potential of the iTEC approach for system-wide adoption in schools?

• Key finding 12: Awareness of the iTEC approach is growing in educational systems, and there are signs of widespread uptake.
• Key finding 13: The scenario-led design process can support mainstreaming of innovation, provided the process is refined.
• Key finding 14: The library of scenarios, Learning Stories and Learning Activities was viewed by policy makers and teachers as a valuable output of iTEC to support system-wide classroom innovation.
• Key finding 15: In countries in which iTEC aligns closely with national policies and strategies, the iTEC approach is likely to be adopted and to influence future practices.

Download the FULL EVALUATION REPORT here (in English, PDF).

A summary of the evaluation report is available in various languages (PDF file):

• English
• Danish - dansk
• German - Deutsch
• Finnish - suomi
• French - français
• Hungarian - magyar
• Italian - italiano
• Lithuanian - lietuvių kalba
• Norwegian - norsk
• Portuguese - português
• Slovak - slovenčina
• Spanish - español
• Turkish - Türkçe

 

Related deliverables

The following deliverables are currently available:

Cycle 1 Evaluation handbook About You and Your School (AYYS) questionnaire   Teachers questionnaire (TQ)  Evaluation report

Cycle 2 Evaluation handbook About Your and Your School (AYYS) questionnaire   Teachers questionnaire (TQ) Evaluation report

Cycle 3 Evaluation handbook Teachers Questionnaite (TQ) Evaluation report

Cycle 4 Evaluation handbook National Case Study summary report Teachers questionnaire (TQ)  Scenario development progress report Evaluation report

 

Evaluation findings so far

July 2013 - Cycle 1-2-3

During the past three years the iTEC project has developed effective mechanisms for ensuring that the advancement of schools, and of teaching practice keeps pace with changes in society and technology.

A detailed meta-analysis of the evaluation data over the first three cycles of iTEC pilots shows that the process has had a positive impact on students’ knowledge, skills and understanding – in particular on their 21st century skills, motivation, engagement,  attitudes, and learning practices. iTEC has also had a beneficial effect on teachers, impacting positively on their technology-supported pedagogy, digital competence, and their motivation and attitudes. Moreover, iTEC is seen as scalable, having the potential to support pedagogical and technological innovation, to increase the effective use of ICT and to enable teachers to experiment with innovative technologies and tools.

• Read here the full report "Evidence of the impact of iTEC on learning and teaching" (PDF)
• Read article "Evaluating the impact of iTEC Learning Activities in schools" in the ITEC magazine (October 2013)

 

---

January 2013 - Cycle 2

Cycle 2 involved 420 pilots from 15 countries. Each teacher piloted a Learning Story with at least one group of students or cohort; in some cases, teachers piloted a Learning Story with more than one cohort completing a separate questionnaire for each. Responses were received for 298 cohorts in Cycle 1 (an overall response rate of 71%) from 261 teachers.

The findings included among others:

• There was an increased use of digital tools with over two thirds of teachers using tools they had not used before and perceiving that their use was essential. In particular 60% of teachers used TeamUp and 52% used social networking tools such as Facebook or Edmodo. Use of ICT also emerged as a benefit for a small number of teachers (main benefit: ES, NO; also noted by individual teachers in PT, LT, TR).
• The three most important enablers identified by teachers were student motivation, ICT (access and infrastructure), and teacher motivation.
• Over half of the 10 main challenges that teachers identified in the survey are technology related. They include insufficient ICT access, unreliable Internet access, and controlled Internet access through filtering for example.
• Training and support were positively received by teachers who particularly enjoyed face-to-face meetings, networking with other teachers, opportunities for hands-on experience of tools, online discussion forums, webinars and video-tutorials.

For more results please see the presentation below and/or read the complete evaluation report. 

 

---

September 2012 - Cycle 1

Improving teachers’ skills

Teachers in the iTEC pilots, from the 17 countries involved in the first cycle, perceived that the learning stories had introduced a range of different pedagogical strategies. 86% of teachers reported that the learning story they piloted presented exciting opportunities to do things differently in the classroom.

“Here there are people who come into the class... people who are journalists, photographers, illustrators... I really like it because we discover lots about jobs which we didn’t really know about before.” France, student interview

iTEC Learning Scenarios are intentionally designed to introduce 21st Century skills including, for example, opportunities for collaboration and creativity. The majority of teachers (88%) agreed that the Learning Stories led to more opportunities for collaboration and 89% agreed that students had used digital tools to support collaboration.

Many specific benefits of this approach emerged from the case study data including enhanced motivation, knowledge building and development, improvements to the social dimension of learning (for example, relationships and interactions, classroom ethos). Teachers in France referred to interactions being ‘real’ and students improving their relationships with their peers with whom they had not worked before. Students found it more fun than other (traditional) approaches which, for example, a Turkish student described as ‘monotonous’.

70% of teachers agreed that participation in the cycle 1 piloting had enabled them to develop their own creative skills. 77% of teachers agreed that there was a positive impact on students’ attitude and 82% agreed that students were deeply engaged in their work.

“The iTEC project is important because it allows us to experiment with new learning scenarios so that we can develop alternative models compared with how we have traditionally worked.” Head teacher

“So the pupils have the opportunity to experiment with new ways of learning and be protagonists in changing contexts and integrating new and old resources.” Italy, head teacher interview.

The survey data suggests that teachers felt that the learning stories led to creative learning. 88% teachers felt that the learning story enabled creative activities to take place and enabled students to develop their creative skills. In addition, 87% of teachers agreed that the learning story enabled students to express themselves in new ways.

Student-centred pedagogical strategies and new methods of assessment

Through the evaluation, a range of themes emerged regarding active learning, learning to learn, knowledge building, student autonomy, drawing on students’ interests, and student choice. 88% of teachers agreed that students were actively involved whilst 70% agreed that they could work at their own pace.

“The students that were interviewed explained that the commitment and motivation they felt was due to the fact that they were given much responsibility and freedom of choice and were dominant in the preparation of materials for the lessons.” Israel, case study report

“Students feel very proud of themselves because they have worked alone, because they have been able to collaborate and because now they think that they are able to have a conversation with an expert.” Spain, teacher interview

“According to the school head teacher, experiments like this bring a positive new approach in schools, where teachers are no longer mere ‘bearers of knowledge’, and pupils ‘become partakers of the construction of their knowledge’.” Italy, case study report

“As there is task-sharing in the groups we learn to take responsibility and it becomes easier to exchange information from each other.” Turkey, student interview

In the teacher questionnaire 80% of teachers agreed that the learning story had enabled them to assess students in a new way.

“Once in our history lesson we were part of a club, everybody was some historical person and teacher asked us questions and we had to react as the person would. This was assessed and it was very interesting.” Slovakia, student interview

“A discussion forum was started on [the VLE] for the students to give positive feedback on the materials. Students were given ground rules that they should give constructive feedback and positive comments. They really enjoyed using the forum to give comments about the resources and each other’s resources.” England, teacher evaluation

The impact of technology

A variety of technological tools have been used during piloting to collect data (scientific measurement, photographic evidence), analyse data and create presentations and podcasts, and assess learning. In Belgium, the use of iPads was perceived to be ‘easy ICT’ and so seen to be an enabler. For the “Bring in the Expert” learning story, Skype and email were used to facilitate communication with experts.

Technology as such was not seen as the core focus of the project – most teachers understood that what is important is the way in which technology is integrated and the pedagogical strategies facilitated by such tools. Students and teachers reported that the technology was motivating, but they also spoke about working together and collaborating as equally important. In addition, technology was seen to make linking home and school easier.

“In cycle 2 the teachers and students worked, as part of collaboration, with SMART Bridgit conferencing software to work together or to present each other results of the pilots. The Spanish and German teachers and students are staying in contact and now multicultural aspects are part of these pilots as well. SMART Bridgit helps them as a collaborating tool to write or paint what they mean. It is easy to use and to combine with SMART solutions (e.g. SMART Board). Via camera, sound and the SMART Board they can use, present and work very intuitively on their pilots and results. The feedback was good and the pilot classes got an overview of other existing pilots in Europe. The learning aspects are positive too because the two pilot students groups present the results themselves with the assistance of SMART Bridgit”. SMART Technologies, National Coordinator.

“The teacher experienced that her pupils were very much motivated by the use of technology. [...] Pupils love the possibility to use technology and they told us that the English class is unique from this point of view: they don’t use web 2.0 tools in other classes, neither do they have online access to learning materials.” Hungary, case study report

“I did not need to use internet and computer out of curricular purposes with the application of this kind of learning. I was chatting and communicating for social purposes. Now I do not need social networking sites. Rather, I am doing school homework with my friends on the internet. Also, I’m using social networking sites to share information purposes with my classmates.” Turkey, student interview

Multiple positive impacts

The evaluation process covers several features of importance to the design of the future classroom. Whilst it is still early in the project, the evaluation results have been positive:

• 75% of teachers agreed that their knowledge of the pedagogical use of ICT had increased.
• 78% of teachers agreed that their understanding of the potential of ICT had developed.
• 77% of teachers felt that the learning stories had a positive impact on students’ attitudes to learning.

The teachers’ commitment to using iTEC learning stories and activities again, outside of the project, is a key indicator of their perceived value and benefit. 53% of teachers suggested that they would definitely implement the learning story they piloted again in the future, and of the remainder, 45% said that they would probably implement it. Only 3% of teachers suggested that they would not implement it again.