Principal Investigator(s): Meltem Alemdar
The federal Ed Tech initiative was developed in an effort to improve student academic achievement through the use of technology.
In Georgia, this goal was used as a platform in creating the Title IID “Engaging AP Students in Mobile Handheld Computing” grant, awarded to 66 schools across 50 school districts throughout Georgia in 2009. This three-year initiative aimed to improve the quality of Advanced Placement (AP) courses, increase enrollment in AP courses, and increase diversity in AP courses through the use of handheld technology. The grant made a variety of mobile handheld devices, such as iPods, netbooks, smart boards, projectors, and Wi-Fi servers available in AP classrooms in order to enhance classroom instruction. Selected by the GADOE as the external evaluators for the grant, the evaluation team at CEISMC has assessed the extent to which the Title IID Mobile Handheld Computing grant met its goals and objectives.
The designed evaluation plan for Title IID was summative in nature, focusing on the outcomes of the program. It emphasized multiple data collection methods and exploratory analyses. The key evaluation question was “to what extent has the program been successful in integrating technology in mathematics classrooms?” The evaluation utilized a mixed methods approach employing both qualitative and quantitative data sources to determine program outcomes and effectiveness. Data sources included:
- Surveys – Program administrators, participating teachers, participating students
- Focus Groups/Interviews – A sample of teachers and administrators were interviewed at each site visit location, separately.
- Classroom Observations – The International Society for Technology in Education (ISTE) Classroom Observation Tool (ICOT) was used to measure the technology usage in classrooms.
- Student Achievement Data - Student achievement was measured using Criterion-Referenced Competency Tests (CRCT) scores for participating school AP courses
The sub-set of classroom observation data and interviews with teachers and administrators were used to supplement survey and student achievement data collected as a part of this program evaluation.
Teachers and students used the technology in the AP classrooms regularly and for a variety of purposes.
Evidence of the utilization of handheld mobile devices in the AP classrooms was apparent across data sources. Student and teacher surveys indicated that the iPod Touches were the most commonly used of the devices, followed by laptops. The frequency of technology reported by teachers and by students was almost identical: 60% of students and 61% of teachers indicating at least weekly utilization of the devices in the classroom. Furthermore, 14% of students and 13% of teachers indicated daily utilization of technology. This close similarity in response rates by teachers and students strengthens the conclusion that these devices were regularly used in the AP classroom. Teacher interviews also supported the conclusion that technology was utilized in the classroom at least once a week.
Indications of increased student engagement were consistent across data sources.
Responses provided by students and teachers through the surveys and by teachers and administrators through interviews supported the conclusion that students were more engaged in the AP courses as a result of the incorporation of the mobile handheld devices. Surveys showed that 75% of teachers indicated that they agreed or strongly agreed that students were more engaged, and 91% of students indicated that they were a little or a lot more engaged in their AP course as a result of the devices. Additionally, when asked to identify the greatest impact of the program upon students, 34% of teachers and 16% of administrators stated increased student engagement. Additionally, the observation of the AP classrooms showed that 83% of students were engaged throughout the entire observation period.