Evaluation of 2013 – 2016 MSU Elementary Education Graduates’ Teaching

 Research Brief Title

An Evaluation of 2013 – 2016 MSU Elementary Education Graduates’ Teaching

As Measured by Missouri Educator Evaluation System Data: An Examination of Methods and Limitations to Assess Impact on Student Learning

Research suggests there are many variables related to student achievement; however, “there is no in-school intervention that has a greater impact on student learning than an effective teacher” (NCATE, 2010 p. 1).  Thus the role that universities play in the preparation of highly qualified and effective teachers is of extreme importance. Teacher education programs have long been criticized, perhaps unfairly, for an approach that often consists of on-campus coursework disconnected from current instructional practice. As such, “learning to teach in isolation does not effectively prepare teacher candidates, nor does it benefit P-12 students” (Heck, 2014).  With many calling on institutions of higher education to transform their teacher preparation programs by reforming curriculum, structure and pedagogy (Strieker, Shaleen, Hubbard, Digiovanni and Lim; 2014), the Council for the Accreditation of Educator Preparation (CAEP) is compelling “urgent changes in educator preparation” (Bullough, 2014).  Yet the question still remains, how do universities “measure” the impact of their teacher preparation programs on student achievement in P-12 classrooms?  Moreover, how do universities work collaboratively with local school districts to collect evaluation data in an effort to determine program effectiveness?

Professional Continuum of Missouri Teacher

Missouri’s Evaluation System

In May 2013, the State Board of Education approved the new Missouri Educator Evaluation System (MEES) which includes nine standards on which both pre-service and practicing teachers are to be evaluated. School districts were given the 2013-2014 school year to adopt the new evaluation system or align their current evaluation system to the state model.  Based on a 7-point scale that spans “candidate” to “distinguished,” evaluations begin during pre-service teaching for interns and student teachers and is carried forward throughout their careers. All summative evaluation scores which are reported to the state must include some indication of the teacher’s impact on student learning.

Sample Study One

In the summer of 2016, the Missouri Department of Elementary and Secondary Education (MoDESE) provided the MSU research team with the names and school assignments of graduates working in Missouri public school districts within their first three years of teaching (2013-2015).  In order to maintain anonymity, researchers identify those districts across the state that had at least three teachers working in their respective districts with 1-3 years’ experience.  This resulted in a final sample of 284 MSU graduates working in their first three years of teaching in 45 Missouri school districts.

District superintendents were contacted and asked to participate.  Nineteen of the 45 (42.2%) agreed to take part in the study resulting in 139 MSU graduates working in 82 different school buildings. With permission granted from the superintendents of the 19 participating districts, researchers began contacting individual school principals to obtain end-of-year evaluation data. Thirty-seven of the 82 (45.1%) schools provided data. Using the rating system of 0-7, these principals reported that on average veteran teachers in their buildings scored an average of 5.44 while new teachers who graduated from MSU scored an average of 5.06. Given variance within self-report data, actual teacher evaluations were examined in one small school district willing to share information.  The results were similar with veteran teachers scoring an average 5.8 and new teachers scoring 5.5. Interestingly, both new and veteran teachers were rated at the proficient level of the professional continuum as measured by the Missouri Educator Evaluation System.

Problems & Limitations of Sample Study One

Through this process, researchers discovered that the type of evaluation model/system used by districts varies greatly ranging from the DESE (state) model, Network for Educator Effectiveness (NEE), and district developed data collection methods.  Districts also collect data using different scales.  While some utilize the recommended 0-7 scale, others may use only a four-point system. In addition, evaluation data are difficult to obtain from school districts as few maintain a database that contains the institutions of higher education from which teachers obtained initial degrees.  Currently there is no system that links a teacher’s evaluation to student academic achievement. 

Sample Study Two

During the fall 2017, one mid-sized school district in southwest Missouri partnered with Missouri State University’s College of Education in an effort to untangle these questions.  Focusing only on elementary certified (grades 1-6) personnel hired during the fall of 2014, 2015 and 2016, teacher evaluation data on both MSU graduates as well as non-MSU graduates was shared with the university.  These data include the average teacher effectiveness ratings (ranging from 1-7) for (1) Classroom Climate, (2) Planning and Preparation, (3) Instruction, (4) Assessment, and (5) Professional Responsibility.

                                                                                                 Table 1

*Note: Less than three per group, data not reported.
In total, teacher evaluation data for 25 of the district’s new hires during 2014-2015 through the 2016-2017 school years were provided.  Of these, fourteen were elementary certified graduates from MSU and eleven were elementary certified graduates from other institutions.

By examining these new teachers’ year-end evaluation summative data, comparisons such as the overall difference in average effectiveness across first year teachers prepared by MSU and by other institutions are displayed.  Within this sample scenario, MSU graduates tend to score higher, on average, than new teachers who graduated from other institutions for each of the three years reviewed. Overall, MSU first year teachers scored an average of 5.1 on a 7-point scale while non-MSU graduates scored an average of 4.6. MSU first year teachers also tend to score consistently higher on each of the five areas evaluated by the school district. However, as teachers spend more time in the school environments these differences appear to become less noticeable after year two with the difference between MSU graduates in the second year of teaching scoring 5.6 and non-MSU scoring 5.3.  Both groups demonstrate growth over time within their respective schools.

Average Evaluation Scores for New Hires

*Note: 2014-2015 group = less than three per group, data not reported.

Problems & Limitations of Sample Study Two

The intended purpose of these this examination of district data was to explore the feasibility of developing methods to determine whether or not valid data regarding teacher quality could be linked to MSU educator preparation programs separate and apart from others.  While these are examined and presented in this brief, they represent a very limited sample from one school district and from only one certification area.   Serendipitously, this collaborative data collection effort demonstrates that such information can, in fact, be collected and shared without publicly identifying schools or teachers.  In addition, the methods described herein demonstrate the additional research questions can be addressed should this method be replicated to include additional schools/districts generating a larger sample of first year teacher graduates in a variety of difference certification areas. Doing so would allow for statistical procedures to be employed, producing more scientific findings.

Average Effectiveness by Area for New hires

Average evaluation score for teachers

Next Steps for Studying Teacher Effectiveness

Given the two sample studies presented here, researchers recommend collaborative data sharing between institutions of higher education and P-12 public schools throughout Missouri take place in order to examine more precisely the impact of educator preparation programs on effective classroom teaching and student learning. While districts utilize varying data tools and systems to collect evaluation data on their teachers, all districts must adhere to the Missouri Educator Evaluation Standards.  These are the same Standards that all educator preparation programs must also utilize when evaluating pre-service teacher candidates during their student-teaching or internship experience.  Sharing these data across both systems while focusing on a teacher’s impact within the first two years of teaching would allow universities to better understand the impact of their programs and provide public schools with information regarding the professional development needs of the first-year teacher.