Overview
Teacher work samples provide prospective teachers with a format for reflecting on their teaching and the impact of their teaching on student learning. A growing body of research confirms the use of work samples as a credible evaluation measure of teacher candidates’ skills in meeting targeted teaching criteria (Salzman et al. 2001; Schalock 1998; Hegler 2003; Girod, 2002). Researchers state that a relationship between knowledge of teaching and learning acquired in teacher preparation programs and student achievement can be shown through the work sample process. Salzman, Denner, Bangert, and Harris (2001), through work with The Renaissance Partnership Teacher Work Sample, discovered a direct link between the teaching process criteria and teaching behaviors as measured by the work sample. Also discovered was a significantly high positive correlation between independent evaluations of the quality of the learning assessments used and the total work sample performance rubric.

The purpose of this paper is to describe the benefits of scaffolding the work sample criteria through a teacher preparation program for elementary education candidates. This article illustrates the way that Fort Hays State University’s teacher education program has successfully implemented a work sample process. A process of scaffolding the criteria was woven throughout the elementary teacher education program to instruct candidates on how to develop, implement, and assess instructional units of study. As a way to describe the teacher work sample process, the authors have contextualized the model, provided components of a teacher work sample, and presented a program design for the Fort Hays State University process, which includes preliminary data of a longitudinal study.

Contextualizing the Model
The National Council for Accreditation of Teacher Education requires teacher preparation programs to document and verify teacher candidates’ performance for bringing about PreK-12 student learning. As part of the accreditation process, our university, as well as other universities such as California State, Eastern Michigan, and Idaho State, has initiated a work sample approach, Fort Hays State University Performance Assessment (FPA), to validate the quality of candidate performance. Candidates at the initial level of elementary education are guided through the work sample process. Candidates begin the work sample process while taking Foundations of Education and Diverse Learners courses, however the bulk of the process is introduced upon entrance into the teacher education program (see Figure 1). Methods courses and internships provide candidates with knowledge of the work sample criteria and practice in preparing a unit of study. The candidates are also provided opportunities to teach their units to elementary students. Candidates are taught to administer diagnostic and summative assessments as a means of establishing a gain score for determining student learning. Importance is placed on data collection and analysis of student learning. The use of that information to guide teaching and learning is a key benefit to the work sample model (Imig and Smith 2002). As a fundamental part of teaching a unit of study, candidates reflect on their teaching by addressing the seven criteria of the work sample process.

The student teaching semester provides a final opportunity for candidates to develop, teach, and reflect upon an additional unit of study using the work sample process. Candidates, engaged in a semester-long student teaching experience, teach the unit under the guidance of a mentor teacher and university supervisor(s) and then reflect on student learning using a work sample rubric. A 25-page paper is prepared to document the process. The work sample process used follows the Renaissance group’s format (Salzman et al. 2001).

Very little research has been done on the preparation of preservice candidates to write work samples. At Fort Hays State University, the Teacher Education Unit faculty members scaffold instruction on work sample criteria throughout the teacher preparation program allowing candidates to be more successful and confident in their abilities. By learning about the work sample through a sequential, guided approach, the candidates should be more capable of producing well-written, reflective evidence of performance.

The Fort Hays State University Performance Assessment (FPA) requires candidates to write responses to specific criteria in a sequential and systematic manner before graduating from the Department of Teacher Education. The work sample system, developed by university faculty with the support of the state department, mentor and cooperating teachers, and candidates, is a systematic approach that has all stakeholders working toward a common purpose. Candidates receive training in and experience teaching and assessing student learning in coursework, internships, and student teaching venues. Early in their careers, candidates become acquainted with the process and each criterion. The teacher candidate is required to plan and teach several units of study. Before teaching each unit, candidates describe contextual factors, identify learning goals based on state and district content standards, and create an assessment plan designed to ensure student performance before, during and after instruction. After teaching each unit, the candidate calculates gain scores, then reflects upon and evaluates the teaching experience related to student learning. Documentation of the criteria is used as partial fulfillment of course objectives.

The sequential introduction of and writing to each criterion is the first level of scaffolding of the work sample process. Criteria one, three, and four of the FPA are introduced in Diverse Learners, a course required of all candidates prior to being admitted into the elementary teacher education program. Since criterion one addresses demographics and the make-up of the class, school, and community, candidates in Diverse Learners learn to evaluate and respond to a variety of demographics. While developing instructional units, candidates plan for adaptations and modifications relative to student diversity while writing criterion three. In criterion four, candidates integrate instruction across or within subject matter fields. Criterion two, which looks at learning goals and objectives, is introduced in the Foundations of Education course. The instructor of the Effective Classroom course, which is taken after admittance into the Teacher Education program, provides initial training on criterion five.

Upon being admitted to teacher education, candidates are required to take three methods courses, each having corresponding internships. These three methods course are Reading and Language Arts, Elementary School Social Studies, and Mathematics and Science. Within each of these courses candidates learn about and respond to multiple criteria on the Fort Hays State University Performance Assessment (FPA) score sheet (see Figure 2). Each course carries requirements that candidates, often working in small teams, follow the rubric and develop a thematic unit (see Figure 3). Instructors of Reading and Language Arts, Elementary School Social Studies, and Mathematics and Science Methods and their corresponding internships provide the initial training on criteria six and seven and continue to help candidates develop the ability to write to the other five criteria. Candidates are required to create and teach units of study that follow the seven criteria in each methods course. Evidence of performance must be provided by the candidate for criteria two, six, and seven. By following the rubric for criteria (Figure 2) and creating a unit, candidates develop the skills necessary to write a work sample. Thematic units help candidates learn to address the criteria of the FPA through practical application.

The next level of scaffolding is a sample FPA. This document follows the same performance assessment criteria, but is written during a fourth methods course and its internship, Corrections of Reading Disabilities. In conjunction with the course, candidates tutor elementary students in reading two days a week for ten weeks. Throughout the course and reading tutorial internship, candidates receive instruction on the seven different criteria, implement their instructional unit, and write a sample FPA. Because candidates are working with a small group of learners, they are able to develop, implement and reflect on the lessons taught. This is the first time candidates are required to teach the lessons they created for the performance assessment process and write an entire sample FPA. Upon completion of the document, instructors of this course read and score the sample FPAs using the scoring rubric in Figure 2 and provide feedback to candidates.

The final level of scaffolding is the candidates’ final FPA. Candidates create at least five lesson plans, include a pre/post test and actually teach the unit during their student teaching experience. Then they write to each of the seven criteria of the FPA and include any resources used. Upon completion of the final FPA, candidates submit two copies for faculty review. In an all day scoring session, two reviewers who provide the candidates with written feedback score each copy. If there is a discrepancy of more than ten points in the two scores, a third reviewer scores the document. By having multiple reviewers score the performance documents, inter-rater reliability is affirmed. The scoring session is preceded by practice scoring of a work sample and discussion so that all scorers review documents in a consistent manner.

Training and inter-rater reliability were instrumental when using work samples to assess teacher performance. Salzman, Denner, Bangert, and Harris (2001) found that inter-rater reliability could be established to a sufficient level to allow decisions to be made regarding the quality of teaching performance. Inter-rater reliability can be enhanced using multiple raters, using pilot readings and discussions, and providing a third reader when scores have a wide discrepancy. Heiman (1998) recommends a 90% or better agreement among raters.

Components of a Teacher Work Sample
The Fort Hays State Performance Assessment (FPA) is a written document, not exceeding 25 pages, describing the events throughout an instructional design and allows for evaluation of the candidate’s performance, personal growth, and professional development. The purpose of the FPA is to evaluate a candidate as he or she develops, implements, and reflects on his or her impact on student learning. The FPA provides candidates the opportunity to document credible evidence that they possess the knowledge and the skills to facilitate learning for all types of students.

In order to make pedagogical decisions, the candidate: analyzes his or her classroom context; constructs and delivers an instructional plan; develops meaningful diagnostic, formative, and summative assessments; evaluates students’ learning; provides information on data, student achievement, and the accreditation status; and analyzes and reflects to promote professional growth. After designing and teaching the unit, the candidate writes a narrative document which sites evidence in seven criteria: 1) contextual information and learning environment adaptation; 2) learning goals and objectives; 3) instructional design and implementation; 4) demonstration of integration skills; 5) analysis of learning environment; 6) analysis of assessment procedures; and 7) reflection and self-evaluation. The work sample was to be submitted to the state department within the first two years of conditional licensure. The document was reviewed and scored by state educators. In order to receive professional licensure teachers had to receive a passing score of 88 out of 113 possible points.

The KPA handbook (Kansas State Department of Education 2007) outlined the work sample process that candidates used to design a unit of study to be taught in the classroom. This documentation of evidence is based on what the candidate knows, what he or she is able to do, and his or her effectiveness for bringing about student learning.

The seven FPA criteria are based on educational and measurable checklist and scoring rubric areas previously listed in the KPA. When writing FPA Criterion 1, Contextual Information and Learning Environment Adaptations, the candidate “understands of how individuals learn, understands of child and adolescent development, demonstrates knowledge of appropriate adaptations, and has the ability to provide instructional strategies that afford learning opportunities for all learners” (p. 22). This criterion measures the candidate’s ability to provide information about his or her awareness of contextual factors and data in order to examine his or her ability to function as a professional educator in a diverse setting. FPA Criterion 2, Learning Goals and Objectives, the candidate “selects goals and objectives based upon knowledge of all students, subject matter, and curriculum outcomes” (p. 24). FPA Criterion 3, Instructional Design and Implementation, the candidate “understands and uses a variety of appropriate instructional strategies, including those that represent a wide range of technological tools, to develop various kinds of students’ learning including critical thinking, problem solving, reading, and subject matter knowledge. The teacher also uses knowledge of parents, community and agencies to support all students’ learning and well-being when planning and implementing instruction” (p. 26). FPA Criterion 4, Demonstration of Integration Skills, the candidate “demonstrates the ability to integrate across and within content fields to enrich the curriculum, develop thinking skills, and facilitate all students’ abilities to understand relationships between subject areas” (p. 29). This encourages candidates to teach thinking skills and avoid using simple knowledge across the fields unless appropriate. FPA Criterion 5, Analysis of Classroom Learning Environment, the candidate “uses an understanding of individual and group motivation and behavior, including effective verbal and non-verbal communication techniques, to create a positive learning environment that fosters active inquiry, supportive interaction and self-motivation in the classroom” (p. 30). The candidate provides information about his or her awareness of group motivation and behavior to examine his or her ability to function as a professional in a classroom setting. FPA Criterion 6, Analysis of Assessment Procedures, the candidate “understands and uses formal and informal assessment strategies to evaluate and ensure the continual intellectual, social and other aspects of personal development of all learners” (p. 32). This criterion promotes analysis and synthesis of all activities and provides accurate data on students’ learning. FPA Criterion 7, Reflection and Self-Evaluation, the candidate “is a reflective practitioner who continually evaluates the effects of his or her choices and actions on others (students, parents, and other professionals in the learning community), actively seeks out opportunities to grow professionally and participates in the school improvement process (Kansas Quality Performance Accreditation)” (p. 35).

Evolution of the Fort Hays State University Performance Assessment (FPA) Process
Beginning in the fall of 2003 and spring of 2004, faculty members began receiving formal training on scoring the Fort Hays State University Performance Assessment (FPA). In the fall of 2004, full time faculty members in the Department of Teacher Education continued to receive formal training on instructing and scoring the FPA. Training on developing inter-rater reliability also began. These processes were completed during December 2004.

September 1, 2004, the Kansas State Department of Education (KSDE) began issuing graduates a conditional license. During that fall semester, candidates began taking an increased interest in writing the FPA because the FPA now had to be passed to receive professional licensure. Faculty members’ improved the scaffolding process, instruction on writing to the criteria, and inter-rater reliability.

In the fall of 2005, the cumulative KPA score changed from a total of 100 points to a total of 113 points. It was at this time also that the State Department of Education notified State institutions that they would be responsible for training candidates for the KPA. During the summer of 2006, KSDE established a cut score of 88 out of 113 total points on the KPA, which was implemented at the KSDE raters’ meeting in October 2006. In July 2008, the Kansas Board of Education moved the KPA to the state’s 22 institutions as a preservice performance assessment.

Program Design and Data Collection
The faculty systematically revise the curriculum, employing curriculum mapping techniques to ensure that when a candidate exits the program they are ‘”consciously competent” in the following areas: knowledge of classroom environmental factors that affect student learning; identification of all relevant types of student diversity within the population they are teaching; identification and employment of specific accommodation and modifications aligned with students’ individual developmental needs; instructional design development and implementation including integration of subjects that are aligned with district and state standards; development of effective assessments strategies and tools, along with systematic analysis of student learning. Finally, candidate self-reflection and development of professional goals for improvement allow the candidate, not just the faculty, to reflect on professional development goals and identify areas of improvement in course instructional strategies to increase candidate learner gains. This reflective analysis is an ongoing process and continuous change will occur. It is through the above-mentioned systematic research the elementary teacher education faculty and candidate will work toward achieving a level of excellence exceeding candidate performance in sister institutions across the state.

Preliminary quantitative and qualitative data were collected on the Fort Hays State University Performance Assessment (FPA) process. The authors collected quantitative performance assessment data at two levels of the scaffolding process. The first level involved collecting scores on the sample FPA for candidates enrolled in Corrections of Reading Disabilities (TEEL 481) course and corresponding tutorial internship for fall 2005, spring 2006, summer 2006, fall 2006, spring 2007, summer 2007, fall 2007, and spring 2008 (see Table 1). The instructor of each section of the course rated the individual FPAs. The second level involved collecting scores on the final FPA for candidates enrolled in Student Teaching-Elementary School (TEEL 496) course for fall 2005, spring 2006, fall 2006, spring 2007, fall 2007, and spring 2008 (see Table 2). Two raters negotiated one final score after scoring each candidate’s FPA. If there was a discrepancy of five or more points between the two raters’ scores, a third rater read the candidate’s FPA and a final score was determined by all three raters.

Data were collected prior to fall 2005 but are not presented in this article due to changes in state scoring procedures, improvement in inter-rater reliability, and comfort level of the faculty members and candidates while learning to write an FPA. Data presented in this article reflect initial collection of scores for future trend analysis and not for significant differences.

Scores collected from fall 2005 to spring 2008 are represented in Table 3: Mean Comparison Scores by Criterion in TEEL 481, Table 4: Mean Comparison Scores by Criterion in TEEL 496, Table 5: Mean Total Scores for FPAs Completed in TEEL 481, and Table 6: Mean Total Scores for FPAs Completed in TEEL 496. At the preliminary stages of a longitudinal study, the mean total score for each semester in the two courses is above the State’s mandated cut score of 88 for the KPA. Strengths and weaknesses of the scaffolding process are being noted as possible indicators of success and areas of improvement.

Qualitative reflections were also collected and analyzed to showcase candidates’ dispositions as they scaffold through the FPA process. Candidates felt that the writing of a reflective piece for each criterion was a great deal of work. However, candidates generally followed that statement with discussion of the ease with which they were able to write their student teaching final FPA receiving little or no guidance from faculty due to the previous scaffolding that had taken place. Candidates, overall, felt that they would have little or no difficulty in writing reflective evidence for the Kansas Performance Assessment during their first two years of teaching.

At the culmination of each semester, candidates are encouraged to evaluate the course, course assignments, and instructor using the university’s teacher evaluation tool. Candidates wrote additional comments on their evaluations about completing the FPA assignment. Although candidates felt the FPA was time consuming and a lot of work, they appreciated having each criterion of the performance assessment introduced, explained, and practiced in a scaffolded, step-by-step procedure. Their written comments on what they liked about the FPA learning process follow:

• “I liked having the FPA explained in parts rather than as a whole.”
• “All the assignments presented and the extra time preparing us for the KPA through doing the FPA.”
• “I like that the FPA is explained and practiced in our courses.”
• “I liked the practice of writing the FPA in a step-by-step procedure and using a template.”
• “The FPA was a pain, but it was good practice for the actual KPA.”
• “Scaffolding of the FPA from classes prior.”

Candidates also wrote additional comments on the university’s teacher evaluation tool about the most important things learned throughout the coursework. After reading the comments, it was determined that the candidates realize the importance of the FPA and its relationship to the KPA. Candidates indicated that they are prepared to write the KPA for licensure and feel confident that they will achieve the cut score of 88 or better. Their written opinions about the most important things learned included the following:

• “Importance of KPA, reading strategies, and creating lesson plans.”
• “FPA, assessment processes, and creating units.”
• “How to write a KPA, all kinds of strategies, guided reading, and assessments.”
• “KPA-I feel better prepared to do this, assessing students, and FPA. Thankful for the opportunity to practice.”
• “The FPA, teaching methods, planning, and assessment methods.”

Post-graduate candidates have reported an increased level of confidence while completing the state required performance assessment. Preliminary unit scores received from KSDE indicate successful application of the template in developing KPAs. One teacher out of 44 did not make the cut score of 88 out of 113 total points.

Conclusion
Several university faculty members were trained by the state in scoring the performance assessments. Additionally, our education faculty has established reliability and validity of the scoring process. Mentors in the field have been apprised of the work sample process and received training to enable them to assist the candidates. Data are collected from the internships and final candidate teaching documents for use in the university accreditation process.

Research has led to generalizations on components of the work sample process that is valuable in training our candidates as well as for making needed program revisions. Candidates, faculty, and school personnel have noticed an increase in candidates’ abilities to prepare and validate their teaching. Dispositions, on the part of our candidates, regarding the teaching and assessing of candidate learning have become more positive.

The authors will continue recording the above-mentioned data in a longitudinal study of performance on the FPA. Also, data will be collected on criteria two, six, and seven in the three methods courses, and each criterion in the Corrections of Reading Disabilities and Student Teaching/Elementary School courses. Triangulation of data will become complete when the scores for the Fort Hays State University graduates are retrieved from the Kansas Department of Education as those candidates complete the KPA necessary to be considered for professional licensure prior to fall 2008.

References:
Girod, G. R. 2002. Connecting teaching and learning: A handbook for teacher educators on teacher work sample methodology. Washington, DC: American Association of Colleges for Teacher Education.

Hegler, K. L. 2003. Evaluating the use of teacher work samples to describe teacher candidate

competence and PK-12 learning. Paper presented at the Annual Meeting of the American

Association of Colleges for Teacher Education, January 24-27, 2003.

Heiman, G. W. 1998. Understanding research methods and statistics: An integrated

introduction for psychology. New York: Houghton Mifflin Company.

Imig, D. and C. Smith. 2002. Forward. In Girod, G. R. (Ed.), Connecting teaching and

learning: A handbook for teacher educators on teacher work sample methodology. Washington,

DC: American Association of Colleges for Teacher Education.

Kansas State Department of Education. 2007. The Kansas performance assessment. (Revised

August 2007) Topeka, KS: Teacher Education and Licensure.

Salzman, S. A., P. R. Denner, A. W. Bangert, and L. B. Harris. 2001. Connecting teacher

performance to the learning of all candidates: Ethical dimensions of shared responsibility.

Paper presented at the Annual Meeting of the American Association of Colleges for Teacher

Education (53^rd, Dallas, TX, March, 2001).

Schalock, M. 1998. Accountability, candidate learning, and the preparation and licensure of

teachers: Oregon’s teacher work sample methodology. Journal of Personnel Evaluation in

Education, 12 (3), 269-286.

Figure 1: Course Timeline

Figure 2: Fort Hays State University Performance Assessment (FPA) Scoring Sheet

Figure 3: Internship I, II, and III Rubric for Integrated Thematic Unit

Table 1: FPA Data-Internship IV: Corrections of Reading Disabilities

Table 2: FPA Data-Student Teaching/The Elementary School

Table 3: Mean Comparison Scores by Criterion in TEEL 481

Table 4: Mean Comparison Scores by Criterion in TEEL 496

Table 5: Mean Total Scores for FPAs completed in TEEL 481

Table 6: Mean Total Scores for FPAs completed in TEEL 496