STANDARD 2:  ASSESSMENT SYSTEM AND UNIT EVALUATION

The education unit is engaged in a systematic assessment system that is linked to the vision, mission and purposes of the education unit and Dalton State College.  The unit’s assessment system is guided by the Conceptual Framework and is responsive to national standards (Interstate New Teacher Assessment and Support Consortium), state standards (Professional Standards Commission), as well as the unit’s professional outcomes.  The assessment system is designed to collect data on applicant qualifications, the candidates’ performance, and unit operations to evaluate and improve the unit and its programs.  The assessment system is reviewed annually and results are used to improve program quality, monitor unit operation, and assess the effectiveness of program candidates.  Decisions regarding candidates’ progress from one transition point to the next are based on the data collected in the course requirements and in candidates’ performance in the field experience and clinical practice.  The assessment program is threefold:  to collect, summarize, and analyze data on the applicant qualifications and candidate performance; to collect, summarize, and analyze data regarding unit operations; and to collect, summarize, and analyze data to evaluate and improve the unit’s programs.

2a.1. How does the unit ensure that the assessment system collects information on candidate proficiencies outlined in the unit’s conceptual framework, state standards, and professional standards? 
The conceptual framework identifies the use of the following sources of proficiencies for the unit assessment system: INTASC principles, diversity principles, ESOL Endorsement proficiencies, technology proficiencies, unit professional outcomes, Georgia state standards and professional dispositions for candidates seeking initial certification and ESOL endorsement.  Candidates in secondary education programs seeking initial certification will follow the same procedures for assessing candidate proficiencies.

 INTASC principles are assessed through candidates’ critical assignments housed in their early childhood e-portfolio ( Kassie Cudd and Jennifer Findley ).  Additionally, INTASC principles are assessed through assignments and evaluations that are linked to course objectives which are aligned with INTASC principles.

Diversity proficiencies are assessed through specific items on the Teacher Candidate Observation Instrument and the Assessment of Professional Behaviors and Dispositions.  Diversity principles adopted by the unit’s faculty are listed below.

• Teacher candidate displays the ability to work with diverse individuals.  (APBD #4) 
• Teacher candidate demonstrates fair treatment for all students (i.e. gender, ethnicity, religion, learning abilities, socioeconomic status, and English language learners).  (APBD #14)
• Teacher candidate demonstrates the belief that all students can learn.  (APBD #18)
• Teacher candidate is sensitive, alert, and responsive to the specific intellectual, social, physical, and personal developmental needs of all students.  (TCOI II-C)
• Teacher candidate exhibits consistent and appropriate management of time, space, and learning resources for diverse students’ learning; active/equitable engagement of students.  (TCOI III-B )
• Teacher candidate uses a variety of appropriate materials and resources to enhance instruction for diverse learners.  (TCOI V-F)

Faculty members teaching ESOL courses assess the candidates’ ESOL proficiencies through course assignments, ESOL e-portfolios, and critical assignments.  ESOL proficiency data are collected in LiveText and housed in LiveText ESOL portfolio reports and in candidates’ ESOL e-portfolio. The ESOL Endorsement standards/proficiencies are: 

• The program shall prepare candidates who know about the nature of language; Language varieties; and the phonology, morphology, syntax, semantics, and discourse of the English language.
• The program shall prepare candidates who are able to listen, speak, read, and write in standard English on technical, abstract, and non-technical or general topics.
• The program shall prepare candidates who understand the principles of first and second language acquisition across age levels.
• The program shall prepare candidates who understand the effects of cognitive, affective, and socio-cultural variables on language learning.
• The program shall prepare candidates who can apply language assessment techniques and instruments in the evaluation of students.
• The program shall prepare candidates who can plan ESOL curriculum and implement a variety of instructional methods and assessment strategies for teaching English as a second language.
• The program shall prepare candidates who understand socio-cultural systems and characteristic features of linguistic/minority cultures.
• The program shall prepare candidates who are familiar with current educational trends, issues, policies, and practices, and their relationships to program planning, instruction, and assessment of ESOL students.
• The program shall prepare candidates who can use technology, particularly multimedia and computer and software, and can evaluate software for ESOL instruction.

The 5 technology proficiencies (NETS) are assessed by the Educational Technology Center (ETC) staff throughout the 50 hours of technology training in assignments in the Teacher Candidate INTECH folders.  Technology proficiency data are housed electronically in the Educational Technology Center (ETC).   Candidate’s INTECH folder shows evidence of mastery of NETS proficiencies. Technology proficiencies are:

• Facilitate and Inspire Student Learning and Creativity: Teachers use their knowledge of subject matter, teaching and learning, and technology to facilitate experiences that advance student learning, creativity, and innovation in both face-to-face and virtual environments.
• Design and Develop Digital-Age Learning Experiences and Assessments: Teachers design, develop, and evaluate authentic learning experiences and assessments incorporating contemporary tools and resources to maximize content learning in context and to develop the knowledge, skills, and attitudes identified in the NETS•S.
• Model Digital-Age Work and Learning: Teachers exhibit knowledge, skills, and work processes representative of an innovative professional in a global and digital society.
• Promote and Model Digital Citizenship and Responsibility: Teachers understand local and global societal issues and responsibilities in an evolving digital culture and exhibit legal and ethical behavior in their professional practices.
• Engage in Professional Growth and Leadership: Teachers continuously improve their professional practice, model lifelong learning, and exhibit leadership in their school and professional community by promoting and demonstrating the effective use of digital tools and resources.

The unit’s professional outcomes for teacher candidates as stated in the conceptual framework are competent, caring, collaborative, and reflective.  These outcomes are aligned with the Teacher Candidate Observation Instrument (TCOI) and the Assessment of Professional Behaviors and Dispositions (APBD). TCOI data are collected and stored in LiveText.  APBD data are collected in LiveText and exported to Excel spreadsheet for reporting purposes.  Additionally, professional outcomes are aligned with course objectives and critical assignments.  Course objectives are assessed through course assignments as stated on course syllabi, and critical assignments are assessed with rubrics. Critical assignments are housed in each professor’s LiveText account and are included in teacher candidates’ e-portfolios.

Early Childhood Program Critical Assignment Chart

Georgia state standards are aligned with unit course objectives and assessed through critical assignments and other course activities and assessments. 

School of Education
Early Childhood Education Program including ESOL Endorsement Courses
Alignment of Courses with C.F. Professional Outcomes,
 INTASC Principles and State Standards

Professional dispositions are assessed through the Assessment of Professional Behaviors and Dispositions.  Midterm and final assessments are conducted each semester by site supervisors for all teacher candidates in Blocks I - IV.  Data are collected in LiveText and stored in Excel reports.

2a.2.  What are the key assessments used by the unit and its programs to monitor and make decisions about candidate performance at transition points such as those listed in Table 6?  Please complete Table 6 or upload your own table at Prompt 2a.6 below.

Table 6
Key Assessments at Transition Points – Early Childhood and ESOL

Key Assessments at Transition Points: Secondary Programs
Biology, Math, Chemistry, History, and English

2a.3. How is the unit assessment system evaluated?  Who is involved and how?
The unit’s assessment system was developed in spring 2006 by the unit’s division chair, faculty, assessment coordinator, and representatives from the professional community.  The systematic evaluation and ongoing improvement of the assessment system is conducted through annual review and analysis of data by the assessment committee, Teacher Education Council, and program faculty. 

Members of the Assessment Committee include the dean of the SOE, assessment/NCATE coordinator, representatives from the unit’s faculty, representatives from Liberal Arts and Sciences and Mathematics faculty, and representatives from P-12 teachers and administrators.  The assessment committee meets annually to review assessment data gathered throughout the year and to make recommendations to the unit faculty for ongoing refinement and improvement of the assessment system as well as unit and program quality.  Assessment instruments, procedures for collecting data, and actual data captured from the instruments are shared with committee members. 

Members of the Teacher Education Council include the School of Education dean, assessment/NCATE coordinator, and different representatives from the same stakeholder groups as the Assessment Committee, thus allowing for input from a larger number of stakeholders.   The Teacher Education Council meets once each semester to share program updates, to discuss assessment system and data, and to gather suggestions for changes/improvements.

Through the work of these committees, faculty and representatives from the professional community consider the assessment instruments to determine if the type of data collected sufficiently informs the unit about the students’ progress and program effectiveness.  A recommendation from the Teacher Education Council resulted in a change of rating procedures for the Assessment of Professional Behaviors and Dispositions Instrument.  Another outcome from one Assessment Committee meeting is a group research project to determine the most effective influences for fourth-year teacher retention (such as program of study, mentor teachers, and school climate) Consequently, the unit will collect additional data that will facilitate program improvement.

Program faculty also meet annually to review assessment data and make recommendations for viable changes to the assessment system.  Faculty members consider Assessment Committee and Teacher Education Council suggestions for change and make additional recommendations as needed.  After two years of collecting candidate and program data as prescribed by the current assessment system, the SOE faculty is currently exploring suggestions for changes in frequency and manner of data collection to ensure the unit collects accurate and meaningful data for program improvement.  Other changes that have been made as a result of program faculty recommendations include:

1. formalized Professional Growth Plans (PGP),
2. development of advisement survey, and
3. the adoption of  two new key assessments (Teacher Candidate Observation Instrument and the Assessment of Professional Behaviors and Dispositions Instrument) in fall 2007 to replace previous candidate assessment instruments that faculty felt were less effective and no longer met the needs of the program.

2a.4. How does the unit ensure that its assessment procedures are fair, accurate, consistent, and free of bias?
The unit continuously strives to provide accurate, fair and bias-free, and consistent assessment procedures.  This effort is guided by the unit and practiced in all teacher preparation programs and ESOL endorsement.  Through collaborative development of rubrics, training of assessors and teacher candidates, inter-rater reliability checks, alignment with the conceptual framework and state standards, clear communication, and feedback from professional colleagues, the unit assures accuracy, fairness, and freedom from bias.

 The unit measures accuracy in assessments to the degree that key assessments, course assessments, and critical assignments assess what has been taught and measure what they are intended to measure.  An alignment matrix for all courses in the Early Childhood Education Program and for the Early Childhood Education Program with ESOL Endorsement was developed to show how course objectives are aligned with the professional outcomes of the Conceptual Framework as well as state and national standards. If applicable, course objectives are also aligned with ESOL Endorsement standards. Key assessments (TCOI and APBD) are also aligned with professional outcomes from the unit’s Conceptual Framework and national standards.  The Teacher Candidate Observation Instrument was designed with input from state universities as well as state and national accrediting agencies.  It was also piloted and tested for validity.  The Assessment of Professional Behaviors and Dispositions was reviewed by a panel of early childhood experts for validity purposes.  Thus, the unit can be confident that its assessment instruments are accurate and valid.

In an effort to ensure fairness, teacher candidates are given training on assessment instruments at the beginning of their professional program so they understand what is expected of them prior to their evaluations.  Teacher candidates receive an orientation covering the Teacher Candidate Observation Instrument and the Assessment of Professional Behaviors and Dispositions at the beginning of their professional program. Providing the orientation at the beginning of the professional program ensures that teacher candidates have the necessary knowledge and instruction for these key assessments far in advance of the application of assessments.  In order to eliminate bias, rubrics have been developed for all critical assignments and any key assessments that require faculty or supervisor evaluation of teacher candidates’ performance.  Faculty members collaborate in the development of the rubrics.  Rubrics are reviewed each semester to eliminate vagueness or weaknesses in scoring instructions.  Students are given rubrics in advance of submission of assignments.  Currently, professors discuss the rubric with teacher candidates to make sure that there is no confusion regarding criteria for evaluation.  Collaboration and ongoing refinement of assessment instruments allow all programs to apply assessment instruments that are free of bias. 

All faculty members teaching the same course use the same objectives and same critical assignment. Critical assignments and their assessments are collaboratively developed by all professors teaching that course.  This helps ensure that all teacher candidates are able to understand and apply course objectives which are linked to the Conceptual Framework’s professional outcomes as well as state and national standards, further facilitating the consistency of assessment procedures.

2a.5.  What assessments and evaluation measures are used to manage and improve the operations and programs of the unit? 

Assessment data for the management and improvement of operation and programs of the unit are collected at regular intervals as part of the ongoing assessment process.  These data are derived from multiple assessments that are scheduled at regular points throughout the academic year.  Additionally, the data come from both internal and external sources.  The following table shows the unit’s assessments for the operations and programs quality.

Assessment of Unit’s Operation

Assessment of Program Quality

2b.1. What are the processes and timelines used by the unit to collect, compile, aggregate, summarize, and analyze data on candidate performance, unit operations, and program quality?
In fall 2006, the School of Education developed and implemented a data management system that operates on an annual cycle of data collection, summarization, and analysis of data.  Data are collected and housed in either Access, Excel, or LiveText database.  The School of Education assessment system flowcharts show which data are collected in Access, Excel, and LiveText. 

The Data Collection and Timeline Chart below defines the data to be collected, the frequency of collection, who is responsible for collecting the data, and how data are used in the assessment system.  Teacher candidate performance data and survey data for program and unit evaluation are summarized in graph format and analyzed by the assessment coordinator.   Demographic data for teacher candidates, site supervising teachers, field placements, and school sites are also summarized and analyzed in report format by the assessment coordinator. 

Data Collection Chart

2b.3. How does the unit maintain records of formal candidate complaints and their resolutions?
Any teacher candidate wishing to file a complaint must follow the unit’s process.  The teacher candidate completes the formal Complaint Document which defines the steps in the process.  This document is submitted to the Dean of Education who considers the complaint and acts as mediator to resolve the issue.  If the complaint is resolved at this point, the complaint resolution is maintained in this office.  If the issue is not resolved at this level, the candidate has the opportunity to appeal to the Vice President for Academic Affairs.  This decision is also maintained in the office of the Dean of Education.  Complaint records and their resolutions are maintained in the office of the Dean of Education.

Teacher candidates may also appeal program admission decisions made by the School of Education.  An appeal form must be completed and filed by the stated due date.  An Appeal’s Committee composed of faculty representation from the School of Education, the School of Humanities, and the School of Math and Science review the appeals and make a decision for the individual appeals. This decision is then shared with the student. If the student chooses, the appeal can then be taken to the Vice President of Academic Affairs. The decision of this complaint is maintained in the office of the Dean of the School of Education.

2c.1. In what ways does the unit regularly and systematically use data to evaluate the efficacy of and initiate changes to its courses, programs, and clinical experiences?
The unit regularly and systematically uses data to evaluate the efficacy of its courses, programs, and clinical experiences. Summarized data for teacher candidate assessment, program assessment, and unit assessment are reviewed.  For example, data are considered to determine if students are progressing systematically; if there are strengths and weaknesses across all students that point to the need for program improvement; if feedback from supervisors or employers point to essential questions about preparation; or if students believe there are needs for change in courses, programs, field experiences.

The unit has established procedures for evaluating each course in the fall and spring semesters.  Faculty members receive their course evaluations at the beginning of the following semester. Professors review these evaluations to make improvements for the course in subsequent semesters.  Critical assignments are also used by faculty members to determine improvements that need to be made in course instruction.

The unit meets annually to review summarized program and unit assessment data. 
Included in these reviews are summarized teacher candidate assessment data (TCOI and APBD), summarized program data, and summarized unit data.  In addition to course evaluations, program and unit data reviewed include site teacher and DSC supervisor evaluations; completer, graduate, and employer surveys; assessment of advisement; and GACE scores. Weaknesses and strengths of the program are examined to determine needed changes.  An example of these changes is frontloading a course currently taught during the clinical practice semester until it can be moved to an earlier semester in the program during the 2010-2011 academic year.  This change was based on course grades and comments on the completer survey.
 
The Teacher Education Council meets once per semester to receive updates on the work and progress of the programs and unit. Members of the Teacher Education Council include selected faculty from the School of Education, School of Liberal Arts, and School of Sciences and Mathematics, as well as representative public school teachers and administrators. Members review program and teacher candidate data. One recommendation from the TEC resulted in a change of procedure for administering the Assessment of Professional Behaviors and Dispositions in order to eliminate possible bias.

The Assessment Committee meets annually to review the work and progress of the program and unit.  Although members of the Assessment Committee represent the same groups as the Teacher Education Council, the individual members from the public school partners and the Schools of Liberal Arts and Sciences and Mathematics are different than those members serving on the TEC.  The role of the Assessment Committee is to review the annual assessment data from teacher candidates, programs, and unit to make suggestions for change and improvement.

In addition to the council’s and committee’s review of program and unit data, the unit has formed focus groups consisting of public school site teacher representatives and school of education faculty representatives. The members of focus groups examine clinical practice assessment instruments and procedures for clinical practice. No recommendations for changes were made regarding the assessment instruments, but did strongly suggest that the unit leave in place the pre-planning opportunity for teacher candidates as part of the clinical practice experience.

2c.2 What data-driven changes have occurred over the past three years?
The unit has made several changes during the past two years as a result of feedback from review of data.  Changes include

• revising the conceptual framework to better support and reflect the goals/outcomes, mission, and vision of the unit;
• planning to schedule a course that conflicts with clinical practice to an earlier semester in the program;
• formalizing the Professional Growth Plan (PGP) for earlier intervention with specific remediation strategies to meet individual candidate’s needs;
• adding a course (Curriculum and Assessment) to meet PSC content standards;
• implementing critical assignment in each EDUC course for documentation that candidates meet INTASC principles and consistency across program courses;
• changing the Completer Survey to allow the unit to disaggregate ESOL endorsement data;
• adding a math unit assignment in the math methods course;
• piloting a new supplemental writing text for the language arts course in order to strengthen candidates’ knowledge of how to teach writing in the elementary classroom;
• changing assessment procedures for critical assignments; and 
• perhaps the most notable change, adopting two new key assessment instruments (Teacher Candidate Observation Instrument and Assessment of Professional Behaviors and Dispositions) to replace two original key assessments. 

2c.3. What access do faculty members have to candidate assessment data and/or data systems?
Summarized candidate assessment data are posted on the School of Education drive and/or shared with faculty annually.  Summarized assessment data include the Teacher Candidate Observation Instrument, Assessment of Professional Behaviors and Dispositions, GACE scores, completer surveys, graduate surveys, DSC supervisor evaluations, site-supervising evaluations, advisement evaluations, and employer surveys. These data are available to any faculty member at any time. Critical assignment data and reports are housed in faculty members’ individual LiveText accounts.  Data are also compiled in an assessment notebook and shared with faculty at annual meetings.

2c.4. How are assessment data shared with candidates, faculty, and other stakeholders to help them reflect on and improve their performance and programs?
The unit shares assessment data with candidates, faculty, and other stakeholders for reflection and improvement of performance and programs.  Candidates receive their individual assessment data while faculty and other stakeholders receive aggregate candidate and  unit program assessment data.

Individual candidate data include the Teacher Candidate Observation Instrument (TCOI), Assessment of Professional Behaviors and Dispositions (APBD), critical assignments, GACE Basic Skills, and GACE Content. Dalton State College supervisors evaluate candidates using the TCOI, and site supervising teachers evaluate candidates using the APBD. Candidates receive TCOI and APBD evaluation data immediately upon completion of the evaluation by their DSC supervisors or site supervising teachers. Candidates are expected to reflect specifically upon their classroom interaction and performance as well as their planning and delivery of lessons as stated on the TCOI and APBD assessment instruments. The purpose of candidates’ reflections is to determine areas for improvement in classroom performance, lesson planning, and lesson delivery. GACE Basic Skills and GACE Content exams are scored by the testing company that sends scores directly to the candidates as well as to the college as soon as they are available. Additionally, candidates receive assessment scores on critical assignments in LiveText as soon as the assignment is evaluated by the professor.

Assessment data are shared with program faculty annually so that faculty can review for areas of needed improvement. Assessment data reviewed by faculty include aggregate teacher candidate assessment data (TCOI and APBD); state mandated examinations (GACE basic and GACE Content); and employer, completer, and graduate survey data. Faculty members also receive individual summarized course evaluation reports for each course taught in fall and spring semesters, which include candidates’ suggestions for course improvement. These reports are given to faculty members at the beginning of the following semester. Faculty members analyze their course evaluations along with critical assignment reports to find ways to improve and strengthen the course in future semesters. Faculty plans for course improvement are included as part of their teaching goals for the following year.

Annual aggregate assessment data (TCOI, APBD, survey data, and unit/program data) as well as program information are shared with representative stakeholders serving on the Teacher Education Council and Assessment Committee.  Currently, the dean of the School of Education has asked for time to share candidate and program assessment data and information annually with area school superintendents through the Regional Educational Service Association (RESA).   Focus groups are also hosted periodically by the unit to share assessment data with representatives of site supervising teachers and area school administrators as well as to seek their input for candidate, program, and unit improvement.