Enhance quality and effectiveness in the academic programs
Objective
Candidate Quality And Effectiveness In The Field
As the leader of the educator preparation unit, the College of Education will assure candidate preparedness for teaching through benchmark assessments at entry, entry to clinical experience, and at exit. In addition, the college monitors candidate performance and employer satisfaction after graduation. The assessment system for initial candidates includes measures of proficiency which occur at specified transition points. These measures are aligned with institutional, state and professional standards. Unit-level learning objectives for candidates completing the undergraduate educator preparation programs are recorded by the College of Education because responsibility for learning outcomes resides in multiple departments within the college.
KPI
Benchmark: Candidate Performance On Teacher Work Sample
Upon completion of the initial certification programs, candidates are required to submit a Teacher Work Sample which provides evidence of their ability to consider the contextual factors of the classroom, develop goals for instruction, develop an assessment plan appropriate to those goals, design and deliver effective instruction, evaluate student learning and reflect upon the teaching and learning related to the unit of instruction. Overall and component scores are reviewed and anyalyzed annually by program faculty to examine both the assessment processes and trends in performance that might prompt program changes. The Teacher Work Sample assessment is scored using a blind scoring protocol and assigned a score of 3, 2 or 1. A reduced number of target level scores in 11-12 may have been related to revisions in the scoring protocol, which were initiated to eliminate bias. The expected performance indicator was not met and will remain in place. As a key performance indicator, the percent of candidates scoring 3 should meet or exceed 60%.
Result
Candidate Performance On Teacher Work Sample
The Key Performance Indicator for candidate scoring on Teacher Work Sample specified that the percent of candidates scoring at 3, indicating performance at the target level, should meet or exceed 60%. For the 11-12 academic year, 59% of candidates scored at the target level of 3; with 1.6% scoring at the "unsatisfactory" level of 1. In 2012-2013, with an N of 580 candidates for the year, 58% achieved a target level score of 3, with 47% of candidates graduating in the Fall Term achieving a 3, and 65% of Spring candidates achieving a 3. Thus, the spring graduating cohort met the goal of 60% or more achieving a three. The data also indicate that 3.3 % of candidates in 12-13 scored in the unsatisfactory range. In 11-12, those scoring a 1 represented only 1.6% of the cohort. A score of one has serious implications for candidates because the capstone TWS must be passed in order to receive credit in the student teaching semester. From a broader perspective, since the TWS is the assessment of the candidate's ability to analyze and impact EC-12 student learning, the number of failures may have implications related to program improvement or analysis of the preparation process. A new scoring protoocol used in spring only, was implemented to eliminate bias in scoring. Further examination of the data is needed to determine whether this change was related to increased failing scores.
KPI
Benchmark: Novice Level Dispositions And Diversity Proficiencies
Ten Standards for Dispositions and Diversity Proficiencies are assessed in both initial and advanced educator preparation programs. At this early transition point to Level 2 Field Experiences, all candidates for initial certification must provide evidence of designated Novice Level Dispositions and Diversity Proficiencies (DDPs). Evidence of lower performance was noted as faculty reviewed 2011-2012 data during Fall 2012 Data Day activities, with relatively lower scoring in two areas evident across multiple assessments. Those areas were 1) effectiveness with English Language Learners and 2) technology integration. As a result, one DDP was added to those assessed at the Novice Level so that candidates would self-assess and also receive formative feedback at an earlier point in their program. At this stage, candidates complete one reflection to provide evidence of their progress on each of the three Novice Level DDPs. Instructors measure progress based upon observation and the reflection. A rubric developed by a faculty group is used for this evaluation. This updated assessment and new rubric will be implemented in Spring 2013.
Result
NOVICE Dispositions And Diversity Proficiencies
In the elementary education program, novice level dispositions were revised in Fall 2012 to include 3 standards with a range of performance scored from 1 to 3. Of the 145 scored assessments, 83% were scored at a level of 7 or above. In the Spring term 2013, candidates were assessed on 4 standards, again using a score of 1 to 3. Most notably in Spring 2013, faculty participation in rating candidates' DDPs increased to 97.8%, yielding more complete data related to candidate proficiencies at Transition Point 1Of the 138 Novice Dispositions assignments submitted by candidates, 80% were scored at a level of 9 or above. Because this assessment is primarily used to identify students with issues of concern, it is the outliers that are of critical interest. In 2012-2013, three candidates were rated at an unacceptable level.
In the secondary program 81 scored assessments of DDPs were rated on 4 standards, with 98% achieving a score of 8 or above. Faculty participation in scoring was 98.8%. Only one secondary certification candidate scored at an unacceptable level on the initial assessment of Dispositions and Diversity Proficiencies.
KPI
Benchmark: Emerging Level Dispositions And Diversity Proficiencies
Demonstration of Emerging level Dispositions and Diversity Proficiencies(DDP) is required for progression toward certification, one step of which is entry to CONTENT METHODS. Performance at this level requires that the candidate complete both a lesson plan that includes applicable DDPs and a reflection for five of ten DDPs to provide evidence of their progress. The instructor evaluates candidate progress based on the lesson plan, the reflection and observations, using a rubric developed by a group of expert faculty. The new instrument and rubric were implemented in Fall 2012.
Result
Emerging Dispositions And Diversity Proficiencies 12_13
Candidates preparing for roles in secondary schools were assessed on 7 standards in Fall 2012, using a scoring range of 1 to 3. Of the 128 submissions, 85 % were scored at a level of 19 or above. In the Spring term, candidates were scored on 8 standards, with 73% achieving a score of 19 or above. Considering the combined results for the academic year, only 2 students (2%) demonstrated a level of Dispositions and Diversity Proficiencies that was deemed unsatisfactory by the faculty.
For students in the elementary program, 114 of 162 submissions by candidates were assessed on 5 standards in Fall 2012, using a range of 1 to 3, with 83 % achieving a score of 10 or greater. An extreme lack of faculty participation was noted with only 70% of submissions scored by faculty. In Spring 2013, candidates were scored on 6 standards, again using a 1 to 3 scale. Of the 145 scored submission, 100% were scored at a level of 12 or above, indicating that all submissions demonstrated an acceptable or target level of performance. Faculty participation in scoring was 85.8 %
KPI
Benchmark: -Dispositions In Student Teaching
At the conclusion of the student teaching semester, candidates are assessed by the University Supervisor and the school-based mentor teacher on several areas of performance. The summative measure of the SHSU Dispositioins and Diversity Proficiencies, in 2011-2012 demonstrated a range of ratings on the ten DDPs with mean scores ranging from 2.6 to 2.83 on a 3.0 scale.
Action
DDP Assessment In Undergraduate Programs
The Dispositions and Diversity Proficiency committee within the college continues to refine the assessment process and to work with program faculty to assure that candidates experience formative experiences and feedback within their coursework and field experiences. Graduate level assessment of candidate DDPs will be reviewed in the coming year to assure that faculty and students are provided with meaningful information.
Action
Teacher Work Sample Performance
A standing committee within the college is charged with monitoring and oversight of TWS methodology and assessment. This committee continues to review challenges and issues related to maximizing candidate abiltiies to impact EC-12 student learning. Efforts to provide professional development for faculty and university supervisors, to include mentor teachers in the assessment process and to support candidates in addressing the TWS components are ongoing. In the coming year, improved practice to assure reliability and validity of this key assessment will be addressed.
Goal
Operational Excellence
Operational excellence is examined by determining the extent to which operations, programs and services within the college are managed in ways that meet candidate needs. In addition, candidates are quesioned about their perceptions of the efficacy and importance of their learning experiences as related to key elements of the Conceptual Framework-i.e. the extent to which 1)Techonology, 2) Assessment, 3) Communication,4) Knowledge, and 5) Experiences with Diverse Populations were a part of their learning. Annual operations surveys are administered to all graduates in the last class in the program.
Objective
Productivity Of The College
Increase semester credit hour production in undergraduate, graduate and doctoral programs.
KPI
SCH Production
Semester Credit Hour production for undergraduate, graduate, and doctoral programs provide a measure for tracking growth in the various departments at each level in the College of Education. This is a critical measure that provides information to our analysis of strengths, weaknesses, opportunities and threats. Since programs are now offered at two distant locations, tracking of SCH production for both initail and advanced programs, by location, should be initiated in 2012-2013.
Result
Semester Credit Hour Production
Semester Credit Hour production compared to 2011-2012 reflected significant gains only within the doctoral level programs. At the undergraduate level, SCH gre by .8%. At the master's level, there was a .2% loss of SCH produced in summer, fall and spring. Doctoral programs reflected an increase of 12.3%.
Action
Semester Credit Hour Production
One challenge for both the graduate and undergraduate programs within the college continues as an important goal. Since many graduates will be employed within the highly diverse school districts in the greater Houston area, recruiting a diverse candidate pool is our constant focus. The college will continue outreach efforts to community college and school district partners, through the reassignment of one employee who will dedicate most days to off-campus recruiting and transfer advising. Faculty and staff will continue to participate in NW Houston Chamber of Commerce events, recruiting events at both the Woodlands Center and the Northwest Houston Campus. A mandated increase in admission requirements for all Texas Educator Preparation Programs may result in reduced applications as the shift from 2.5 to 2.75 GPA at entry commences in the coming year.
Objective
Candidate Satisfaction With College Operations, Services And Learning Experiences
Candidate ratings of the quality of services and experiences offered by the College of Education provide several measures of efficiency and effectiveness. The Conceptual Framework of the unit is the organizing framework for programs, services, teaching and evaluation. Thus, an important measure of the efficacy of programs is evaluation of the extent to which learning experiences of candidates were connected to key elements of the Conceptual Framework. In addition, candidates rate the assessment practices within the programs, the quality of interactions that they experience with diverse peers, faculty and P-12 students. They also rate the quality of management and organization of their learning experiences. A survey to assess the quality of operations, services and programs is disseminated to all graduates during the final class in their program and data is collected, analyzed and reviewed in order to promote improved efficiency and effectiveness of operations and services in the college.
KPI
Quality Of Learning Experiences Related To The Conceptual Framework
Candidates are surveyed upon completion of their program as to the quality of learning experiences connected to the conceptual framework. Questions in this section relate to creating authentic technological environments, communicating effectively, use of assessment and feedback to improve P-12 learning, use of learner profiles for improving instruction, all key elements of the unit's Conceptual Framework. Target level performance requires 80% of respondents will rate the connection of the program to the elements of the Conceptual Framework as high or very high.
Result
Quality Of Learning Experiences Related To The Conceptual Framework
In 2012-13, 85% of candidates seeking an initial certification reported that their learning experiences relating to the Conceptual Framework were of high or very high quality. Of candidates in the advanced programs, 86 % reported high or very high quality learning experiences. This result can be interpreted as verification that there was clear articulation and alignment of the Conceptual Framework within the program's learning activites.
KPI
Candidate Rating Of Quality Of Assessment Of Performance
Candidates rating of the quality of assessment by professors and supervisors within the unit. Questions in this section related to accuracy and fairness of assessments, how assessment practices were modeled and used, and assessment of field experiences. No fewer than 80% of respondents should rate the quality of performance assessment as high or very high.
Result
Quality Of Assessment
Upon completion of their programs in 2012-2013, 85% of candidates in the initial certification programs reported that the quality of assessment of their performance was of high or very high quality. In the advanced certification programs, 86% reported high or very high quality. It is important to note that of the advanced candidates, many of whom are classroom teachers engaged in frequent assessment of student learning outcomes, 2 % indiated that assessment was characterized by low or very low quality. This lower end result represents a 5% improvement over data from the previous year.
KPI
Candidate Rating Of Quality Of Interactions With Diverse Peers, Faculty And P-12 Students
Candidates report their perception as to the quality of their interactions with faculty, peers, and P-12 students of other races, languages and ethnic traditions. Questions in this section relate to interactions with diverse faculty, peers, school based faculty and P-12 students. Based on previous results, the target level performance on this indicator is 85% % of of respondents will rate the quality of interactions with diverse persons as high or very high.
Result
Candidate Rating Of Experiences With Diverse Faculty, Peers And EC-12 Students
Undergraduate and post-baccalaureate candidates working toward an initial teaching certificate rate the quality of their interactions with diverse peers, faculty and EC-12 students on a survey submitted as they complete their programs. In 2012-13, 86% of initial completers indicated that they experienced high or very high quality interactions during their program and field experiences. Of the graduate students seeking advanced certificates, 87% reported similar satisfaction.
Action
Candidate Satisfaction With College Operations, Services And Learning Experiences
Annual review of graduating candidates' responses is used by the college in several ways. Information related to services is analyzed and communicated to each staff member who provides services to educator preparation candidates in initial and advanced programs. These results are discussed by the supervisor and each employee, considered as a vital part of the employee's annual performance review and goals for improvement are guided by this critical input from the students who are served within the college. In addition, faculty are provided feedback from the survey to guide their assessment, their planning and their teaching. In planning field experiences, candidate satisfaction with their interactions with diverse faculty, peers and EC-12 students is considered.