Back to Top
Of PageSTANDARD II: ASSESSMENT SYSTEM AND UNIT EVALUATIONS
| The unit has an assessment system that collects and analyzes data on the applicant qualifications, the candidate and graduate performance, and unit operations to evaluate and improve the unit and its programs. |
The COEHS Assessment System evaluates candidates, programs, and unit; the current plan is divided into two areas– initial and advanced preparations. The first area of the Assessment System focuses on all licensure areas within PEP. While slight variations were developed to accommodate specific licensure areas, all initial preparation candidates are subject to a defined standardized set of guidelines. The second area of the System addresses the five advanced preparation programs. For these candidates, each program has defined its assessment guidelines with some variation among programs. However, this area is unified by these elements: Each program 1) has an assessment plan grounded in the COEHS Conceptual Framework; 2) is aligned with the appropriate set of Wisconsin Standards; 3) has considered appropriate SPAs; 4) has met expectations for North Central Association review; 5) satisfies all university-wide policies as set by the Graduate School; 6) has defined entry, transitional, and culminating experiences; and 7) participates in the UW System Program Review process as part of a program evaluation and improvement cycle. The latter element becomes a critical part of the assessment data used at the unit level for evaluation and improvement.
Unit assessment includes response to external expectations from the UW System program reviews, DPI annual assessment reports, and US Department of Education Office of Postsecondary Education Title II regulations. In April 2001 the Unit submitted the first Annual Institutional Questionnaire on Teacher Preparation: Academic Year 1999-2000 to DPI. The Unit reported 1013 candidates enrolled during 1999-2000 and 336 candidates supervised in student teaching by a total of 11.87 full-time employees (FTE). The candidate/faculty ratio was 28.3 to 1. Wisconsin is in the process of identifying and developing content tests for licensure areas which will not be in place until 2004. The Unit submitted its Assessment System for review to DPI in July 2001 and satisfied current expectations.
During this transitional period, COEHS is discussing how to improve the tracking of program completers as they move from our programs to employment situations. Currently, surveying candidates, alumni, and employers is a requirement of the UW System Academic Program Review Process. COEHS Initial and Advanced Preparation Programs are on a five-year cycle of data collection, review, and program improvement. Student satisfaction with programs is documented by surveys of alumni and students; placement and graduate school admission rates are cited. DPI requires employer evaluations of candidates. COEHS is intensifying the tracking of candidates through collaboration with the Career Counseling, Graduate School, Institutional Research, and Alumni Offices. We are assessing current efforts for coordination improvement, especially with the campus-wide implementation of the PeopleSoft System, and recommending additional steps to take in the future. Future directions will incorporate new licensure structures such as the initial educators’ requirement of pre-Professional Development Plans, electronic portfolios promoted through the PT3 VIVA grant, and campus-based electronic assessment tools developed in conjunction with CESA 6.
Initial Preparation Program
In accordance with Wisconsin PI 34 regulations and aligned with the COEHS Conceptual
Framework, a dynamic assessment system was developed
to collect and analyze data on applicant qualifications, candidate performance,
and program operations to evaluate and improve PEP. It continues to evolve as
COEHS moves through this transitional period. Working with all constituent groups,
the assessment plan involves the external and internal voices of our professional
community including COEHS faculty, staff, and candidates; COLS faculty
and staff; university administrators; members of the higher education consortium,
and P-12 partners. Completed work is shared with all constituency groups.
The catalyst for all decisions made for the assessment plan is the Conceptual Framework, in alignment with Wisconsin Teacher Standards and professional standards such as INTASC and other learned societies. Certain aspects of the assessment plan directly reflect state standards (e.g., the student teaching evaluation form) or SPAs (e.g., portfolio evidence based on CEC competencies).
The assessment plan includes a comprehensive and integrated set of evaluation measures, operating both to inform our thinking about candidate performance as well as program operations. It is based on multiple assessments made at three critical points in the PEP. At admission into the program, candidates must show eligibility by 1) meeting minimal standards on the PPST; 2) submitting disclosure statement and receipt for criminal background check; 3) completing a minimum of 40 credits; 4) maintaining a minimum 2.75 cumulative GPA; 5) earning a ‘C’ or better in Fundamentals of Speech; 6) filing a clear TB test; and 7) completing Admission I application indicating licensure area. Prior to application for admission, candidates are provided an orientation by the PEP Director and view a prepared video on specific requirements. Each candidate prepares a portfolio of evidence focusing on three critical aspects of the Framework – knowledge of content and learning, understanding of curriculum, and commitment to lifelong learning. Likewise, the evidence directly addresses other aspects of the Framework – knowledge of culture, understanding of curriculum and pedagogy, and ability to become a reflective professional. Portfolio evidence includes: PPST scores, general education GPA, recommendation from a general education instructor, recommendation of a significant adult, reflection on general education experience, personal statement, documentation of completed Human Relations Code requirement, reflection on professional education experience, and grade(s) from gateway courses. Two PEP faculty and staff review each portfolio and interview each candidate.
Candidate performance is also monitored at the next appropriate transition point prior to student teaching or Admission II. All candidates must demonstrate eligibility with appropriate GPA requirements and course work completion; verification of and reflection on Human Relations Code requirements; and successful passage of external content area examinations (in 2004). The central focus of candidate assessment will be on three aspects of the Conceptual Framework – knowledge of learning, being a skillful practitioner, and being a reflective professional. Proposed assessment approved for piloting in Spring 2002 includes this additional portfolio evidence: reflection and evaluation of curriculum unit; management case, log, study, or project; and self-assessment and successful completion of clinical teaching experience. Addressed evidence of other Framework aspects are: knowledge of content and culture, learning, diversity, and pedagogy; and being a lifelong learner. Candidates will be reviewed by PEP faculty and staff involved in the professional core courses through a system of faculty discussion, portfolio review, and candidate interview.
The final aspect of the PEP assessment plan being discussed is for Program Completion, including student teaching and a core block of graduate course work. Proposed evidence included in this portfolio is: self-assessment and successful completion of student teaching; development of working pre-Professional Development Plan; visual documentation of teaching abilities; refined philosophical professional statement; and final reflection on issues related to culture and diversity. The latter two pieces of evidence would be integrated in graduate course work and monitored by those instructors; other evidence would be from student teaching using the seminar infrastructure and monitored by university supervisors.
These changes are better predictors of a candidate’s potential success for admission in, continuation in, and completion of the programs. The Assessment Task Force, and PEP and Human Relations Committees are collecting, aggregating, and analyzing data generated by the assessment system to verify validity and reliability as it relates to candidate success.
Prescriptive directions and explicit rubrics were developed and shared with candidates and PEP faculty and staff in an effort to establish fairness, accuracy, and consistency, and to eliminate bias in performance assessments. Assessment tools were integrated into course work maximizing candidates’ ability to receive support and assistance. Additional help sessions are held for candidates. PEP systematically solicits faculty and staff feedback for refining the process to better insure fairness, accuracy, and consistency. All candidates have access to a well-defined appeals process to challenge decisions within the assessment system when fairness, accuracy, and/or consistency are in question. An impartial committee, not involved in the original decision-making, provides candidates a forum for their grievances.
During this transitional period, COEHS is developing and improving structures to maintain the assessment system to provide regular and comprehensive information on applicant qualifications, candidate proficiencies, graduate competencies, unit operations, and program quality. In place are multiple assessments from internal sources (course-level performance assessments and data generated by program level assessments) and external sources (PPST scores, content area examination scores, and recommendations from outside sources). COEHS continues to collect data from applicants, candidates, recent graduates, faculty, and other members of the professional community. These data are being regularly and systematically compiled, summarized, and analyzed to improve candidate performance, program quality, and unit operations. Without duplicating existing data collection, aggregation, and reporting efforts at the University level, COEHS is studying how best to use information technology to maintain the assessment system internally.
As each stage of the assessment system was developed, PEP asked four critical questions: 1) What data is generated? 2) How can it inform our thinking? 3) How should it be analyzed? and 4) How often should it be analyzed? By asking these questions, PEP began to develop a plan to regularly and systematically use data. For example, at the program level the aggregation of candidate data particularly from clinical experiences and student teaching evaluation tools allows faculty and staff to evaluate the efficacy of the courses and the overall programs. From data analysis faculty and staff initiated changes addressed in the Human Relations Code requirement procedures. In the second case, a task force was formed to explore the need for a discrete course in classroom management. This information is regularly shared with four key internal governance groups – PEP Committee, Human Relations Committee, Assessment Task Force, and Administrative Council, each of which has the charge of recommending changes to improve the program. Collection, aggregation, and analysis of these data feed into an internal university program review process used to analyze and evaluate all programs.
Advanced Preparation Programs
MSE – Counseling: The Program’s Assessment plan,
first developed in June 1995, continues to be reviewed during department curriculum
discussions. Measures used to assess each include: faculty observation, course
requirements, portfolios, self-reports, evaluation of counseling skills, attendance
records at professional meetings, eligibility for professional credentials,
and the optional National Counselor Examination. The entire faculty also makes
a group assessment when each candidate applies for Admission to Candidacy.
MSE – Curriculum and Instruction: This Assessment Plan, established in 1995, continues to be reviewed by the department as part of their assessment discussions. The Plan includes candidate assessment, a five-year program review cycle, and current candidate and alumni surveys. The transitional plan will also include a data collection and tracking system.
MS – Educational Leadership: The MS - Educational Leadership program implemented a comprehensive assessment plan with three critical checkpoints: 1) entrance interview with program coordinator; 2) admission to candidacy (assessment of progress); and 3) conclusion (enrollment in capstone seminar). Assessment occurs through the use of comprehensive examinations and professional portfolios.
MSE – Reading: The Reading Education Program developed a comprehensive assessment plan in 1996 and is continually updated. A plan refinement includes each required course listing its objectives with specific performance based evaluation measures linked to the COEHS Conceptual Framework, IRA standards, and the MSE Reading Program Assessment.
For overall program assessment, yearly graduate and employer surveys are conducted. This formal information, combined with informal feedback continuously solicited from candidates, identify areas or procedures needing refinement or modification. Constant contact is also maintained with professionals in the field. The program review completed on a five-year cycle, considers the compilation of data and allows for very specific program assessment and strategic planning. The program will be sent to the IRA for review in February 2002. For complete Assessment Plan and Program Folio consult Standard II Notebook.
MSE – Special Education: The department developed a comprehensive assessment plan in 1995. Recent efforts align the Plan with the Conceptual Framework, Wisconsin Teacher Standards, and the CEC Eight Common Core Categories. The Graduate Program recently began the first step to establish a database to monitor the progression and current status of all students in their portfolio assessment process. For overall Program Assessment, yearly surveys are conducted with current candidates and alumni. In addition, the department seeks program feedback from the Special Education Advisory Board on a yearly basis.
