EHHD_ED_GR_EDD_AHE_REPORT_YEAR_0
Assessment Plan – Year 0 Report
College: Education, Health, & Human Development Department: Education Submitted by: Sarah Pennington |
Year 0 Assessment Plan Report is due September 15th . |
Indicate all majors, minors, certificates and/or options that are included in this new assessment Plan
Majors/Minors/Certificate | Options |
Doctor of Education - Education (Ed.D.) | Adult & Higher Ed: Higher Education Academics |
Adults & Higher Ed: Higher Education Administration | |
Part 1: Program Learning Outcomes (PLOs)
PLOs should be written as specific, measurable statements describing what students will be able to do upon completion of the program. The assessment
of PLOs provide feedback on the expected knowledge, skills, and attitudes that students
develop as they progress through their program.
List the program learning outcomes:
PLO# | PLO Description |
1. | Demonstrate disciplinary skills and knowledge, applying these to analyze problems. |
2. | Develop meaningful solutions to complex problems of practice from a position of equity and social justice. |
3. | Integrate personal, practical, theoretical, and empirical knowledge with systemic and systematic inquiry. |
4. | Communicate effectively using multiple modes. |
5. | Conduct scholarly and professional activities in an ethical manner. |
Part 2: Development of Assessment Plan
Each plan will require the following information:
Threshold Values: Along with PLOs, plans should include threshold values; minimums against which to
assess student achievement for learning outcomes. Threshold values are defined as
an established criteria for which outcome achievement is defined as met or not met.
Methods of Assessment & Data Source: Assessment plans require evidence to demonstrate student learning at the program
level. This evidence can be in the form of a direct or indirect measure of student
learning. Both direct and indirect assessment data must be associated with the program’s learning outcomes. An assessment rubric will also need to be included that demonstrates how evaluation
of the data was used to assess student achievement.
Timeframe for Collecting and Analyzing Data: Develop a multi-year assessment schedule that will show when all program learning
outcomes will be assessed. As graduate assessment reports are biennial, faculty review
of assessment results may only occur every other year, however, annual faculty meeting
to review these data and discuss student progress may be beneficial.
2a. Curriculum Map
ASSESSMENT PLANNING CHART | ||||||
Program Learning Outcomes | Course Alignments: Include rubric, number and course title |
Identification of Assessment Artifact | ||||
1, 5 | EDU 600 – Doctoral Seminar | Literature Review | ||||
1, 4, 5 | EDU 690 – Doctoral Thesis | Doctoral Dissertation | ||||
1, 2 | EDLD 605 – Higher Education History & Philosophy | Paper 3 (American Indian HE); Final Paper | ||||
3 | EDLD 616 – Organization & Administration of Higher Education | Final Position Paper | ||||
3, 4 | EDLD 628 – College Students | Research proposal & Presentation | ||||
EDU 637 – Institutional Research & Assessment | ||||||
3 | EDU 614 – Program Evaluation | Executive Summary of Final Evaluation Report | ||||
3, 5 | EDU 610 – Qualitative Research | Research Proposal/Project/IRB app | ||||
3, 5 | EDU 607 – Quantitative Educational Research | Research Proposal/Project/IRB app | ||||
3, 4 | Program Benchmark: Comprehensive Exams | Written comprehensive exams with oral defense | ||||
ASSESSMENT SCHEDULE | ||||||
Year to be assessed | ||||||
PLO | Course | 2022-2023 | 2023-2024 | 2024-2025 | 2025-2026 | 2026-2027 |
1 | EDU 600 – Doctoral Seminar | X | X | |||
1 | EDU 690 – Doctoral Thesis | X | X | |||
1 | EDLD 605 – Higher Education History & Philosophy | X | ||||
2 | EDLD 605 – Higher Education History & Philosophy | X | ||||
3 | EDLD 616 – Organization & Administration of Higher Education | X | ||||
3 | EDLD 628 – College Students | X | ||||
3 | EDU 614 – Program Evaluation | X | ||||
3 | EDU 610 – Qualitative Research | X | X | |||
3 | EDU 607 – Quantitative Educational Research | X | ||||
3 | Program Benchmark: Comprehensive Exams | X | ||||
4 | EDU 690 – Doctoral Thesis | X | ||||
4 | EDLD 628 – College Students | X | X | |||
4 | Program Benchmark: Comprehensive Exams | X | ||||
5 | EDU 690 – Doctoral Thesis | X | ||||
5 | EDU 610 – Qualitative Research | X | X | |||
5 | EDU 607 – Quantitative Educational Research | X | ||||
Part 3: Program Assessment
The assessment plan will need to include: 1. how assessment will be conducted; 2. who receives the analyzed assessment data, and 3. how it will be used by program faculty for program improvement(s).
1) How will assessment artifacts be identified?
The adult and higher education program faculty in collaboration with the Director of Accreditation and Department Head. Identification of artifacts will consider how effectively each artifact provides evidence of the appropriate PLO.
The EdD Higher Education Academics specialization is intended for individuals who will teach and/or provide academic leadership and support in the area of college teaching and learning within a diverse range of post-secondary settings. Individuals pursuing this option may hold a Master's degree within higher education or another discipline in which they will teach or provide academic leadership. The EdD in Higher Education Administration specialization is intended for individuals who will contribute to the administrative leadership and support within a range of diverse post-secondary settings. The primary objective of the degree is to produce informed scholarly practitioners for mid- to upper-level management or administrative positions. These individuals typically manage or direct either academic or student affairs operational functions.
EDLD 616: The final paper for EDLD 616 will be assessed for evidence of the program learning outcomes. This paper synthesizes students’ work over the semester on a problem faced in administration, resulting in implications for practice in higher education or student affairs. The paper is between 6-8 pages in length and prompts students to analyze and evaluate research evidence to propose a solution to their problem of focus.
EDU 628: Ethical Considerations, Validity, and Reflection: There are four areas to cover as part of the ethical considerations. One, describe any ethical considerations such as power differences, sensitive populations, anonymity, and how you will address them in the study. Two, explain how you will adhere to ethical standards for practice and any potential concerns. Three, discuss any limitations to the evaluation based on the methodology. Four, after the evaluation you will provide a reflexivity of your experiences such as pressures, bias, or problems associated with the evaluation and how you managed or mitigated them. Problems may include but are not limited to financial or resource constraint, political pressures, responsiveness of participants or leadership, other.
Comprehensive Exams/Dissertation
2) How will they be collected (and by whom)?
Data will be collected by the Graduate Programs Coordinator, course instructors, and committee chairs as appropriate. Data may include course and project-specific grades as well as results of comprehensive exams and dissertation defenses.
b. How sample will be collected?
All final papers from the course were collected, and those completed by students in
the M.Ed. program will be assessed for the learning outcomes. Samples will be collected
through the D2L Course Home Page typically in the Assignment Drop Box: This is where students submit formal assignments. In the Drop Box Forum, students
see the titles for each formal assignment. Students click on the tile of the corresponding
assignment you want to submit and add the document as a file and submit. A census
sample will be collected for the comprehensive exams for both Fall and Spring semesters.
3. Data Analysis
a. Who participated in the process, the nature of the rubric utilized (or other norming
methods), and the threshold outcome desired.
For EDLD 616, the rubric developed to assess the final papers was informed by both the specific course learning objectives and the broader program learning outcomes as stated above. Three outcomes, reflective of the two program learning outcomes being assessed, were assessed for the paper along a 10-point scale:
Criterion | Exemplary (10 points) |
Proficient (8 or 9 points) |
Developing (7 points) |
Unacceptable (6 or fewer) |
Discussion of problem | Work shows evidence of synthesis, summary, and critique of empirical findings and professional experience to articulate understanding of problem; demonstrates use of at least 10 empirical articles in support | Work shows evidence of selection and explanation of empirical findings, and some professional experience, to articulate understanding of problem; demonstrates use of at least 7 empirical articles in support | Work shows some evidence of reliance on empirical findings to articulate understanding of problem, but leans heavily on experience; demonstrates use of at least 5 empirical articles in support | Work shows little evidence of reliance on empirical findings to articulate understanding of problem and tends to solely situate conclusions in personal experience; does not include at least 5 empirical sources |
Appropriateness of proposed solution | Work demonstrates ability to infer potential solutions from understanding of problem; work fully evaluates proposed solution and one alternative (particularly in relation to empirical support) | Work demonstrates ability to identify potential solutions from understanding of problem, but may fail to fully justify connection; work comprehensively but not fully evaluates proposed solution and/or one alternative | Work outlines major elements of proposed solution but fails to fully connect these elements to problem; evaluation of proposed solution not fully developed, alternative omitted | Work fails to connect solution to problem and fails to fully explain solution and to provide an alternative |
Feasibility of implementation plan | Work demonstrates evidence of application of frames and conceptual knowledge of the higher education system to problem in a manner that depicts a thorough, feasible, multi-faceted approach to influence change toward the proposed solution to the identified problem; work also pays heed to potential weaknesses and limitations of implementation plan | Work demonstrates evidence of application of frames and conceptual understanding of the system that supports implementation plan feasibility, but fails to assess potential program weaknesses or limitations | Work demonstrates explanation of feasibility and depth of proposed implementation plan, but fails to explicitly connect to identified problem and/or conceptual understanding of higher education; also fails to attend to potential weaknesses or limitations | Work fails to fully demonstrate feasibility and thoroughness of implementation plan, particularly in relation to frames and conceptual understanding of the system; fails to attend to weaknesses and/or limitations |
EDLD 614:
Developing | Emerging | Sophisticated | |
Poster Elements | The element is overly generalized, lacks precision, with little evidence of support and synthesis | The element is more general, yet with reasonable support and adequate synthesis | The element is original, creative, specific, substantiated, defendable, and complex |
Title | |||
Positionality | |||
Evaluation questions & evaluand context | |||
Critique of related literature | |||
Theory driven | |||
Methodological cohesion and soundness | |||
Analysis of data | |||
Conclusions and recommendations to important stakeholders | |||
Clear, well organized, and visually aesthetic representation to all important stakeholders |
EDU 628:
Learning Outcomes 1 and 2
The AHE program faculty formalized their comprehensive exam, these efforts were intended
to (1) make the process and expectations clear to students and faculty, (2) ensure
consistency, and (3) systemize the execution of the procedures and assessment. The
processes for the procedures are in Appendix A. We expect every student to pass the
important milestones associated with degree completion. The committee chair will assess
the students’ readiness for their exams. This proactive strategy uses a developmental
approach to ensure a successful outcome for the student. If the committee chair thinks
that the student may need further development in a content or research area, the chair
will provide the students with additional resources and recommendations for preparation
for the exam. Therefore, our program strives for a 100% success rate for comprehensive
exams based on the use of these proactive and developmental procedures.
AHE Rubric for Assessing Learning Outcomes | |||||
Introductory | Developing | Mastery | |||
Remembering: Student is able to recall information. Key words for assessment include: define, list, recall, and reproduce. | Understanding: Student is able to explain ideas and concepts. Key words for assessment include: classify, described, discuss, explain, translate, and paraphrase. | Applying: Students is able to use information in a new way. Key words for assessment include: choose, demonstrate, illustrate, interpret, solve, and write. | Analyzing: Students is able to distinguish between different concepts/parts to recognize relationships and patterns. Key words for assessment include: compare, contrast, examine, experiment, question, test, criticize, and differentiate. | Evaluating: Student is able to justify a position or decision. Key words for assessment include: argue, defend, judge, support, value, and appraise. | Creating: Student is able to create a new product or point of view. Key words for assessment include: assemble, construct, create, design, develop, and formulate. |
1 | 2 | 3 | 4 | 5 | 6 |
This rubric was created using Bloom’s Taxonomy |
The program does not currently use a common rubric for assessing students’ work. The faculty are working on a common rubric that can be used for all five learning outcomes above. Here is a draft of the rubric that should be implemented in future reviews. It is expected that MEd students would meet the “developing” or “mastering” criteria
Dissertation/Comprehensive Exams
Pass rates will be used to assess learning outcomes one and two listed above. All
XX EdD students who took their comprehensive exams passed, resulting in a XX% pass
rate.
Dissertation
Pass rates will be used to assess learning outcomes one and two listed above. All
XX EdD students who took their comprehensive exams passed, resulting in a XX% pass
rate.
4. Results
a. What was revealed from analyzing learning outcome data regarding student learning?
Describe any result, pattern, or trends that you identify as meaningful.
EDLD 616
On average, M.Ed. students scored xxx on discussion of the problem, reflecting mastery
of content; and xx each on appropriateness and feasibility of proposed solution, reflecting
proficiency (but not mastery) of developing potential solutions—around x-x on the
proposed six-point scale below. In general, our master’s students appear to be reaching
the desired program learning outcomes as a result of participation in EDLD 616 which
is then reflected in the integration of their practical knowledge with course material.
EDU 628:
Students will be assessed on the overall contributions they make to the class as a
whole including class meetings and online engagement as well as on the attributes
of their written/oral submissions.
Comprehensive Exam/Dissertation
3) Who will be assessing the artifacts?
Program faculty, in collaboration with the Graduate Programs Coordinator, Director
of Accreditation, and Department Head, will be assessing data on an annual basis to
identify areas for program improvement. Additionally, program learning outcome assessment
scores for the specific artifact assignment
will not influence the student’s earned grade in the course.
Part 4: Program Assessment Plan
All plans must include assessment rubrics (the methodology of how student artifacts are to be assessed, and a threshold for student success attainment. (The chart below is an example of the information requested…you can configure your rubrics in different ways)
PLO #1 Demonstrate disciplinary skills and knowledge, applying these to analyze problems. | Threshold Values | ||||
Indicators | Level 4 | Level 3 | Level 2 | Level 1 | 80% of students will meet or exceed Level 3 competency |
Identification of problems | Provides multiple perspectives on a problem relevant to their field of study | Describes the complexity of a problem relevant to their field of study | Identifies a problem relevant to their field of study | Partially identifies a problem OR identifies a problem that is not clearly relevant to their field of study | |
Contextualized analysis of problem(s) | Applies disciplinary skills and knowledge to thoroughly analyze a problem relevant to their field of study resulting in a nuanced discussion of perspectives | Applies disciplinary skills and knowledge to thoroughly analyze a problem relevant to their field of study | Logical analysis of a problem relevant to their field of study is guided by elements of disciplinary skills & knowledge | Analysis of a problem relevant to their field of study is incomplete and may include assumptions and statements not supported by disciplinary knowledge. | |
PLO #2 Develop meaningful evidence‐based solutions to complex problems of practice from a position of equity and social justice. | |||||
Indicators | Level 4 | Level 3 | Level 2 | Level 1 | 80% of students will meet or exceed Level 3 competency |
Meaningful evidence-based solutions | Designs a study utilizing appropriate, evidence-based methods firmly grounded in theory and previous literature (or gaps therein) to study a complex problem of practice relevant to their field | Designs a study utilizing appropriate, evidence-based methods grounded in theory and/or previous literature (or gaps therein) to study a complex problem of practice relevant to their field | Designs a study utilizing minimally appropriate, evidence-based methods which may or may not be grounded in theory and/or previous literature (or gaps therein) to study a problem of practice relevant to their field | Designs a study that does not utilize appropriate, evidence-based methods grounded in theory and/or previous literature (or gaps therein) to study a problem of practice relevant to their field | |
Social Justice & Equity | Integrate understanding of social justice and equity into investigation of solutions related to a complex problem relevant to their field, resulting in solutions that support the vitality and self-determination of diverse stakeholders. | Integrate understanding of social justice and equity into investigation of solutions related to a complex problem relevant to their field | Apply minimal understanding of social justice and equity into investigation of solutions related to a complex problem relevant to their field | Does not apply understanding of social justice and equity into investigation of solutions related to a complex problem relevant to their field | |
PLO #3 Integrate personal, practical, theoretical, and empirical knowledge with systemic and systematic inquiry. | |||||
Indicators | Level 4 | Level 3 | Level 2 | Level 1 | 80% of students will meet or exceed Level 3 competency |
Synthesis | Synthesizes personal knowledge, as well as practical, theoretical, and empirical knowledge from the literature to inform systemic and systematic inquiry. | Synthesizes personal knowledge, as well as two of the following: practical, theoretical, or empirical knowledge from the literature to inform systemic and systematic inquiry. | Synthesizes personal knowledge, as well as one of the following: practical, theoretical, or empirical knowledge from the literature to inform systemic and systematic inquiry. | Does not synthesize personal knowledge, as well as practical, theoretical, and empirical knowledge from the literature to inform systemic and systematic inquiry. | |
Critical Analysis | Critically analyzes sources identified for synthesis to identify biases and gaps and utilizes these to inform systemic and systematic inquiry. | Critically analyzes sources identified for synthesis to identify biases or gaps and utilizes these to inform systemic and systematic inquiry. | Conducts surface analysis of sources identified for synthesis to identify biases or gaps and utilizes these to inform systemic and systematic inquiry. | Conducts surface analysis of sources identified for synthesis that does not identify biases or gaps and/or utilizes these to minimally inform systemic and systematic inquiry. | |
PLO #4 Communicate effectively using multiple modes. | |||||
Indicators | Level 4 | Level 3 | Level 2 | Level 1 | 80% of students will meet or exceed Level 3 competency |
Effective Communication | Consistently and successfully utilizes strategies for effective communication, including awareness of the intended audience, purpose of the communication, and clear organization and support to enhance the message. | Frequently and successfully utilizes strategies for effective communication, including awareness of the intended audience, purpose of the communication, and clear organization and support to enhance the message. | Inconsistently utilizes strategies for effective communication, including awareness of the intended audience, purpose of the communication, and clear organization and support. | Rarely utilizes strategies for effective communication, including awareness of the intended audience, purpose of the communication, and clear organization and support. | |
Communication in Multiple Modes | Consistently and successfully demonstrates the ability to communicate effectively in multiple modes, including a variety of written/visual products, oral presentations, and interpersonal communications. | Demonstrates the ability to communicate effectively in multiple modes, including a variety of written/visual products, oral presentations, and interpersonal communications. | Demonstrates basic ability to communicate effectively in multiple modes, including a variety of written/visual products, oral presentations, and interpersonal communications. | Does not demonstrate the ability to communicate effectively in multiple modes, including a variety of written/visual products, oral presentations, and interpersonal communications. | |
PLO #5 Conduct scholarly and professional activities in an ethical manner. | |||||
Indicators | Level 4 | Level 3 | Level 2 | Level 1 | 80% of students will meet or exceed Level 3 competency |
Ethical Scholarship | Adhere to guidelines for ethical scholarship, including those for the protection of human subjects, as well as principles of Diversity, Equity, and Inclusion within all scholarly activities. | Adhere to guidelines for ethical scholarship, including those for the protection of human subjects within all scholarly activities. | N/A | N/A | |
Professionalism | Consistently and successfully adhere to guidelines for professional conduct, including respectful interactions with others, collegiality, and collaboration. | Frequently and successfully adhere to guidelines for professional conduct, including respectful interactions with others, collegiality, and collaboration. | Inconsistently adhere to guidelines for professional conduct, including respectful interactions with others, collegiality, and collaboration. | Infrequently adhere to guidelines for professional conduct, including respectful interactions with others, collegiality, and collaboration. |
Part 5: Program Assessment Plan:
1) How will annual assessment be communicated to faculty within the department? How will faculty participating in the collecting of assessment data (student work/artifacts) be notified?
Faculty, staff, and graduate students in the program meet bi-weekly to discuss programmatic matters. The assessment reports will be communicated in a Fall department meeting each year. This will be communicated ahead of time through the weekly Monday Minutes communication that is sent out to the department community.
2) When will the data be collected and reviewed, and by whom?
Data will be collected each semester and reviewed on an annual basis by program faculty and the Assessment & Alignment Committee within the department. This committee consists of faculty and staff within the department.
3) Who will be responsible for the writing of the report?
The report will be written by program leadership with input and feedback from the faculty.
4) How, when, and by whom, will the report be shared?
The report will be shared by program leadership with the faculty at a fall department meeting. It will be sent to faculty ahead of the meeting in order to provide time for them to preview and prepare to provide feedback at the faculty meeting.
5) How will past assessments be used to inform changes and improvements? (How will Closing the Loop be documented)?
As part of each cycle’s assessment, we will set goals for continuous program improvement. These goals will be revisited throughout the cycle to determine what progress is being made toward the goals and what additional adjustments need to be made to continue progress.
6) Other Comments:
Submit report to programassessment@montana.edu