EHHD_ED_GR_MED_AHE_REPORT_YEAR_0

Graduate Annual Program Assessment Report

MEd Graduate Assessment Process

Program Learning Outcomes (PLOs) one and two will be assessed during this reporting cycle for the MEd

1. Demonstrated disciplinary skills and knowledge, applying these to analyze problems.
2. Develop meaningful evidence-based solutions to complex problems of practice from a position of equity and social justice.
3. Integrate personal, practical, theoretical, and empirical knowledge with systemic and systematic inquiry.
4. Communicate effectively using multiple modes.
5. Conduct scholarly and professional activities in an ethical manner

1. Assessment Methods

a.  List the courses or instruments used to assess each outcome.

2. What Data Were Collected

a. What was collected to assess learning outcomes listed above?
MEd

EDLD 616: The final paper for EDLD 616 will be assessed for evidence of the program learning outcomes. This paper synthesizes students’ work over the semester on a problem faced in administration, resulting in implications for practice in higher education or student affairs. The paper is between 6-8 pages in length and prompts students to analyze and evaluate research evidence to propose a solution to their problem of focus.

EDU 628: Ethical Considerations, Validity, and Reflection: There are four areas to cover as part of the ethical considerations. One, describe any ethical considerations such as power differences, sensitive populations, anonymity, and how you will address them in the study. Two, explain how you will adhere to ethical standards for practice and any potential concerns. Three, discuss any limitations to the evaluation based on the methodology. Four, after the evaluation you will provide a reflexivity of your experiences such as pressures, bias, or problems associated with the evaluation and how you managed or mitigated them. Problems may include but are not limited to financial or resource constraint, political pressures, responsiveness of participants or leadership, other.

Comprehensive Exams (Master’s): three-part comprehensive exam: foundation, specialization, research.
Near the end of completing course work, MEd students will complete a written comprehensive exam or a master’s thesis. Students are expected to demonstrate mastery of the program of study and the ability to interact with the research in that area.  The substantive content of the written exam questions will be determined by the chair in consultation with the committee. Students may be required to meet with the committee to provide oral clarification of their written responses.

b. How sample will be collected?

All final papers from the course were collected, and those completed by students in the M.Ed. program will be assessed for the learning outcomes. Samples will be collected through the D2L Course Home Page typically in the Assignment Drop Box: This is where students submit formal assignments. In the Drop Box Forum, students see the titles for each formal assignment. Students click on the tile of the corresponding assignment you want to submit and add the document as a file and submit.  A census sample will be collected for the comprehensive exams for both Fall and Spring semesters.

3. Data Analysis

a. Who participated in the process, the nature of the rubric utilized (or other norming methods), and the threshold outcome desired.

For EDLD 616, the rubric developed to assess the final papers was informed by both the specific course learning objectives and the broader program learning outcomes as stated above. Three outcomes, reflective of the two program learning outcomes being assessed, were assessed for the paper along a 10-point scale:

EDLD 614:

EDU 628: 

Learning Outcomes 1 and 2

The AHE program faculty formalized their comprehensive exam, these efforts were intended to (1) make the process and expectations clear to students and faculty, (2) ensure consistency, and (3) systemize the execution of the procedures and assessment. The processes for the procedures are in Appendix A. We expect every student to pass the important milestones associated with degree completion. The committee chair will assess the students’ readiness for their exams. This proactive strategy uses a developmental approach to ensure a successful outcome for the student. If the committee chair thinks that the student may need further development in a content or research area, the chair will provide the students with additional resources and recommendations for preparation for the exam. Therefore, our program strives for a 100% success rate for comprehensive exams based on the use of these proactive and developmental procedures.

The program does not currently use a common rubric for assessing students’ work. The faculty are working on a common rubric that can be used for all five learning outcomes above. Here is a draft of the rubric that should be implemented in future reviews. It is expected that MEd students would meet the “developing” or “mastering” criteria.

4. Results

a. What was revealed from analyzing learning outcome data regarding student learning?  Describe any result, pattern, or trends that you identify as meaningful.

EDLD 616
On average, M.Ed. students scored xxx on discussion of the problem, reflecting mastery of content; and xx each on appropriateness and feasibility of proposed solution, reflecting proficiency (but not mastery) of developing potential solutions—around x-x on the proposed six-point scale below.

In general, our master’s students appear to be reaching the desired program learning outcomes as a result of participation in EDLD 616 which is then reflected in the integration of their practical knowledge with course material.

EDU 628:
Students will be assessed on the overall contributions they make to the class as a whole including class meetings and online engagement as well as on the attributes of their written/oral submissions.

Comprehensive Exams
Pass rates will be used to assess learning outcomes one and two listed above. All XX MEd students who took their comprehensive exams passed, resulting in a XX% pass rate.

b. Describe how the above results were communicated to the department and used to develop plans for improvement.

The AHE faculty meet on a bi-weekly basis. The faculty have conducted “in-program” assessments for the past ten years. The university's decision to formalize program assessment is allowing our faculty to make the assessment process continuous, systematic, and well documented for program improvement. Per department leadership direction, faculty will revise our assessment plans to standardize the procedures and to make them a routine part of our program.

5. Assessment and Subsequent Action

a. Based on assessment, we will describe any changes (content, pedagogical, structural, etc.) that are an outgrowth of the assessment to include timelines and number of graduates

Sample
EDLD 616
No major changes are planned for this fall based on the assessment results, although the course assignments will be more tightly coupled to program and course learning outcomes to aid in assessment of the student’s learning and of the course itself. However, the next time EDLD 616 is assessed, scores on the final paper this semester will be compared to scores presented in this report to evaluate the need for changes to better facilitate student learning toward these outcomes.

EDLD 614: Two new evaluation frameworks were added to this course including an Indigenous evaluation framework and a International sustainable development framework of the United Nations. Both frameworks expand the scope of evaluation to address pressing needs in Indigenous and international communities.

EDU 628: 

Comprehensive Exams

We will continue to assess each learning outcome on an annual basis to evaluate overtime progress. In preparation for the biennial report, we will take a random sample of the material collected in between reporting cycles to prepare a meta-assessment of the program. We will also revise our current assessment plans to reflect the biennial reporting cycle.

We have begun to maintain records on student outcomes including research presentations, publications, and grant activity at the program level. Previously, each faculty member maintained their own records in the Faculty Activity Database, which didn’t allow us to readily track student productivity. Creating a database for student productivity will allow us to ensure students are gaining valuable experiences outside of the classroom, networking, and engaging with national scholars. These scholarly products and experiences address learning outcomes 1-5. 

6. Closing the Loop

a. If there have been changes in program/curriculum to reflect concerns from previous assessments, what impact have the changes had (if any) on achieving the desired level of student learning outcomes?

We made two important programmatic changes from the last reporting cycle. First, we changed our continuous admission cycle to Fall only. This has enabled us to create a cohort model for incoming students and a consistent schedule of classes. This change has resulted in two important outcomes. One, the annual enrollment has helped us better plan for our schedule of classes by knowing which courses we need to offer, how often, and when. Second, the graduate students have created a grassroots organization—Department of Education Graduate Students (DEGS). The DEGS provides a social and academic network for students to support their integration and involvement in the department and university.

To align with the national recruitment trends, we adapted an early applicant submission of February 1 for fall admission. All additional fall applicants are due by the secondary application date, April 1. The early admission deadline is to recruit competitive students in and out of the state and to give them priority access to graduate assistantships. We currently have xx students with assistantships in areas such as athletics, Office of Student Engagement, Office of Inclusion and Diversity, Gallatin College, and residence life to name a few. These GA positions provide a valuable resource to the university in the way of human capital. In turn, students gain a valuable applied learning context to experience first-hand the blending of theory and practice. This latter point goes far to meet Learning Outcome 1 and 2.

Appendix A

Comprehensive Exams
The Master’s comprehensive exam has three parts—comprised of three, multi-part questions including foundations, specialization, and a research article critique.  The page length for each question ranges from 12-15 pages excluding references. The written exam should be in APA 7th Ed. format and returned to the committee. The chair of the committee will work with the student to develop appropriate and relevant exam questions. 

The student will receive the exam questions from the chair on a Friday and have seventeen days to complete the written portion. The student should acknowledge the receipt of the exams when they receive the initial email from the chair. The student should contact the chair if they have any questions during the exam process. Upon completion of the written work, the student should return the written responses to the chair. The chair will send a reply email acknowledging she received the exams. The chair will then send the exam responses to the entire committee for review, lasting no more than two weeks, and subsequent communication to students of the result and graduate school forms completed.

Committee Procedures
The chair should notify the committee and let them know the proposed timeline for the student’s exam period prior sending the exam questions to the student. The chair will send the committee the completed exams and ask the committee to respond with their evaluation of the written work within two weeks. If the committee agrees that the student’s written work warrants a “pass,”

If the committee views the comprehensive exam and does not warrant a pass on any one of the three responses, the committee will prepare questions for the candidate to present an oral presentation. The committee’s questions may address only the part of the written exam in question. The scope of the questions should probe the student to (1) expand on what is written, (2) clarify any ideas or analysis of the work, and (3) anything else that would absolve the non-Pass. 

Upon completion of the responses, the student will be excused from the room while the committee deliberates on the evaluation of the student’s oral responses. The committee will decide on whether the student “passed” or “did not pass” the exam. If the student passes the exam, then the committee is not expected provide the candidate with any formal written feedback. If the committee decides that the student did not pass the exam, the committee chair will prepare recommendations for remediation. The department will follow the graduate student guidelines for retaking any part of the written and oral exams.