2019-2020 BS in Natural Resources and Rangeland Ecology

Annual Program Assessment Report

Academic Year Assessed: 2019/2020
College: College of Agriculture
Department: Animal and Range Sciences
Submitted by: Patrick Hatfield

Program(s) Assessed:   
Indicate all majors, minors, certificates and/or options that are included in this assessment:

Annual Assessment Process (CHECK OFF LIST)

1. Data are collected as defined by Assessment Plan
                        YES__X___                         NO_____            
2. Population or unbiased samples of collected assignments are scored by at least two faculty members using scoring rubrics to ensure inter-rater reliability.

                                YES__X___                       NO_____            

3. Areas where the acceptable performance threshold has not been met are
                       YES__X___                         NO_____             NA_____            
4. Assessment scores were presented at a program/unit faculty meeting.
                       YES__X __                           NO_____
5. The faculty reviewed the assessment results, and responded accordingly (Check all appropriate line)

             Gather additional data to verify or refute the result. _____

             Identify potential curriculum changes to try to address the problem _____

             Change the acceptable performance threshold, reassess _____

             Choose a different assignment to assess the outcome _____

             Faculty may reconsider thresholds_____
             Evaluate the rubric to assure outcomes meet student skill level _____
             Use Bloom’s Taxonomy to consider stronger learning outcomes _____
             Choose a different assignment to assess the outcome_____
OTHER:  

6. Does your report demonstrate changes made because of previous assessment results (closing the loop)?            YES__X___                   NO_____

1. Assessment Plan, Schedule and Data Source.
a. Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data). (You may use the table provided, or you may delete and use a different format).

*Data sources can be items such as randomly selected student essays or projects, specifically designed exam questions, student presentations or performances, or a final paper.  Do not use course evaluations or surveys as primary sources for data collection.

b. What are your threshold values for which you demonstrate student achievement? (Example provided in the table should be deleted before submission)

2. What Was Done
a. Was the completed assessment consistent with the plan provided? YES_X____ NO_____
If no, please explain why the plan was altered.

b. Please provide a rubric that demonstrates how your data was evaluated.

The Rubric for the Assessment of: Knowledge (Learning outcome 1) was used in evaluating these assignments (see below). 

3. How Data Were Collected
a. How were data collected? (Please include method of collection and sample size).

b. Explain the assessment process, and who participated in the analysis of the data.

Student papers from WILD 355 (Wildlife-Livestock Habitat Restoration, Fall 2019) were used in the evaluation of this learning outcome.  The class assignment is included below.  There were 10 NRRE students in the class and all 10 students had their work evaluated against the rubric.  Dr. Clayton Marlow reviewed three papers, Dr. Bret Olson reviewed four papers and Dr. Craig Carr reviewed three papers.  Papers were read and evaluated against the four indicators of subject content knowledge and each indicator given a score out of 3. The final score was an average value of the four indicator scores.  The threshold value for this learning outcome is 80% of students scoring above two. 

The results of our evaluation are presented in table 1.  Eight of the 10 students evaluated had a score of two or better meeting our desired threshold.  The average score across all students was 2.1.  Ninety percent of students scored two or better for indicators one and two, 100% scored two or better for indicator three and 80% scored two or better for indicator four. 

Table 1.: Student evaluation scores.

Class Assignment:

WILD 355

Mid-Term Paper Assignment

Learning Outcome:  Increasing the likelihood of successful habitat improvement projects by recognizing and quantifying competition among ungulate species using the same habitat. 

The Challenge:  In ecological and wildlife conservation literature competition within and between species is described in terms of overlapping similarities in diet, foraging behaviors, reproductive habitats and access to water. Failure to recognize the extent (or degree) of overlap in habitat requirements between species has led to the failure of many habitat improvement projects because an unrecognized species often outcompetes the target species for the improved features. To minimize the likelihood of failure managers need to identify the extent of competition among ALL species using the habitat and incorporate alternatives to lessen the cumulative pressure on the resource you’re trying to improve.

General Background:

1. The study area is the Yellowstone Northern Winter Range comprising the Gardiner and Cinnebar Basins north of Gardiner, Montana.
2. The area has historically supported high numbers of elk, moderate numbers of mule deer, low numbers of pronghorn and bighorn sheep.
3. The Interagency Bison Management committee now wants to allow increased bison use of the basin to reduce pressure on winter ranges within Yellowstone National Park. Their target wintering population outside the Park is 3,000.
4. The Forest Service and Montana Department of Natural Resources have undertaken a series of major habitat improvement on the winter range. The improvement covers both returning former haylands to native grasslands and rehabilitating big sagebrush communities that where heavily overused by elk from 1980 to 2005.

Approach:

1. Review the ecological literature to learn what vegetation communities (habitat types) exist on the winter range and then obtain species composition metrics, e.g. 30% bluebunch sagebrush, 70% sagebrush, for each of the identified vegetation community types.
2. Review the wildlife data to:
   1. Determine winter diets for each species (diets must be measure, i.e. 5% juniper, 12% wheatgrass, etc.)
   2. Return to the literature to find descriptions of the landforms (and slopes) preferred by these species during the winter
3. Construct a general table showing:
   1. the species composition of the vegetation communities identified for the winter range
   2. diet composition for each wildlife species
   3. preferred landform for each wildlife species
4. Analyses
   1. Use a ChiSquare or similar nonparametric analysis to determine
      1. The similarity or dissimilarity in diet and landform preference for all five species
      2. Similar diets/landform preferences = competition for either forage or landform
   2. Use the same statistical tool to determine which wildlife species are most likely to use:
      1. existing haylands
      2. ecologically intact native grass or shrub communities
      3. ecologically intact big sagebrush

5. Develop a paper that first addresses the likelihood of continued habitat improvement progress if bison are allowed to move onto the Northern Winter Range. Second, will the current suite of wildlife species compete with each other for forage and space ultimately limiting rehabilitation efforts? Finally, list at least two management actions (other habitat improvement actions) that could be undertaken to limit bison impact on current habitat improvement efforts.

Due Date:  23 October 2019

Grading Criteria:

1. Paper length – 6 to 7 single spaced pages, citation of references used in developing paper following format in Northwest Science or Rangeland Ecology and Management.
2. Organization, punctuation, spelling, sentence structure or syntax = 40% of grade; 15% on adequacy of references, the remaining 45% will be based on how well you demonstrate what you’ve learned.

4. What Was Learned
Based on the analysis of the data, and compared to the threshold values provided, what was learned from the assessment?

a. Areas of strength

Students demonstrated knowledge of the ecological information needed to develop management recommendations including specific information on vegetation, animal diet and habitat preferences, along with the physical attributes of the landscape. Students did a reasonable job at identifying the problem, although some did a better job than others at developing a robust ecological question – maybe more of a writing issue than a deficiency in knowledge and is better addressed in learning outcome 3. Students were able to locate and synthesize scientific evidence appropriate to the question at hand, however, there was some variability in the richness and relevance of the literature surveyed and included.  This evaluation showed that NRRE students seem generally well prepared to seek out and synthesize appropriate scientific evidence to support sustainable management of natural resources and in the development of habitat management plans.

b. Areas that need improvement

Improvement in some students in framing the problem and providing the necessary background information would benefit those papers that were evaluated slightly lower (indicators 1 and 2).  A richer discussion/understanding of the problem being addressed should lead to the development of a more robust evaluation and synthesis of the literature which in turn will aid in the creation of management objectives and actions.  Some students need to be more critical of the sources they select and ensure they are relevant to the ecosystem at hand (indicator 3).

Improvement in the development of specific management actions that are tied to the evidence provided would generally be beneficial (indicator 4).  The student papers we evaluated seem to put a lot of energy in to analyzing and synthesizing data from the literature, yet they lacked a robust linkage from these data to their management recommendations.  More in-depth data driven explanations for their management suggestions will benefit our students as they graduate into the workforce because this becomes the defensible basis for professional management decisions that they will be asked to make.

The students evaluated varied in their writing skills and a poorly written paper will mask the knowledge possessed by any particular student.  Although writing is not the learning objective evaluated in this assessment, it is a skill that should continue to be cultivated in our students so they can convincingly convey their knowledge and provide sound and defensible natural resource management recommendations.

5. How We Responded

a. Describe how “What Was Learned” was communicated to the department, or program faculty. Was there a forum for faculty to provide feedback and recommendations?

We provided a copy of this report to the Range Program Faculty and provided them an opportunity to comment or seek clarification.

b. Based on the faculty responses, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?
YES______ NO___X____

 If yes, when will these changes be implemented?

Please include which outcome is targeted, and how changes will be measured for improvement.  If other criteria is used to recommend program changes (such as exit surveys, or employer satisfaction surveys) please explain how the responses are driving department, or program decisions.

c. When will the changes be next assessed?

NA

6. Closing the Loop
a. Based on assessment from previous years, can you demonstrate program level changes that have led to outcome improvements?

No.  Based on this assessment, our outcome scores declined slightly since the last evaluation.  The 2017 evaluation had an average score of 2.29 while this evaluation had an average score of 2.1.  The recommendations from the last evaluation of this learning objective were geared toward the development of better writing skills for NRRE students.  This remains an issue and the suggestions brought forth in 2017 remain valid in 2019/20.  These include:

1. Incorporate more writing assignments in NRRE courses.
2. Provide example papers, grading rubrics and the common mistakes of most papers.
3. Create a writing packet for our majors that would help identify problems.
4. Create writing studios similar to the MSU Writing Center which would be run by our graduate students

Other suggestions for improvement in this learning outcome that are a result of this assessment include:

1. Strengthening the connection between data and management recommendations. Provide examples of well written management plans that objectively evaluate the literature and provide a strong scientific direction in the development of management strategies.  Providing examples and deconstructing the components may aid students in understanding how to connect knowledge to management activities. 
2. Increased effort from students in developing management plans – several of the lower scoring papers seemed to lack effort (in writing, in literature search, in thoroughness of investigation). Providing more opportunity to develop plans with time scheduled for peer review/instructor review may provide the time to incrementally increase effort put into term papers and address some of the deficiencies we noticed during this evaluation.  Although lack of student effort is not a deficiency in the NRRE program this suggestion identifies a potential opportunity to improve students’ understanding of the effort required to develop a successful natural resource management plan.

Submit report to programassessment@montana.edu