2019-2020 BS in Plant Science – Plant Biology Option

Annual Program Assessment Report

Academic Year Assessed: 2019-2020 College: College of Agriculture

Department: Plant Science and Plant Pathology (PSPP)

Submitted by: Matthew Lavin. Reviewed by Ryan Thum, Andreas Fischer, Mike Giroux, Alan Dyer, Jennifer Britton, and Bill Hoch

Program Assessed: Plant Biology

Indicate all majors, minors, certificates and/or options that are included in this assessment:

1. Assessment Plan, Schedule and Data Source.

A multi-year assessment schedule showing when all program learning outcomes will be assessed, and by what criteria. Data sources can be items such as randomly selected student essays or projects, specifically designed exam questions, student presentations or performances, or a final paper. Course evaluations and course surveys are not sources of data.

2.  What Was Done

a) Was the completed assessment consistent with the plan provided? YES_X_ NO

Assessment committee members for this academic year were Matthew Lavin, Andreas Fischer, and Ryan Thum. This committee assessed the performance of students in two courses, Botany BIOO 220 and Evolution BIOB 420 (sections 001 and 002). These are two of the required biology courses in the Plant Biology degree option. Fisher instructs BIOO 220. Lavin and Thum co-instruct BIOB 420.

To assess the Plant Biology degree program, committee members Lavin and Fisher met during October 2018 to identify data to collect from students in Botany BIOO 220. Committee members Lavin and Thum met January 2019 and 2020 to identify data to collect from students in Evolution BIOB 420.

The Plant Biology degree option caters to diverse students, including those interested in plant ecology and evolution, bee keeping, pre-biomedical and pre-veterinarian school, medicinal and herbal botany, and wildlife management, for example. Because of this, we generalized the learning outcomes following those of the Department of Biology at Boston University (https://www.bu.edu/biology/). The above table reports three learning outcomes that cover the diversity of general biology undergraduate students.

b) Rubric that demonstrates evaluation of data. This rubric pertains to learning outcome 1, as assessed in BIOO 220 during 2018-2019 and learning outcome 2, as assessed in BIOB 420 during 2019-2020.

3.  What Data Were Collected

a) Methods of data collection.

Assessment of learning outcome 1. For BIOO 220 Botany, an essay question was included in the final exam. Students enrolled = 34. Students answering the essay question = 27. This question asked, “Please explain how green plants (e.g., bryophytes, ferns, gymnosperms, and angiosperms) differ from animals in terms of alternation of generations and the production of gametes. The answers to this question potentially demonstrate a student’s ability apply information, ideas, and concepts related to plant diversity and a student’s ability to evaluate and synthesize information, ideas, and concepts. Specifically, demonstrating an ability to apply information and synthesize knowledge derives from an essay question that allows a student to expound on what are the underlying similarities of diverse groups of plant and how do diverse plant groups differ collectively from animals by other than the ability to photosynthesize. Answers to the essay question reside in the document entitled “BIOO 220 Botany evaluation essay Fall 2018.pdf”(PDF page 47).

Assessment of learning outcome 2. For BIOB 420 Evolution, we designed questions that demonstrate whether students could apply information, ideas, or concepts related to study design and critical assessment of scientific literature. We also designed the questions to demonstrate whether students could evaluate or synthesis information, ideas, and concepts related to study design and critical assessment of scientific literature. Question on daily formative assessments preceded related question on monthly exams. Students enrolled = 230, almost all of whom answered all questions. Of the 230 students enrolled in Spring Semester2020 BIOB 420 Evolution, 149 students enrolled in section 01 and 81 students in section 02.

The following three examples questions (a1-a3) represent the kind of questions designed to assess whether students were learning how evolutionary biologist design historical studies dealing with origins of populations, sources of infections, and how traits evolve. These examples reflect the general trends of the class of 230 students enrolled in BIOB 420 Evolution. The study of evolutionary history is particularly vulnerable to misinterpretation by the public (or the scientifically untrained mind). Specifically, evolutionary studies commonly produced phylogenetic trees. Inevitably, phylogenies of human population samples are open to misrepresentation. The Evolution course thus not only addresses study design but also the biases the public have when exposed to the results of well-designed historical evolutionary studies, particularly when humans are the subject of study.

A standard design of evolutionary studies involves population genetic samples that serve reference points. Such samples often resolve in a phylogenetic analysis as early branching or old lineages. Such population genetic samples could include suspected source areas of migration events (in wildlife management studies), sources of infection (in phylogenetic epidemiological studies), or as references to determine how traits evolve from older ancestral forms into more recent derived forms (where ancestral and derived refer to character states, not individuals, populations, or species).

a1. Identifying source versus founder populations. Populations samples resolved as early branching (older) lineages in a phylogeny often represent samples intentionally included in order to test a “place of origins” hypothesis. For human population samples, African samples are consistently resolved as the earliest branching oldest lineages, which supports the hypothesis that modern humans population ultimately originated in Africa. However, a mental bias of people not exposed to science often includes the ideas that modern African populations are “primitive” or ancestral relative to perhaps “superior” non-African populations. These biases become apparent when the same study design applies to questions involving non-human samples (e.g., populations of North American wolves and species of Ponderosa pine trees).

The above human population phylogeny is a consensus of all human population phylogenies derived from various study designs that include differences in population or genetic methods of sampling.

African populations resolve as early branching lineages with older average age estimates and non- African populations consistently resolve as more recently evolved lineages and with younger average age estimates. What does this imply?

This question first asked to the class on 4 March 2020 (BIOB 420 Quiz Mar 4.pdf - p. 34) as part of an in-class formative assessment had the following responses:

• 24.5% of the class incorrectly responded, “Modern African human populations are ancestral to all modern non-African populations.”
• 2.5% of the class incorrectly responded, “African human populations are primitive compared to more advanced non-African populations.”
• 52% of the class correctly responded, “Africa is the ancestral area from which all non-African
• 21% of the class correctly respnded, “All of these responses are implications of the above phylogenetic ”

Essentially the same question asked again on 4 May 2020 (BIOB 420 Exam 4 May 4.pdf - p. 10) as part of the final exam revealed an improvement in understanding study design of evolutionary historical studies:

• 8% of the class incorrectly responded, “Modern day African populations are ancestral to modern day European populations.”
• 4% of the class incorrectly responded, “African populations are primitive.”
• 4% of the class incorrectly responded, “African populations are unperfected compared to European populations.”
• 3% of the class correctly responded, “The continent of Africa served as the source area from which modern non-African human populations ultimately migrated.”

The increase from about 52% to 90% of the class was the general trend for this sort of “place of origins” question, which formed course content throughout the semester.

a2. Using control samples to identify infection routes. Populations samples resolved as early branching (older) lineages in a phylogeny often represent control samples. If one person allegedly infected another with HIV1, control samples allow this hypothesis to be properly tested.

Questions related to this aspect of study design and critical assessment of scientific literature required students to infer a phylogeny from a genetic distance matrix (above) and understand the design of that matrix. Such questions first asked to the class on 22 April 2020 (BIOB 420 Quiz Apr 22.pdf - p. 38) were part of an in-class formative assessment. Included was the following: RNA virus evolve rapidly with respect to nucleotide substitutions. When RNA viral genetic evidence is sampled years after the alleged criminal event, the genetic evidence has evolved. What constitutes a genetic "match" between victim samples and the alleged source samples if the genetic evidence evolves quickly over time?

• 45% of the class correctly responded, “Monophyly of the victim and the alleged source ”
• 17% of the class incorrectly responded, “Victim samples are more closely related to samples from a local control than to samples from the alleged ”
• 26% of the class incorrectly responded, “Alleged source samples are more closely related to samples from a local control than to samples from the ”
• 12% of the class incorrectly responded, “A genetic match cannot be ”

A similar question asked again on 4 May 2020 (BIOB 420 Exam 4 May 4.pdf - p. 10): Phylogenetic evidence revealed that Janice Trahan was infected by HIV from a blood sample taken from Donald McClelland. The evidence included:

• 6% of the class responded partly correct with, “The combined viral samples from Trahan and McClelland formed a monophyletic group with respect to Louisiana local control ”
• 3% of the class responded partly correct with, “Trahan HIV strains averaged younger in age than those sampled from ”
• 3% of the class responded partly correct with, “Trahan HIV strains were not monophyletic with respect to McClelland HIV strains and were thus not genetically differentiated from McClelland ”
• 88% of the class correctly responded, “All ”

The increase from about 45% to 88% of the class was generally the trend to this sort of “control sample” question that was a focus of course content more towards the end of the semester.

a3. Using reference samples to determine the direction of character state evolution. Populations samples resolved as early branching (older) lineages in a phylogeny often represent reference samples that permit the inference of character state evolution. Study design includes specifying an outgoup (e.g., other mammals is data set below) and ingroup (the other samples in data set below). Ancestral character states occur in the outgroup, whereas character states found only in the ingroup are derived. Ancestral and derived refer to character states, not the samples (taxa).

Questions related to this aspect of study design and critical assessment of scientific literature required students to infer a phylogeny from a data matrix of character states (above) and understand the design of that matrix. Such questions first asked to the class on 29 April 2020 were part of an in-class formative assessment. Included was the following (BIOB 420 Quiz Apr 29.pdf - p. 42): Characters T4-T7 are genomic characters indicating the absence (0) or presence (1) of transposable elements. When present, each of these transposable elements is inserted at a specific position in the genome of artiodactyl and cetacean species. In the above data set, the values of m and g for each of these four characters, T4-T7, is, respectively:

• 16% of the class incorrectly responded, “2 and ”
• 46% of the class correctly responded, “1 and ”
• 21% of the class incorrectly responded, “4 and ”
• 17% of the class incorrectly responded, “8 and ”

A similar question asked again on 4 May 2020 (BIOB 420 Exam 4 May 4.pdf - p. 10): Baboons represent the outgroup in this data set. For the locomotion (gait) character, 0 = quadrupedal, 1 = quadrupedal but planting sides of palms on ground, 2 = quadrupedal but planting knuckles on ground, and 3 = fully bipedal. Which are the derived character states in this locomotion character?

• 14% of the class incorrectly responded, “0, 1, 2, and ”
• 70% of the class correctly responded, “1, 2, and ”
• 8% of the class incorrectly responded, “2 and ”
• 8% of the class incorrectly responded, “only ”

The increase from about 46% to 70% of the class was generally the trend to this sort of “character state evolution” question, which was a focus of course content towards the end of the semester.

b) Explain assessment process and data analysis.

During January 2019, Lavin and Fisher read and evaluated the essay question written by students in Botany BIOO 220 as part of their final exam. Lavin and Fisher used the above rubric to evaluate student answers to the essay question. They each read the answers separately but ultimately came to conclusions about student performance during a meeting. The committee concluded that students met the basics of learning outcome 1, “Demonstrate knowledge of fundamental principles spanning the breadth of biology.” In this context, students demonstrated an ability to apply information, ideas, or concepts at a competent level. However, students rarely demonstrated an ability to synthesize information, ideas, or concepts at a competent level (e.g., how plants fundamentally differ from animals including ecologically). Regardless, they demonstrated an ability to apply fundamental knowledge to answering the question, which included a knowledge of diverse plant groups and how they collectively differ from all animal groups.

During May 2020, Lavin and Thum read and evaluated the questions from daily formative assessments and related question presented on monthly exams. Student demonstrated an ability not just to apply information, ideas, and concepts, but also the ability to evaluate or synthesize information, ideas, or concepts. They did so at a competent level because they were able to connect the underlying similarities of study designs from diverse kinds of case studies (e.g., wildlife management, epidemiology, and paleontology) that formed the course content. In summary, the majority of students demonstrated increasing knowledge of study design and critical assessment of the scientific literature.

4.  What Was Learned

a) Areas of strength

The committee of Lavin and Fisher concluded from an assessment of learning outcome 1, “Demonstrate knowledge of fundamental principles spanning the breadth of biology,” that students can readily report the facts, inter-relate them, and sometimes extend and connect them to disciplines outside of botany. The committee of Lavin and Thum concluded from an assessment of learning outcome 2, “Demonstrate effective communication in the scientific method, experimental design, or critical assessment of the scientific literature” that students understand the design of evolutionary historical studies and are unlikely to misrepresent the finding compared to the public untrained in historical sciences.

b)Areas that need improvement

A long-term goal from our assessments is to have addition biology courses incorporating exam questions, projects, and essays that allow students to make connections among the diverse disciplines of biology from astrobiology and microbiology to the omics fields and data science, all of which now regularly use data science.

6.  How We Responded

a) Communication of “What Was Learned” to the department or program faculty. Was there a forum for faculty to provide feedback and recommendations?

The long-term goal reported above requires curriculum modifications. A recent step taken in this direction involved PSPP faculty discussions that led to the new requirement that students enrolled in the Plant Biology program now take both introductory statistics courses, STAT 216 and 217. This is because STAT 217 introduces the R programming language, which is an important programming language in evolutionary biology and ecology, as well as data science. No forum yet exists for faculty to provide feedback and recommendations but all involved are on board with incorporating data science into the Plant Biology program.

b) Based on the faculty responses were there any curricular or assessment changes for the following year? (Such as plans for measurable improvements or realignment of learning outcomes).

PSPP faculty have a strong genetics focus and five PSPP faculty, Drs. Michelle Flenniken, Robert Sharrock, Jamie Sherman, and Ryan Thum are involved in teaching Genetics BIOB 375. BIOB 375 is required of many biology degree programs including Plant Biology. Following discussions with these five PSPP faculty, we are considering having the Genetics course participate in the program assessment of Plant Biology. The positive side of this is that Genetics is a quantitative science and a solid knowledge of this field is required for success in many biology graduate programs. Thus, it would be straightforward to develop specifically designed exam questions or student projects on which to evaluate student performance. The downside of including Genetics is that four faculty members teach it and with varying course content. This results in not being able to integrate program and course learning outcomes consistently.

c) Next assessment of program and any changes.

We will assess learning outcome 3, “Attain the technical and/or analytical skills required for employment or post-graduate education in biology or biology-related careers, including professional careers and science education” in BIOB 420 Evolution during Spring Semester 2021. PSPP faculty including Drs. Matt Lavin and Ryan Thum will collect data from this class from specifically designed exam formative assessment and exam questions. We will design these to learn whether students understand methods that transform quantitative data into graphical output and whether they can look at a data set and readily infer results that apply to wildlife management, forensics, and paleontological questions.

7.  Closing the Loop

Based on assessment from previous years, can you demonstrate program level changes that have led to outcome improvements?

The assessment from 2017-2018 failed in many ways (e.g., no rubric to evaluate essay questions) and we submitted our 2018-2019 assessment of the Plant Biology program too late for review. Regardless, what we are learning is that the assessment tools in Brightspace D2L provide the opportunity to evaluate regularly, via formative assessment questions, whether students understand course concepts. We will be taking more advantage of what D2L offers in forthcoming assessments.