2014 Master of Science in Science Education (MSSE) Program Assessment Report

Evidence was collected throughout the 2012-2013 and 2013-2014 academic years and analyzed to determine program progress toward the MSSE Program Learning Outcomes (Appendix A). Data analyzed for this report included science content and core course grade averages, MSSE Graduate Exit Survey responses, capstone paper/project scores, pass rates for project/presentation defense, IRB submission and approval rates, and the inquiry course project scores (see MSSE Program Assessment Plan in Appendix A). Below are the results of the analysis by learning outcome. All grades and scores are reported on a four point scale.

Program Learning Outcome 1 - Science educators who graduate from MSSE will have increased content knowledge required to effectively teach within their respective science disciplines.

Content from various science disciplines is taught in approximately 70 different courses offered both online and on campus through MSSE and Extended University’s National Teacher Enhancement Network (NTEN). These are courses designed to enhance the science content knowledge for educators in disciplines such as biology, chemistry, physics, earth sciences, plant sciences, astronomy, and microbiology, etc. Course assessments include quizzes, tests, and labs as well as graduate level writing assignments that include the organization of content for lesson development. Thus, course scores are good indicators of progress made toward the content knowledge learning outcome. Average overall scores for science courses over the 2012-2013 and 2013-2014 academic years ranged from 3.15 to 4.00 with the lowest average scores calculated in chemistry and physics (see Appendix B). In addition, 82% of MSSE graduates reported, through the MSSE Exit Survey (2013), that the program, to a great extent, is effective in increasing educators’ knowledge and understanding in the scientific content areas. One hundred percent of graduates reported that the program was effective in this area to some or a great extent. These data are indicative of the progress that MSSE students make toward increasing their science content knowledge, MSSE Program Learning Outcome #1.

Program Learning Outcome 2 - Science educators who graduate from MSSE will have acquired research-based skills and strategies required to effectively teach science.

Research-based science specific pedagogy skills and strategies are taught in core and elective courses in the MSSE curriculum. These are courses in which students learn important strategies for the teaching and learning of science such as inquiry, science and engineering processes, formative assessment and evaluation methods, prior knowledge probes, and discrepant events, etc. Assignments include the study, development, implementation, and reporting of lessons and activities utilizing these skills and designed around such strategies. Thus, course scores are good indicators of progress made toward the pedagogy outcome. Average overall grades for these courses during the program year ending August 2013 ranged from 3.52 to 3.94. Average overall grades for these courses during the program year ending August 2014 ranged from 3.59 to 4.0 (see Appendix B). In addition, MSSE graduates (77%)

reported through the MSSE Exit Survey (2013) that the program, to a great extent, is effective in increasing educators’ knowledge and understanding in teaching science. Again, 100% of students reported that the program was effective in this area to some or to a great extent. Graduates also reported at a rate of 86% that the program, to a great extent, is effective in increasing educators’ knowledge and experience regarding the use of assessment and evaluation strategies for improving practice. Responses to the survey’s open ended question, “In what ways, if any, did the MSSE degree program make you a better science teacher?” were overwhelmingly positive, indicating that the program is effective, from graduates’ perspectives, in providing experiences that help its students to become better science educators. Statements in response to open ended questions indicate that as a result of their program coursework graduates have become more effective at teaching in the following ways: increase in content knowledge; acquisition and understanding of learning assessment strategies, inquiry strategies, experience in teaching in the field as well as other research-based tools for teaching science; an understanding of how students learn science and how to differentiate learning for students; a greater awareness of the impact of teaching practice on student learning; and a working understanding of reflective research processes. These data are indicative of the progress that MSSE graduates make toward becoming better science educators during their time in the program, MSSE Program Learning Outcome #2.

Program Learning Outcome 3 - Science educators who graduate from MSSE will be able to identify, analyze, and ethically investigate problems specific to teaching and learning science; develop solutions and strategies to solve those problems through reflective research processes; and to effectively communicate findings.

The MSSE culminating project and professional paper requires students to identify problems related to teaching and learning science within the context in which they teach, investigate problems using ethically sound research methods, to collect, analyze, and interpret data, and to report findings to a public audience during an annual summer science symposium. Students not only present their projects to the public, but they must defend their findings, written in professional papers, to their graduate committees. Since MSSE students do much of the work toward this culminating project in the core courses, EDCI 505, 509, and 575, course grade averages are good indicators of the extent to which this research/communication outcome is being met. Average overall grades for these courses during the program year ending August 2013 ranged from 3.52 to 3.94. Average overall grades for these courses during the program year ending August 2014 was 3.59 to 3.98 (see Appendix B). Average capstone project scores (which include problem identification, research design, IRB training and approval, project implementation, professional paper authorship suitable for MSU Library archive, presentation and paper defense) were 3.796 in 2013 and 3.781 in 2014 (Appendix C). In addition, 86% of MSSE graduates reported through the MSSE Exit Survey (2013) that the program, to a great extent, is effective in providing the educator with methods of research for improving practice. Ninety-nine percent of students reported that the program was effective in this area to some or a great extent. These data are indicative of MSSE graduates’ ability to effectively conduct research regarding problems specific to teaching and learning science and to effectively communicate their findings, MSSE Program Learning Outcome #3.

Conclusion

Assessment data collected during the 2012-2013 and 2013-2014 academic years indicate that program learning outcomes are being met. MSSE graduates complete their degree with an increase in science content knowledge and knowledge of science processes as a result of the science courses offered in the program as well as through interactions with MSU science faculty. Students overwhelming report that they are more prepared to teach science as they gain new skills in assessing the learning of their own students and learn new strategies for effective science instruction. As a result of the program’s capstone process, MSSE students become proficient at conducting reflective educational research and in communicating their findings of such research in writing, through oral communication, and through public presentation. The following quotes taken from the 2013 MSSE Graduate Exit Survey support these ideas.

The program made me a better science teacher in many ways. Being a student again made me more understanding and empathetic of the demands placed on my own students. The science courses improved my content knowledge and the education courses improved my teaching practice. Being a part of a larger community of educators broadened my perspective on education and gave me additional resources.

I increased my content knowledge, my own abilities to research and write, and a strong desire to continue my own professional development. I feel like I became a better scientist.

By making me more analytical, developing a deeper content knowledge, and perhaps most importantly by causing me to systematically reevaluate assessment in my classroom.

Communication with program graduates indicates that graduates have increased confidence in their abilities to effectively teach science and have been provided opportunities to become teacher leaders as a result of their work and experience in MSSE as evidenced by the following quotes taken from the 2013 Graduate Exit Survey.

By allowing for a range of science course offerings, I could boost my skills and knowledge of topics I currently taught. It gave me confidence with my content knowledge so that I could then focus on learning the best teaching and assessing practices.

I now serve on several committees within my district because of my work in the MSSE program. I was also recently chosen to serve on a state curriculum committee.

It taught me how to evaluate myself and monitor my progress. The data don't lie. I had no formal education in teaching before this, so it was nice to realize that some of the crazy ideas that I had were actually supported by research. It helped me to decide my approach to teaching new classes. It gave me perspective on how to improve the ones I already teach.

Formal assessment data along with other communication with MSSE program graduates indicate that the program is meeting the needs of these professional science educators, allowing them to remain in their profession while pursuing a degree within their field. While the evidence is strong that program outcomes are being realized, the MSSE program believes that its quality depends on continuous improvement as a result of evidence collected on its success. Over the past several years, the following changes have been made as the program evolves to better meet the need of its students.

• IRB training and approval required of all research involving human subjects
• Professional papers are now being archived in the MSU Library
• A successful model of distance advising has been developed and implemented
• Program inquiry course has been updated to include NGSS and is now a required course

In efforts to improve already strong student research skills, the program is now investigating the possible addition of a “data analysis and representation” unit to its core curriculum. The unit will be piloted in the 2014 fall semester within one section of the program’s core EDCI 509 course. This pilot will help to determine how to best implement an opportunity for students to improve data analysis skills during their time in the program.

Appendix A

MSSE Program Assessment Plan

*Applicable MSSE Graduate Exit Survey Questions

Using the following rating scale, please indicate the extent to which you feel the MSSE program contributes to the following goals: 3 = to a large extent, 2 = to some extent, 1 = not at all

1. To increase the educator’s knowledge and understanding in the scientific content
2. To increase educator’s knowledge and understanding in teaching
3. To increase educator’s knowledge and experience regarding the use of assessment and evaluation strategies for improving practice.
4. To provide the educator with methods of research for improving

---

9. In what ways, if any, did the MSSE degree program make you a better science teacher?

Appendix B

MSSE Course Assessment Outcomes

September2014

Academic Year 2012-2013 Fall 2012, Spring 2013, Summer 2013

Academic Year 2013-2014 Fall 2013, Spring 2014, Summer 2014

Appendix C

Click here to download a printable copy of this report