Data on this page is being updated over the summer of 2024.

2022-23 Academic Year CAEP Standard 4.2 Impact and Outcome Measures

These measures relate to our programs accredited by the Council for the Accreditation of Educator Preparation.

 

There were a total of 268 responses to our 2022 Completer Survey.  Of those 268 responses only, complete responses were analyzed (n=195).  There was a 4-point Likert scale (1-not effective, 2-somewhat effective, 3- effective, and 4-very effective).

Of the 20 questions, only 3 averaged above a 3.0 indicating completers felt that MSU was effective in their preparation of those areas.  Completers felt that their teacher education program was most effective in preparing them for implementing instruction aligned with Montana State Content Standards (M=3.09, SD=.85), demonstrating accurate content knowledge (M=3.06, SD=.77), and engaging with colleagues in a professional manner (M=3.06, SD=.79).  In contrast, their teacher education program was less effective in preparing them for differentiate instruction effectively for English Learners (EL) (M=1.88, SD=.86), responding productively to negative behavior (M=2.10, SD=.91), and initiating professional conversations with supervisors (M=2.54, SD=.99).

MSU TEP Completer Survey 2022 - Chart of data in table below

Click chart to enlarge.

Completer Survey 2022

 

Mean

SD

N

% Not Effective

% Somewhat Effective

% Effective

% Very Effective

Design instruction based on learners' development.

2.71

0.73

195

6%

29%

55%

11%

Differentiate for English Learners (EL).

1.88

0.86

195

39%

38%

18%

5%

Collaborate to meet needs of all students.

2.79

0.81

195

5%

31%

44%

20%

Actively engage students in learning.

2.98

0.79

195

3%

23%

47%

27%

Respond productively to negative behavior.

2.10

0.91

195

29%

39%

24%

8%

Purposefully use instructional technology.

2.91

0.84

195

4%

28%

41%

27%

Demonstrate accurate content knowledge.

3.06

0.77

195

2%

22%

45%

31%

Encourage critical thinking for problem solving.

2.91

0.79

195

3%

28%

45%

25%

Analyze assessment data to improve instruction.

2.74

0.84

195

7%

31%

43%

19%

Plan instruction based on knowledge of community context.

2.61

0.80

195

8%

34%

46%

12%

Implement instruction aligned with Montana State Content Standards.

3.09

0.85

195

4%

21%

38%

37%

Use a variety of instructional strategies.

2.95

0.82

195

4%

25%

44%

27%

Participate in ongoing professional development opportunities.

2.79

0.85

195

7%

29%

44%

21%

Reflect on how instructional choices affect students.

2.89

0.85

195

5%

28%

41%

26%

Respect beliefs, norms, and expectations of families.

2.97

0.87

195

7%

19%

44%

30%

Apply knowledge of the legal responsibilities of teachers.

2.65

0.90

195

10%

34%

37%

19%

Initiate professional conversations with supervisors.

2.54

0.99

195

19%

26%

38%

17%

Engage with colleagues in a professional manner.

3.06

0.79

195

3%

19%

47%

31%

Accurately incorporate the 7 Essential Understandings of Indian Education for All.

2.63

0.96

195

13%

33%

32%

22%

Relevance of preparation for job

2.66

0.81

195

6%

37%

42%

15%

We value our stakeholders and utilize their feedback through a variety of ways in our Educator Preparation Program.

Field Placement

For in-area placements, administrator approval is collected via emails sent from the MSU Field Placement Office with lists of all pre-service/in-service teacher matches for practicum and student teaching. In this way, placement partnerships are verified at both district and building levels. Currently, MSU FPLO houses MOUs with districts in 20 states, and in 2021-22, began a systematic process of updating all local, in-state, and out-of-state MOUs to reflect stakeholder feedback. Feedback from CTs is also highly valued. Each spring, CTs are included in the Partner Survey. This survey openly invites a variety of feedback from all partner stakeholders.

Employer Survey

We participate in the state-wide Continuous Improvement Cooperative in collecting data from employers of our program completers.

Stakeholders’ Breakfast

Each spring, local administrators and district leaders are invited to campus for the annual Stakeholders' Breakfast.  Although curtailed during the pandemic, this meeting was renewed in Spring 2021. The event is a valued opportunity to meet, discuss policy, elicit feedback, and build relationships.

Montana State University - Bozeman

Employer Survey

Survey sent to 133 employers of MSU-Bozeman teacher education program graduates.

48 started the survey (36.1%)

36 completed the survey (27.1%)

Map with twenty-one markers across Montana and one in Salt Lake City

Locations where surveys were completed according to reported longitude and latitude data.

Please indicate the type of school which you currently lead.
Please select the option(s) which is most representative of your school.

N

N
Reply*

[no response]

1 0

Elementary P/K-5

11 6

Elementary P/K-5, Middle School 6-8, High School 9-12

1 1

Elementary P/K-8

6 5

Elementary P/K-8, Middle School 6-8

1 1

Middle School 6-8

6 4

Middle School 6-8, High School 9-12

3 2

High School 9-12

9 9

Full District P-12

10 8

Totals

48 36

 

Please identify grade band in which your teacher(s) is assigned.
Choose all that apply.

N

N
Reply*

[no response]

1 0

Preschool P, Elementary K-5

3 2

Preschool P, Elementary K-5, Middle School 6-8

1 1

Elementary K-5

17 10

Elementary K-5, Middle School 6-8

5 5

Elementary K-5, Middle School 6-8, High School 9-12, K-12 Subject-specific

1 1

Elementary K-5, High School 9-12

1 1

Middle School 6-8

4 2

Middle School 6-8, High School 9-12

5 4

High School 9-12

9 9

K-12 Subject-specific

1 1

Totals

48 36

*Surveys links were emailed to 133 school adminstrators known to have hired MSU graduates. Of those, 48 began the survey, while 36 completed the evaluative questions.

Nineteen items were evaluated on a scale from Very Satisfied (1) to Very Unsatisified (4). Note that lower scores indicate more satisfaction.

Seventeen respondents evaluated our graduates with a mean Very Satisfied (1.00 to 1.47) including five respondents evaluating our graduates with Very Satisfied on all nineteen items.  Eighteen respondents evaluated our graduates with a mean Somewhat Satisfied (1.53 to 2.39). One respondent evaluated our graduates with a mean Somewhat Unsatisfied (2.53).

The most common response from all respondents on all items was Very Satisfied.

Response Score N
Very Satisfied 1 340
Somewhat Satisfied 2 265
Somewhat Unsatisified 3 70
Very Unsatisfied 4 4

 

The overall mean from all respondents on all items was Somewhat Satisfied.

N Mean SD
679 1.61 0.69

 

The mean scores on each of the nineteen items were as follows:

Question N Mean SD

Design instruction based on learners’ development.

36 1.58 0.69

Differentiate instruction effectively for English Learners (EL).

34 1.91 0.75

Collaborate to meet the learning needs of all students.

36 1.47 0.61

Actively engage students in learning.

35 1.57 0.70

Respond productively to negative behavior.

36 1.78 0.76

Purposefully use instructional technology.

35 1.29 0.46

Demonstrate accurate content knowledge.

36 1.39 0.60

Encourage critical thinking for problem solving.

36 1.64 0.59

Analyze assessment data to improve the effectiveness of instruction.

36 2.03 0.81

Plan instruction based on knowledge of students in their community context.

36 1.83 0.74

Implement instruction aligned with Montana State Content Standards.

36 1.36 0.49

Use a variety of instructional strategies.

36 1.64 0.64

Participate in ongoing professional development opportunities.

36 1.53 0.61

Reflect on how instructional choices affect students.

36 1.75 0.73

Respect beliefs, norms, and expectations of families.

35 1.54 0.74

Apply knowledge of the legal responsibilities of teachers.

36 1.56 0.77

Initiate professional conversation with supervisors.

36 1.61 0.60

Engage with colleagues in a professional manner.

36 1.44 0.61

Accurately incorporate the 7 Essential Understandings of Indian Education for All.

36 1.75 0.81

 

     

Stakeholders Map with indicators at Willow Creek, Three Forks, Manhattan, Belgrade, Bozeman, and Livingston

Each year, the Department of Education, partnered with the Field Placement and Licensure Office, hosts Stakeholder Meetings. We invite K-12 administrators and teachers from our local area (all public and private schools from Livingston, Bozeman, Belgrade, Manhattan, Three Forks, and Willow Creek), our current students, Department of Education staff and faculty, the EHHD Dean’s office, and our University Teacher Education Committee partner faculty to meet and discuss our Teacher Education Program.  We provide updates to the program, talk about student trends, receive input and perceptions of our students in the field, and talk about ways our program can be bolstered via work the districts are undertaking.

 

mailboxCall for Comments - The Department of Education at Montana State University—Bozeman hosted an accreditation visit by the Council for the Accreditation of Educator Preparation (CAEP) on September 23, 2023. Programs under review were the undergraduate Teacher Education Program, the graduate Master of Arts in Teaching, the graduate Master of Education in Educational Leadership/Principal Preparation and the graduate Superintendent Certificate. Interested parties were invited to submit third-party comments to the CAEP evaluation team on or before August 12, 2023. Laudatory, critical, and merely observational comments were all welcome. Details at this link.

In 2022, the Montana Office of Public Instruction released new licensure requirements which made Praxis exams optional rather than mandatory for our graduating teacher candidates seeking licensure in Montana. We strongly encourage students to complete a Praxis as part of their licensure journey. Praxis is required for licensure in many states, especially states bordering Montana where many of our graduates have moved in the past several years.  It is advised to take a licensure exam, like a Praxis, as close to completion of coursework as possible, but the dropping of the mandate has reduced the percentage of our graduates taking a Praxis exam.

Praxis is an external exam administered and assessed through the Montana Office of Public Instruction and the national Educational Testing Service. A pre-service teacher (PST) seeking a K-8 or P-3 licensure answers content questions related to Math, Language Arts, Social Studies and Science, while those seeking K-12/5-12 licensure complete the exam designated for their specific content area. For example, an undergraduate PST seeking Secondary Science Broadfield licensure will complete an exam containing questions in the fields of Biology, Chemistry, Physics and Earth Science.

The US Median score here is the median of all scores on that test taken between August of 2020 and July of 2023. This gives us a national benchmark to compare with MSU's average score for tests taken by completers who finished in the four academic years between 2020-21 and 2023-24. If too few completers from that period take the test in an area, those low-N scores cannot be reported for reasons of student privacy as well as statistical validity. Some completers elect to delay the taking of the Praxis test, so scores may change over time as more come in.

Praxis Test of Content Knowledge Scores
Undergraduate Teacher Education Program
Praxis
Test
Code

 

Praxis Test

 

N

 

MSU Mean Score

 

US Median

5018 Elementary Education 142 176 167
5038 English Language Arts 29 179 178
5701 Agriculture 27 166 166
5081 Social Studies 26 169 166
5435 General Science 17 173 161
5134 Art 16 164 163
5113 Music 11 170 166
5857 Health and Physical Education 11 170 164
5161 Mathematics old test 11 163 157
5025 Early Childhood Education 10 180 169
5941 World and U.S. History 8 163 160
5051 Technology Education 7 180 178
5195 Spanish: World Language 6 167 176
5204 Teaching Reading old test, minor only 4 * 170
5122 Family and Consumer Sciences 4 * 162
5235 Biology 3 * 162
5174 French: World Language 2 * 170
5165 Mathematics new test 2 * 167
5245 Chemistry 2 * 159
5183 German: World Language 0 * 182
5931 Government/Political Science minor only 0 * 166
5652 Computer Science minor only 0 * 166
5206 Teaching Reading new test, minor only 0 * 164
5571 Earth and Space Sciences minor only 0 * 161
5265 Physics 0 * 152
5911 Economics minor only 0 * 152

* Not enough scores to report valid statistic

This data was last updated 7/17/2024.

Unlike undergraduates whose program includes their subject area content knowledge, graduate students in the Master of Arts in Teaching (MAT) program must already possess that knowledge. They take their Praxis content knowlege tests as part of their admission process to the program. Elementary level candidates take the four-part Elementary Education: Multiple Subjects test. Secondary level candidate take the content knowledge test appropriate to their subject area. Statistics here are for MAT completers from 2019-20 to 2022-23. The US Median score here is the median of all scores on that test taken between August of 2020 and July of 2023. If too few completers from that period take the test in a area, those low-N scores cannot be reported for reasons of student privacy as well as statistical validity.

 

Praxis Test of Content Knowledge Scores
Master of Arts in Teaching
Praxis
Test
Code

 

Praxis Test

 

N

 

MSU Mean Score

 

US Median

5002 Elementary Education: Reading and Language Arts 30 179 169
5003 Elementary Education: Mathematics 30 181 171
5004 Elementary Education: Social Studies 30 184 165
5005 Elementary Education: Science 30 183 169
5081 Social Studies 14 176 166
5435 General Science 13 176 161
5038 English Language Arts 6 187 178
5018 Elementary Education 3 * 167
5161 Mathematics 3 * 157
5174 French: World Language 1 * 170
5235 Biology 1 * 162

* Not enough scores to report valid statistic

This data was last updated 7/18/2024

Teacher candidates who wish to graduate and attain licensure must reach competency standards during student teaching for the Danielson Performance Assessment through evaluation by both their field supervisor and their cooperating classroom teacher. Content knowledge is measured through a Content Area Assessment of proficiency (evaluated by their cooperating teacher), GPA, and performance on the Praxis Exam (an external evaluation administered and assessed by the Office of Public Instruction) collectively labeled the MACK (Montana Assessment of Content Knowledge).

The Danielson Tool

Both field supervisors and cooperating teachers evaluate pre-service teachers using this performance assessment at the halfway point and at the end of student teaching. Any unsatisfactory score (a score of less than 2 on the rubric) assigned by either the field supervisor or the cooperating teacher at the end of student teaching may result in the pre-service teacher needing to re-take student teaching. 

Praxis Exam

During the review cycle, each graduating UG PST was required to complete the Praxis Exam, an external exam that assesses PST content knowledge, administered and assessed through the Montana Office of Public Instruction. PSTs seeking a K-8 or P-3 licensure have content questions related to Math, Language Arts, Social Studies and Science, while those seeking K-12/5-12 licensure complete the exam designated for their specific content area. For example, a UG PST seeking Secondary Science Broadfield licensure will complete an exam containing questions in the fields of Biology, Chemistry, Physics and Earth Science. PSTs in the MAT are initially evaluated for subject matter knowledge through completion of the appropriate content area PRAXIS exam. (See scores under Measure 3 on this page.)

The MACK

The Montana Assessment of Content Knowledge (MACK) is a single score based on content area GPA, a content area assessment during Student Teaching/clinical practice, and the Praxis content knowledge score. Points are assigned to each area.

Completer Endorsements

In the 2021-22 academic year, 142 individuals completed and graduated from the MSU-Bozeman undergraduate educator preparation program. Of these, 101 were prepared to teach in a single subject area, 22 prepared to teach in two areas, and 19 prepared to teach in three areas. It is important to note that completers often choose not to go directly into teaching following graduation (see table below) and so may not seek licensure until a later date, and some requests for licensure are in process. As of July 18, 2023, the MSU Field Placement Office has recommended 81 of these individuals to teach in one or more of the areas for which they were prepared. Of those, 75 were recommended for licensure in Montana, and 11 were recommended for licensure in other states. Similar situations exist for the graduate Master of Arts in Teaching program as well as the advanced Principal and Superintendent programs.

 

Educator Preparation Program Undergraduate Completers
Data as of July 18, 2023
Academic Year Number of Completers Number Recommended
by Field Placement 
Recommended
in Montana
Recommended
Out of State
2022-23 133    74    73 4
2021-22 142    81    75 11
2020-21 172    140    138 11

 

Master of Arts in Teaching Program Graduate Completers - Elementary
Data as of July 18, 2023
Academic Year Number of Completers Number Recommended
by Field Placement 
Recommended
in Montana
Recommended
Out of State
2022-23 11 3 2 1
2021-22 9 6 6 0
2020-21 11 9 8 1

 

Master of Arts in Teaching Program Graduate Completers - Secondary
Data as of July 18, 2023
Academic Year Number of Completers Number Recommended
by Field Placement 
Recommended
in Montana
Recommended
Out of State
2022-23 10 2 2 0
2021-22 10 6 6 0
2020-21 9 9 8 1

These are the CAEP Accountability Measures for the undergraduate and graduate initial teacher licensure programs. We have also posted our CAEP Accountability Measures for the advanced programs in Educational Leadership.