Maroon and Write
Ms. Clary Photo

Mississippi State University’s (MSU) Quality Enhancement Plan (QEP), Maroon and Write, seeks to improve undergraduate student writing by implementing a writing across the curriculum model, including writing-to-learn strategies and formal writing components. Maroon and Write utilizes direct and indirect measures, qualitative and quantitative measures, and internal and external instruments to evaluate the success of the initiative for both students and faculty. The assessment plan tracks student learning in the designated classes taught by MIWE-trained faculty and in non-MIWE senior capstone classes.

Student Learning Outcomes

Maroon and Write has established three outcomes for this goal, and each outcome has targets for measuring the success toward achieving the outcomes.

  1. Outcome 1: Students will write documents that are appropriately organized, well developed, and clearly worded.
  2. Outcome 2: Students will use Standard English correctly, avoiding errors in syntax, grammar, and usage.
  3. Outcome 3: Students will be more engaged in writing activities.

Evidence of Student Learning Outcomes

Maroon and Write evaluates all three student learning outcomes on an annual basis (refer to the Maroon and Write Scorecard to track the progress of the program). Targets were selected based on baseline data from the target year; however, the program identified intermediate targets that increase every year. Only the final target is listed on the scorecard, but the intermediate targets are provided on the individual Institutional Effectiveness (IE) Reports.

Results have been both positive and negative. Student writing as measured by the ETS Proficiency Profile has improved over the past several years of the program; however, engagement in writing activities as measured by the National Survey of Student Engagement (NSSE) has remained the same. Writing scores based on the Maroon and Write rubric have varied from year to year, and the program has several hypotheses for why the variation has been occurring, which is explained in the Use of Results section. Feedback from students and faculty during focus groups has been constructive. This feedback combined with results from the rubric have inspired the majority of the improvements made to the program. Refer to the section on Assessment Instruments for more information about the instruments and how they were administered.

Summary of Results and use of Results to Improve the Program

Maroon and Write began as a pilot program with a small number of faculty in 2013-14 to gather baseline data and test implementation. The program will continue through 2018-19, and will complete annual assessment cycles. The following paragraphs provide a summary of results and how the assessment results were used to improve the program.

Pilot Year: 2013-14

After the pilot year, the co-directors of Maroon and Write implemented several changes to improve the assessment procedures. The rubric scores seemed to be higher than the Maroon and Write team anticipated for two potential reasons: (1) most of the student writing samples came from classes taught by the MIWE-trained faculty and (2) only one grader evaluated each student writing sample based on the rubric. Therefore, the program implemented the dual-grader system where each writing sample is evaluated independently first and then as a team to determine a validated score. Additionally, Maroon and Write clarified its expectations from faculty to include a formal writing assignment in every Maroon and Write class, which could then be used for assessment purposes. The Maroon and Write team sought additional writing samples from other non-MIWE classes to serve as comparative data.

Pilot Year: 2014-15

The first year of the program resulted in having significantly higher writing scores for students from MIWE classes than from non-MIWE classes (refer to Table 1). Further analysis revealed that the majority of the non-MIWE writing samples were coming from “service” classes rather than classes from the students’ major. The co-directors hypothesized that the level of effort students put into major classes versus service classes was different. Subsequent years of writing sample collections excluded services classes.

Feedback from the MIWE faculty during the focus groups indicated that they would have liked more emphasis in developing a writing assignment. Implemented in summer of 2015, Maroon and Write dedicated one of the three writing coordinators for each MIWE faculty member, and all faculty were required to meet with the writing coordinator to develop an appropriate writing assignment.

Table 1. Independent samples t-test analysis of 2014-15 MSU seniors' writing samples by MIWE students and non-MIWE students

MIWE N Mean Std.Deviation
Problem MIWE 217 3.894* .8293
Non-MIWE 633 3.717* .9698
Thesis MIWE 217 3.742 .9168
Non-MIWE 633 3.592 1.080
Support MIWE 217 3.673* .7928
Non-MIWE 633 3.542* .8707
Word Choice MIWE 217 3.940 .7941
Non-MIWE 633 3.833 .8626
Correctness MIWE 217 3.512** .8114
Non-MIWE 633 3.292** .9571
*p<.05      **p<.001

This chart indicates that seniors in MIWE courses scored higher than their counterparts in non-MIWE courses during 2014-15. For three of the components, the seniors in MIWE courses scored significantly higher: Defining the problem, providing support for the argument, and correctness/grammar.
Read more details about this year’s assessment in the 2014-15 IE Report

Pilot Year: 2015-16

MIWE classes in 2015-16 included students other than seniors, and therefore, the scorecard includes results for seniors and all MIWE students. All comparative data are from senior-level students only. Maroon and Write attributes the significantly higher scores in MIWE students’ “problem” area compared to non-MIWE students to the emphasis placed on assignment design during the MIWE training program (Refer to Table 2). However, three of the other rubric areas had lower scores than their non-MIWE counterparts.

Consistent with the rubric scores, the 2015-16 administration of the ETS Proficiency Profile included results for 155 students from MIWE classes. Maroon and Write felt it appropriate to report the score for all MSU seniors as it has in previous years, and also to identify the scores of students from MIWE classes. The General Education Subcommittee of the University Committee on Courses and Curricula (UCCC) requested that the transfer students be distinguished from the non-transfer students, which is also indicated in the scorecard. Students in MIWE classes had lower proficiency than non-MIWE students, and non-transfer students had higher proficiency than transfer students.

The writing coordinators identified the “thesis” and “support for thesis” components of the rubric as the areas in more need of improvement. Therefore, the writing coordinators implemented a series of workshops through the Center for Teaching and Learning to help faculty better understand these components and how best to improve them.

Table 2. Independent samples t-test analysis of 2015-16 MSU seniors' writing samples by MIWE students and non-MIWE students

MIWE N Mean Std.Deviation
Problem MIWE 473 3.70* .829
Non-MIWE 390 3.55* 1.025
Thesis MIWE 473 3.45 .951
Non-MIWE 390 3.43 .996
Support MIWE 473 3.40* .807
Non-MIWE 390 3.54* .813
Word Choice MIWE 473 3.34* .769
Non-MIWE 390 3.48* .838
Correctness MIWE 473 3.16** .906
Non-MIWE 390 3.41** .922
*p<.05      **p<.001

Read more details about this year’s assessment in the 2015-16 IE Report

Assessment Instruments for Student Learning Outcomes

Maroon and Write has developed or identified four instruments for measuring the impact on student learning: Maroon and Write Rubric, ETS Proficiency Profile exam, NSSE survey, and focus groups.

Maroon and Write Rubric

MSU faculty developed Maroon and Write Rubric as a direct, internal measure of student learning. The rubric contains five components to represent:

  • Background, context, or problem
  • Thesis
  • Support for the Thesis
  • Word Choice and Sentence structure
  • Correctness
Five levels of proficiency:

1= Poor
2= Acceptable
3= Good
4= Excellent
5= Superior
measure the students’ performance in each of the five components.

Starting in 2014-15, Maroon and Write employs two teams of two graders each summer to apply the rubric to all student writing samples. Each grader reviews the writing sample individually and then compares scores with his or her partner. The pair then arrives as a validated score for each writing sample, which is logged in an Excel spreadsheet. If the pair is more than one number different on the rubric, then a third grader is asked to score the assignment to determine a validated score. This process reduces the potential variation among graders and ensures more accurate evaluation of writing skills.

ETS Proficiency Profile

The ETS Proficiency Profile is a standardized, external instrument that directly measures students’ performance in writing, mathematics, critical thinking, and reading skills. The abbreviated version of the test contains 36 questions, with nine questions for each of the skill-set areas. Students have 40 minutes to complete the exam. The writing section of the exam measures students’ knowledge of grammar, language organization, and figurative language. The results of the exam indicate what percentage of students score at proficient, marginal, and not proficient within three levels of understanding. The following levels demonstrate a student’s ability to do the following:

  • ETS Level 1: recognize grammar and word usage
  • ETS Level 2: build upon simple components of writing and incorporate those simple components into more complex sentence structures
  • ETS Level 3: recognize how complex sentences work together for parallelism, idiomatic language, correct constructions, and reduction in redundancy

Each year, the Office of Institutional Research and Effectiveness administers a paper and pencil exam to freshmen and seniors to track the institution’s progress in these skill sets. The institution then compares its results to a group of 10 peer institutions to establish a peer-level benchmark. MSU has performed below its peers in writing, but the gap is narrowing as the Maroon and Write program progresses.

National Survey of Student Engagement

The National Survey of Student Engagement (NSSE) represents an indirect, external measure that has been standardized with hundreds of four-year colleges and universities. The NSSE measures students’ engagement with coursework and studies and how the university motivates students to participate in activities that enhance student learning. This survey is used to identify practices that institutions can adopt or reform to improve the learning environment for students.

Each year, the Office of Institutional Research and Effectiveness deploys the online survey to freshmen and seniors. The institution then compares its results to a group of peers from the same Carnegie classification, from a peer group determined by the NSSE examiners, and from a group of peers that MSU has identified.

Focus Groups

Maroon and Write engages students and faculty in focus groups. Discussion pertains to their experiences with writing in their classes and the engagement with course content. The co-directors assemble these groups and coordinate the discussions each spring.

Technical problems, contact the Help Desk
Information about Mississippi State University, contact opa@msstate.edu.
© 2012 Mississippi State University. All rights reserved.