To produce graduates who have acquired measurable skills in critical thinking, researching, and writing about English literature, language, and writing disciplines and have acquired demonstrable breadth of knowledge in the field. While the number of graduates who have entered PhD programs or taken teaching positions at two- and four-year colleges is an objective measure of our success in accomplishing this goal, not all of our students pursue further graduate degrees or post-secondary teaching. That in mind, the department has determined three measurable learning objectives that apply uniformly to all students taking a graduate degree in English from Sam Houston State University: (1) the demonstration of critical thinking, researching, and writing skills, as measured by their class writing; (2) the demonstration of critical thinking and writing skills and breadth of knowledge, as measured by their performance on the written comprehensive examination; and (3) the demonstration of critical thinking skills and breadth of knowledge, as measured by their performance in oral examinations.

English graduate students will demonstrate their abilities as independent critical thinkers, researchers, and writers capable of employing sophisticated skills in written analysis, synthesis, and evaluation of knowledge and of using a professional idiom in making written arguments. The program's success in achieving this objective will be measured by a holistic assessment of graduate class writing.

The ability of students to write according to accepted professional standards is a direct indicator of the English MA and MFA programs' success in producing graduates who have acquired appropriate critical thinking, researching, and writing skills and are prepared for future professional endeavors. To that end, a significant amount of student writing is required in English graduate coursework.

To assess the effectiveness of class writing assignments in developing students' ability to make sophisticated arguments about literature, language, and writing disciplines in a critical idiom appropriate to professional standards, the faculty will undertake an annual holistic review of representative graduate student writing produced during the reporting period.

At least 92% of representative graduate essays evaluated during the holistic assessment will be scored as acceptable or excellent (a combined score of 5 or higher on the scale described below).

A rubric for evaluating graduate student writing is attached.


Assessment Process:

1. To assure that the assessment reviews a representative sampling of writing, graduate professors in both long terms are asked to submit term papers or other significant writing from every third student listed on their class rosters.

2. Two primary readers from among the graduate English faculty independently read and score each essay under review; in the case of an unreliable result, the essay is referred to a secondary reader, who reads the essay independently, without any knowledge of the previous results (see number 5, below)

3. Each primary reader scores each essay on a 4-point scale, with a score of 4 the highest possible. The two primary scores are added to yield a total, with the final scores ranging from 8 (highest possible) to 2 (lowest possible). A combined score of 5 or higher is passing. A score of 7 or 8  indicates an excellent essay; a score of 5 or 6 indicates an acceptable essay; a score of 4 or less indicates an unacceptable essay.

4. Reliability of the two scores is assumed when both scores from the primary readers are congruent, that is, when they are within 1 point of each other. For example, a score of 6 that would be seen as reliable would mean that both readers marked the essay as a 3. A reliable score of 5 would mean that one reader assessed the essay as a 3 while the other reader assessed it as a 2.

5. Should the primary scores for an essay not be reliable—for example, a 4 and a 1, a 3 and a 1, a 4 and a 2—the essay is referred to a secondary reader. If that reader agrees with the higher score, the essay is certified as acceptable or excellent; if the secondary reader agrees with the lower score, the essay is certified as unacceptable.

On July 8, 2015, a committee of seven English graduate faculty from across a wide range of areas--literature, professional writing, and creative writing--undertook the holistic review of graduate student writing for the 2014-2015 academic year. The committee reviewed fourteen essays chosen without prejudice from six graduate courses in literature, language, and writing disciplines. Two committee members read each essay and rated it on the scale of 1-4 described above; the combined scores are as follow:

Score of 8 (excellent): 2 essays
Score of 7 (excellent): 4 essays
Score of 6 (acceptable): 3 essays
Score of 5 (acceptable): 4 essays
Score of 4 (unacceptable): 1 essay

A single essay was ranked as unacceptable (4). All others were ranked as passing (5 or above), with seven essays scored as acceptable (5 or 6) and six essays scored as excellent (7 or 8). The 93 percent rate of acceptable papers exceeds the target of 92 percent.

Outlining a course of action for improving student writing remains difficult: While graduate professors and administrators uniformly agree that critical writing and expository writing are among the most important professional skills that our students must develop, we also concede that there are different ways to develop these skills. As evidence is the variety of writing that the holistic reading committee reviewed: annotated bibliographies, descriptive and critical book reviews, papers applying specific critical theories to works of literature, close readings of texts, linguistic analyses, research papers on pedagogical methods, and technical/professional reports. These represent not only writing about different subjects but also writing in different academic modes.

Notwithstanding this variety in both writing topics and writing modes, the graduate faculty still agree on certain standards for excellence, as measured within their respective contexts. We are satisfied that the holistic reading rubric is adaptable to the various types of writing.

As noted in the action section for the 2013-2014 reporting cycle, one obvious class in which to impress upon all of our degree candidates the standards of excellence for graduate-level/professional writing is ENGL 5330, the research and methods course required of all incoming students. The problem, however, is that even here instructors take different approaches to researching and writing: Some treat the class as a scientific approach to gathering information and editing manuscripts; others treat it as an introductory course in graduate writing and critical approaches. The department decided several years ago that it could not mandate that the professor for this class specifically teach writing, as long as she or he fulfills the contractual obligations for the class, as stated in the graduate catalogue.

Because we cannot presume that professors in this single required course are teaching writing in the same modes and by the same methods and are introducing students to the same standards of writing excellence, the responsibility for developing students' writing is then in the hands of their other graduate classroom instructors. The results of the last several holistic reading sessions suggest that these professors are doing their jobs satisfactorily, even well. Nevertheless, deciding upon uniform methods for teaching writing at the graduate/professional level remains problematic.

Because the results of the last several reporting cycles show that we are producing competent writers in our graduate courses, the best actions at this time seem, as before, to ask that the faculty continue to keep in mind the objective standards outlined in the reading rubric and to continue to encourage them to provide useful models from both professional and peer writing.

English students will demonstrate that they have a graduate-level breadth of knowledge in literature, language, and writing disciplines and that they can express that knowledge in writing.

English students will demonstrate that they have a graduate-level breadth of knowledge in literature, language, and writing disciplines and that they can express that knowledge in writing. The program's success in achieving this objective can be measured by the pass rate for the written comprehensive examination required of all students who take a graduate English degree at Sam Houston State University.

A passing score on the written comprehensive examination is a direct indicator that a student in English has acquired a breadth of knowledge in the subject, has developed critical reading and writing skills appropriate to a graduate-level education in English, and is well-prepared for future professional endeavors. For the examination, students choose three comprehensive areas from among thirteen broad topics in literature, language, and writing disciplines. To demonstrate their mastery of a broad range of materials, they are required to choose at least one British literature area and one American literature area and at least one early (pre-1800) British or American literary area and one later (post-1800) British or American literary area. For each area, students are given a reading list of works selected by faculty area experts.

During the exam itself, the student chooses one of three questions for each area and has two hours to respond to that question. A double-blind grading system is used to evaluate the candidates' proficiency. Three graduate faculty members read and evaluate each essay.

At least 90% of examination essays will pass (with a grade of pass or high pass).

An examination grading rubric and sample pass, fail, and high pass essays are attached.

During the reporting year 2014-2015 (including Summer 2015), MA candidates in English wrote thirty-seven comprehensive examination essays; this number includes retakes of essays that had previously failed. The results follow:

Total number of passing essays: 24 (65%)
Total number of failing essays: 11 (30%)
Total number of high passes: 2 (5%)

Seventy percent of the total essays passed (with a grade of pass or high pass).

Conclusions about finding: While the pass rate had risen steadily from 69% for 2011-12 to 89% for 2013-14, this year's results show a sharp decline. It is difficult to account for this result, since students had the same resources for preparing as those before them had, and there were no noticeable differences in such things as the areas that candidates chose or the circumstances under which they took the exam.

One possible variable is that students from the new MFA in Creative Writing Program sat for the exam for the first time. However, differences in the pass-fail-high pass rates for English MA and MFA students were negligible.

If we cannot attribute the decline in the pass rate to the type of student who sat for the exam (MA or MFA), there are other possible variables: One is that, despite express warnings, the students sometimes "gamble" by not reading all of the required works on the prescribed reading lists; when they come in to the exam room, they find that they are required to use in their responses certain works or certain kinds of works that they have not read. Another possibility is that they rely too heavily on classes in the specified areas to prepare them for the exams; while certainly classwork should provide foundations for the exam areas, the guidelines expressly caution that part of the preparation process--and part of what the exam sets out to measure--are the students' independent reading skills and their ability to synthesize materials, outside of courses.

1. The first necessary action is to discuss in a departmental meeting the importance of students' developing independent critical skills. While classroom instructors should never be "teaching the exam," a few brief comments in a class about the distinction between coursework and exam work would be appropriate.

2. With the same qualification that graduate faculty should not be teaching students the exam in their courses, some faculty may continue to use typical exam questions for midterm and final tests, as a way of accustoming their students to the expectations and circumstances of the comprehensive examination.

3. The Graduate Director will continue to publish an exam preparation booklet and to conduct biannual prep sessions. In these sessions, students are taken through the exam process, given strategies for preparing and sitting for the exam, and shown exemplary responses to typical questions.

4. We will continue to encourage individual faculty to meet with students preparing for the exam, to suggest strategies for preparing and sitting for the comps.

English graduate students will demonstrate their knowledge and critical thinking skills through oral arguments. We believe that the ability to make such arguments is necessary for future professional pursuits like teaching and further graduate education. The program's success in achieving this objective can be measured by the pass rate for the oral defense required of all thesis students and the oral comprehensive examination required of all non-thesis students.

A passing grade on the oral examination required of all students who take the English MA or MFA degree at Sam Houston State University is a direct indicator that graduates are able to demonstrate their critical thinking skills and breadth of knowledge in the field. Thesis students sit for a one-hour oral defense of the thesis; having passed the written comprehensive examination, non-thesis students sit for a one-hour oral comprehensive examination covering the same three areas as those on the written exam. A committee of three graduate faculty members examines each student, awarding the candidate a pass, high pass, or fail, according to her or his ability to respond to specific questions. The committee for the oral defense of thesis comprises the members of the student’s reading committee; the oral comprehensive examination committee comprises area experts appointed by the Graduate Director.

At least 92% of degree candidates will pass the oral defense of thesis or oral comprehensive exam at the first sitting or upon retaking it.

Thesis defense and oral comprehensive exam grading rubrics are attached.

During the reporting year (Fall 2014-Spring 2015), two students sat for oral comprehensive examinations; two students sat for oral defenses of their theses. Three of the four students earned passes, and one student earned a high pass (for a thesis defense).

One hundred percent of the students passed the oral examination during the reporting period. This number exceeds the ninety-two percent target.

Conclusions about findings: In last year's assessment of the oral examination, we suggested that the oral defense of thesis and the oral comprehensive examination are unequal measures of our candidates' abilities to demonstrate critical thinking skills and to make oral arguments: Thesis students know the subjects of their projects as well as, sometimes even better than, the examining faculty and have a much narrower range of material; the thesis defense sometimes becomes an exercise in congratulations upon a job well-completed. (One other important factor is that supervising faculty do not allow a thesis defense until the candidate is ready to defend her or his project; whille the non-thesis student is required to take the oral comprehensive exam immediately after passing the written exam, then, the thesis student has greater scheduling flexibility.) Non-thesis students, who sit for the oral comprehensive exam, must show mastery of a much wider range of topics in literature, linguistics, and writing disciplines; have less control over the questions asked and the direction of the discussion; and are sometimes examined by faculty experts whom they have not met before the examination.

While faculty have often expressed disappointment with the peformances of students in the oral comprehensive examination, the examiners during this reporting cycle were generally well-pleased with the candidates' ability to make oral arguments and with their demonstration of comprehensive knowledge in the field. Because only two students sat for oral comps, however, the results may not be statistically significant.

One hundred percent of students who have sat for the oral defense of thesis or oral comprehensive exam in the last six reporting cycles have passed.

In our statement of action for the last reporting cycle, we stressed that the oral examination should not be the only measure of a student's ability to express critical thinking skills and breadth of knowledge orally. It is, however, one of the few uniform measures, since not all graduate classes require oral presentations.

In a 2014 department meeting devoted specifically to graduate program matters, the issue of the oral exam was raised briefly, without much further discussion or consensus, since most attention was focused on the written examination (also one of the few uniform measures of our students' critical thinking abilities and breadth of knowledge).

Beginning in August 2014, during the written comprehensive examination prep sessions, the Graduate Director incorporated suggestions for preparing for the oral examination as well. It is difficult to tell whether these remarks had any effect upon the students' performance: As before, one hundred percent of students passed the oral exam. And, as before, students who sat for the exams were not required to attend the exam prep sessions, so there is no way to measure the effectiveness of this preparation for the oral exam.

We do suggest as one action, however, that graduate faculty continue to encourage students to participate in academic conferences, at which they must not only present their arguments about literature and language orally but also respond to questions and challenges from the professional audience.