To produce graduates who have acquired measurable skills in critical thinking, researching, and writing about English literature, language, and writing disciplines and have acquired demonstrable breadth of knowledge in the field. While the number of graduates who have entered PhD programs or taken teaching positions at two- and four-year colleges is an objective measure of our success in accomplishing this goal, not all of our students pursue further graduate degrees or post-secondary teaching. That in mind, the department has determined three measurable learning objectives that apply uniformly to all students taking a graduate degree in English from Sam Houston State University: (1) the demonstration of critical thinking, researching, and writing skills, as measured by their class writing; (2) the demonstration of critical thinking and writing skills and breadth of knowledge, as measured by their performance on the written comprehensive examination; and (3) the demonstration of critical thinking skills and breadth of knowledge, as measured by their performance in oral examinations.

English graduate students will demonstrate their abilities as independent critical thinkers, researchers, and writers capable of employing sophisticated skills in written analysis, synthesis, and evaluation of knowledge and of using a professional idiom in making written arguments. The program's success in achieving this objective will be measured by a holistic assessment of graduate class writing.

The ability of students to write according to accepted professional standards is a direct indicator of the English MA and MFA programs' success in producing graduates who have acquired appropriate critical thinking, researching, and writing skills and are prepared for future professional endeavors. To that end, a significant amount of student writing is required in English graduate coursework.

To assess the effectiveness of class writing assignments in developing students' ability to make sophisticated arguments about literature, language, and writing disciplines in a critical idiom appropriate to professional standards, the faculty will undertake an annual holistic review of representative graduate student writing produced during the reporting period.

At least 92% of representative graduate essays evaluated during the holistic assessment will be scored as acceptable or excellent (a combined score of 5 or higher on the scale described below).

A rubric for evaluating graduate student writing is attached.


Assessment Process:

1. To assure that the assessment reviews a representative sampling of writing, graduate professors in both long terms are asked to submit term papers or other significant writing from every third student listed on their class rosters.

2. Two primary readers from among the graduate English faculty independently read and score each essay under review; in the case of an unreliable result, the essay is referred to a secondary reader, who reads the essay independently, without any knowledge of the previous results (see number 5, below)

3. Each primary reader scores each essay on a 4-point scale, with a score of 4 the highest possible. The two primary scores are added to yield a total, with the final scores ranging from 8 (highest possible) to 2 (lowest possible). A combined score of 5 or higher is passing. A score of 7 or 8  indicates an excellent essay; a score of 5 or 6 indicates an acceptable essay; a score of 4 or less indicates an unacceptable essay.

4. Reliability of the two scores is assumed when both scores from the primary readers are congruent, that is, when they are within 1 point of each other. For example, a score of 6 that would be seen as reliable would mean that both readers marked the essay as a 3. A reliable score of 5 would mean that one reader assessed the essay as a 3 while the other reader assessed it as a 2.

5. Should the primary scores for an essay not be reliable—for example, a 4 and a 1, a 3 and a 1, a 4 and a 2—the essay is referred to a secondary reader. If that reader agrees with the higher score, the essay is certified as acceptable or excellent; if the secondary reader agrees with the lower score, the essay is certified as unacceptable.

On July 23, 2014, a committee of eight English graduate faculty from across a wide range of areas--literature, linguistics, professional writing, and creative writing--undertook the holistic review of graduate student writing for the 2013-2014 academic year. The committee reviewed fifteen essays chosen without prejudice from seven graduate courses in literature, language, and writing disciplines. Two committee members read each essay and rated it on the scale of 1-4 described above; the combined scores are as follow:

Score of 7 (excellent): 2 essays
Score of 6 (acceptable): 3 essays
Score of 5 (acceptable): 7 essays
Score of 4 (unacceptable): 2 essays
Score of 3 (unacceptable): 1 essay

Eighty percent of the essays were deemed excellent or acceptable. This number falls short of the ninety-two percent target.

Critical writing continues to be an important requirement for graduate students in English, not only because it demonstrates their critical thinking and researching abilities and the breadth of their knowledge but also because it trains them practically for professional researching and writing of their own and for teaching research and writing skills to undergraduates and secondary students. 

Outlining an action for improving student writing is difficult: Professors in graduate English courses often presume that their students come to them with sophisticated critical thinking and writing skills in hand. Although they sometimes provide models of good writing in the field, rarely do they devote time--nor do the restrictions of a busy term allow them to devote time--to teaching writing in their classes.

After the last reporting cycle, the Graduate Director consulted with Dr. Carroll Nardone, Director of Writing in the Disciplines, who revised the assessment rubric for the holistic review. The Graduate Director plans to send this rubric to all graduate professors and students in Fall 2014.

The professor assigned to teach the research and methods course (ENGL 5330) required of all incoming English graduate students in Fall 2014 will also give substantial attention to critical researching and writing, especially by providing creditable models from both peer and professional writing. (A qualification: The catalogue description of the research and methods course allows professors to take different approaches. Some treat it as a scientific approach to gathering information and editing; others treat it as an introductory course in graduate writing and critical approaches. The department decided several years ago that it could not mandate that the professor for this class specifically teach writing.)

English students will demonstrate that they have a graduate-level breadth of knowledge in literature, language, and writing disciplines and that they can express that knowledge in writing.

English students will demonstrate that they have a graduate-level breadth of knowledge in literature, language, and writing disciplines and that they can express that knowledge in writing. The program's success in achieving this objective can be measured by the pass rate for the written comprehensive examination required of all students who take a graduate English degree at Sam Houston State University.

A passing score on the written comprehensive examination is a direct indicator that a student in English has acquired a breadth of knowledge in the subject, has developed critical reading and writing skills appropriate to a graduate-level education in English, and is well-prepared for future professional endeavors. For the examination, students choose three comprehensive areas from among thirteen broad topics in literature, language, and writing disciplines. To demonstrate their mastery of a broad range of materials, they are required to choose at least one British literature area and one American literature area and at least one early (pre-1800) British or American literary area and one later (post-1800) British or American literary area. For each area, students are given a reading list of works selected by faculty area experts.

During the exam itself, the student chooses one of three questions for each area and has two hours to respond to that question. A double-blind grading system is used to evaluate the candidates' proficiency. Three graduate faculty members read and evaluate each essay.

At least 90% of examination essays will pass (with a grade of pass or high pass).

An examination grading rubric and sample pass, fail, and high pass essays are attached.

During the reporting year 2013-2014 (including Summer 2014), MA candidates in English wrote sixty-two comprehensive examination essays; this number includes retakes of essays that had previously failed. The results follow:

Total number of passing essays: 50 (81%)
Total number of failing essays: 7 (11%)
Total number of high passes: 5 (8%)

Eighty-nine percent of the total essays passed (with a grade of pass or high pass).

Conclusions about finding: While this result falls just short of the 90% target, it is substantially better than last year's 82%. It is also probably better representative of our graduate students' abilities and preparation because there was a much larger sample this reporting period (sixty-two essays, as compared with last year's thirty-three).

Because the pass rate has risen steadily over the past three reporting cycles, from 69% for 2011-2012 to 82% for 2012-2013 to this year's 89%, measures implemented to help students prepare for the written comprehensive examination may be yielding results. (An informal survey suggests, for example, that most of the students whose essays passed had attended one or more of the Graduate Director's exam preparation sessions.). However, we need to continue to monitor the pass rate over the next two or three cycles before drawing any firm conclusions.

Because the pass rate has risen over the past three reporting cycles, we seem to be achieving some success in preparing students for the exam. As much as anything else, this may be a matter of impressing upon the students the importance of sound preparation. We will continue to help students prepare:

1. The Graduate Director will continue to publish an exam prep booklet and to conduct biannual sessions, during which he discusses the exam process, suggests strategies for preparing and for addressing exam questions, and presents exemplary questions and responses.

2. Although the comprehensive examination is expressly separated from graduate coursework, some graduate faculty continue to use typical exam questions for midterm and final examinations, as a way of accustoming students to comprehensive exam expectations and circumstances.

3. Faculty continue to give advice about the exam to students who approach them. Despite some discussion at a recent department meeting, the suggestion that faculty be assigned as mentors to a certain number of students has not yet been implemented.

English graduate students will demonstrate their knowledge and critical thinking skills through oral arguments. We believe that the ability to make such arguments is necessary for future professional pursuits like teaching and further graduate education. The program's success in achieving this objective can be measured by the pass rate for the oral defense required of all thesis students and the oral comprehensive examination required of all non-thesis students.

A passing grade on the oral examination required of all students who take the English MA or MFA degree at Sam Houston State University is a direct indicator that graduates are able to demonstrate their critical thinking skills and breadth of knowledge in the field. Thesis students sit for a one-hour oral defense of the thesis; having passed the written comprehensive examination, non-thesis students sit for a one-hour oral comprehensive examination covering the same three areas as those on the written exam. A committee of three graduate faculty members examines each student, awarding the candidate a pass, high pass, or fail, according to her or his ability to respond to specific questions. The committee for the oral defense of thesis comprises the members of the student’s reading committee; the oral comprehensive examination committee comprises area experts appointed by the Graduate Director.

At least 92% of degree candidates will pass the oral defense of thesis or oral comprehensive exam at the first sitting or upon retaking it.

Thesis defense and oral comprehensive exam grading rubrics are attached.

During the reporting year 2013-2014 (including Summer 2014), twelve students sat for oral comprehensive examinations; six students sat for oral defenses of their theses. Sixteen students earned passes, and two students earned high passes (one each for the oral comprehensive exam and thesis defense).

One hundred percent of the students passed the oral examination during the reporting period. This number exceeds the ninety-two percent target.

Conclusions about findings: In last year's assessment of the oral examination, we suggested that the oral defense of thesis and the oral comprehensive examination are unequal measures of our candidates' abilities to demonstrate critical thinking skills: Thesis students know the subjects of their projects as well as, sometimes even better than, the examining faculty and have a much narrower range of material; the thesis defense sometimes becomes an exercise in congratulations upon a job well-completed. (One other important factor is that supervising faculty do not allow a thesis defense until the candidate is ready to defend her or his project; whille the non-thesis student is required to take the oral comprehensive exam immediately after passing the written exam, then, the thesis student has greater scheduling flexibility.) Non-thesis students, who sit for the oral comprehensive exam, must show mastery of a much wider range of topics in literature, linguistics, and writing disciplines; have less control over the questions asked and the direction of the discussion; and are sometimes examined by faculty experts whom they have not met before the examination.

Faculty continue to express disappointment with the peformances of students in the oral comprehensive examination. During this cycle, however, one student who sat for the comprehensive exam was awarded a high pass (with one student also being awarded a high pass for the thesis defense). Although the number is statistically insignificant, it does suggest that students can meet a standard of excellence in the oral comprehensive exam (see the attached rubric).

One hundred percent of students who have sat for the oral defense of thesis or oral comprehensive exam in the last five reporting cycles have passed. There are still complaints among examining faculty, however, that students taking the oral comprehensive examination respond with weak arguments and inadequate knowledge.

In our statement of action for the last reporting cycle, we stressed that the oral examination should not be the only measure of a student's ability to express critical thinking skills and breadth of knowledge orally. It is, however, one of the few uniform measures, since not all graduate classes require oral presentations.

In a department meeting specifically about graduate matters, the issue of the oral exam was raised briefly, without much further discussion or consensus, since most attention was focused on the written examination (also one of the few uniform measures of our students' critical thinking abilities and breadth of knowledge).

Although an oral component is not yet required in any graduate courses, the professor assigned to teach the mandatory research and methods class (ENGL 5330) to incoming students in Fall 2014 has agreed to include a formal unit on oral presentations.

Beginning in August 2014, during the comprehensive examination prep sessions, the Graduate Director will also incorporate suggestions for preparing for the oral comprehensive examination. (A qualification: Students are not required to attend the prep sessions.)

Faculty will also continue to encourage students to participate in academic conferences, at which they must not only present their arguments about literature and language orally but also respond to questions and challenges from the professional audience.