Improving Critical Thinking And Analytic Reasoning
Students completing the critical thinking and logic courses in our curriculum will develop a broad-based skills in critical thinking and formal logic.
Objective
Demonstrate Critical Thinking Skills
Critical thinking skills are an essential component of philosophical work. The ability to analyze arguments and draw conclusions from available information serve students in practically every aspect of life.
Indicator
Response Scores On TACTS
All students who take PHL 262 will be tested on their critical thinking skills. All faculty who teach PHL 262 will administer the Texas Assessment of Critical Thinking Skill (TACTS)s, an externally validated test of critical thinking skills, in a pre-test/post-test format.
The TACTS is a broad-based assessment of critical thinking skills that goes beyond the current scope of PHL 262. This will allow the faculty to determine areas that may be added to our current curriculum in the future. A copy of the current TACTS is attached. A copy of the credited responses is attached.
The Philosophy Program Coordinator, currently Dr. Fair, will be responsible for ensuring that all faculty who teach PHL 262 effectively administer the pre- and post-tests in every section of their course.
One faculty member, currently Dr. Sanford, will be responsible for gathering pre- and post-test data from the faculty members who teach PHL 262.
Criterion
Statistically significant improvement from the TACTS pre-test to the TACTS post-test.
A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test.
Finding
PHL 262 Findings
In both the fall and spring semesters of the 2008-2009 academic year student progress was measured using the Texas Assessment of Critical Thinking Skills (TACTS). Combining Fall and Spring, 365 students took the pre-test while 340 took the post-test. 300 students took both tests. Among students who took both tests, the pre-test measure was an average of 8.17 items answered correctly while the post-test measure was 14.07. The improvement of 5.9 items was statistically significant with a p-value less than .05.
Their is no breakdown for philosophy majors because that sample was too small for analysis.
The attached spreadsheet contains the data for all students who completed both tests.
Action
Expanding Critical Thinking Skills Taught to Students
During 2008-2009, the Philosophy Program was successful in demonstrating that our critical thinking course (PHL 262) significantly improved student knowledge of critical thinking skills.
The data from the team that developed the TACTS show that a knowledge gap exists with respect to decision making when an outcome depends upon the conjunction of two probabilistic events. Their data show that less than 20% of those tested correctly answered the following question:
"George is waiting for two of his customers, Fuzzy Logic Computers, Inc. and Stalking Horse Designs, to pay their bills. If either of them pays before the end of the month, then George can pay his supplier. But if neither of them pays, then George will have to take out a bank loan. George estimates that the chance that Fuzzy Logic will pay in time is 70% and the chance Stalking Horse Designs will pay in time is 60%. Assuming that his estimates are correct and that the two events are independent, what is the chance that George will have to take out a bank loan?
(a) 12%
(b) 40%
(c) 65%
(d) 42%
(e) 88%"
Given the role of probabilistic information in decision-making situations, the Philosophy Program will add a component to its critical thinking course during 2009-2010 aimed a improving student performance in this area.
A copy of the slide showing the proportion of students who correctly answered each question and a copy of the TACTS (version 8) that was used to gather this data.
Action
PHL Data Collection
Due to ongoing issues with data collection/reporting, Dr. Sanford will create a spreadsheet template for all PHL faculty members to follow. It will indicate the data that they are required to report for each section. Once it is approved, the template will be distributed to all PHL faculty by the Program Coordinator prior to the start of each semester to ensure that data is collected and reported in a format that allows the Philosophy Program to measure its progress. The template will be distributed to the PHL faculty prior to 8/1/09.
Objective
Demonstrate Competence With Formal Logic
Student's understanding of formal logic provides a foundation for rigorous analysis of arguments. All philosophy majors will have the opportunity to acquire competences in propositional logic and predicate logic.
Indicator
Response Scores On Embedded Logic Questions
All students who take PHL 362 (Introduction to Contemporary Logic) will be tested on knowledge of basic logical concepts, propositional logic, and predicate logic. All faculty who teach PHL 362 will administer the a series of embedded questions through the semester and as part of the final examination. These will serve as an indicator of student competence with the subject matter.
Following a review of best practices for the teaching of this course, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a an introduction to the concepts of formal logic, propositional logic, and predicate logic.
The attached documents provide the sample questions similar to those that will be used in the embedded assessments and the credited responses. Because some of these questions, particularly the logical proofs, have answers that could be memorized if the questions were known in advance, exam security requires that the actual questions not be posted in a public document. The Program faculty will use questions that are essentially similar, but sufficiently distinct to limit the potential for academic dishonesty. A group of Program faculty has reviewed the particular questions that will be embedded each semester.
Criterion
Correct responses on 70% of embedded questions by 70% of students
At least 70% of students will correctly answer 70% of the embedded questions.
Finding
PHL 362 Findings
The questions were not embedded in the PHL 362 sections during 2008-2009. As such there is no data.
Action
PHL 362 Action
Dr. Sanford, who is responsible for conglomerating assessment data within the Philosophy Program, has discussed the missing data with the Program Coordinator, Dr. Fair. Dr. Fair and Dr. Sanford will remind all faculty who teach PHL classes of the importance of collecting assessment data. They send reminder e-mails to all faculty two weeks prior the start of class and two-weeks prior to finals to ensure that the data is collected. Also, Dr. Fair and Dr. Sanford will distribute an Excel template to all faculty. The template will indicate the data that they are required to report for each section. As a final step, Dr. Sanford will check that all data has been collected during Fall 2009 and report any issues to Dr. Fair and Dr. WIlson, Chair of Psychology and Philosophy, to allow for corrective action between semesters if necessary.
Goal
Understanding Of General Philosophical Concepts
Students progressing through the introduction and history of philosophy courses in our curriculum will develop a broad-based knowledge of metaphysics, epistemology, and moral theory. These areas form the basis for a general competence in philosophy. Given the small size of the Philosophy Program, it will focus on providing all students with a basic background in each area. In addition, the Program will seek to ensure advanced understanding in the history of metaphysics and epistemology.
Objective
Demonstrate Basic Understanding Of Core Concepts In Philosophy
As students progress through the Philosophy BA, they will acquire a basic understanding of metaphysics, epistemology, and moral theory. This basic information, provided by our introductory courses serves as the foundation for student success in upper-division courses.
Indicator
Statistically Significant Improvement Of Student Scores From Pre-test To Post-test (261/263)
All students in PHL 261 and PHL 263 will be tested on their knowledge of basic concepts in metaphysics, epistemology, and moral theory using a locally standardized pre-test and post-test for each course. Following a review of best practices for the teaching of these courses, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a basic competence in metaphysics, epistemology, and moral theory. The attached documents provide the assessment instruments for PHL 261 and PHL 263 as well as the credited responses for each.
Criterion
Statistically significant improvement from the pre-test to the post-test (261/263)
A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test.
Finding
PHL 261 Findings
In both the fall and spring semesters of the 2008-2009 academic year student progress was measured using a pre-test post-test model. The test was developed by the Philosophy Program faculty to test general knowledge that we would expect students completing PHL 261 to have acquired. This instrument consists of 20 items designed to measure students grasp of basic themes and concepts in philosophy. The same instrument was administered both at the start and end of the semester. Combining Fall and Spring, 447 students took the pre-test while 340 took the post-test. 322 students took both tests. The pre-test measure was an average of 6.94 items answered correctly while the post-test measure was 11.39. The improvement of 4.45 items was statistically significant with a p-value less than .01.
Finding
PHL 263 Findings
In both the fall and spring semesters of the 2008-2009 academic year student progress was measured using a pre-test post-test model. The test was developed by the Philosophy Program faculty to test general knowledge that we would expect students completing PHL 263 to have acquired. This instrument consists of 25 items designed to measure students grasp of basic themes and concepts in philosophy. The same instrument was administered both at the start and end of the semester. Combining Fall and Spring, 392 students took the pre-test while 386 took the post-test. 359 students took both tests. The pre-test measure was an average of 11.4 items answered correctly while the post-test measure was 14.9. The improvement of 3.5 items was statistically significant with a p-value less than .05.
Action
PHL 261/263 Action
The Philosophy Program was pleased that both PHL 261 and PHL 263 both met our expectations for student learning outcomes. However, the assessment procedures for both courses must be enhanced to allow for more detailed analysis of target areas for future improvements in these courses. As such, the Program will change its assessment of these courses for 2009-2010 and moving forward to allow for analysis of individual questions for targeted improvement. All faculty will be asked to record the number of students who correctly answered each question on the pre- and post-tests. This will allow the Program to identify particular areas for growth and/or improvement.
Dr. Sanford will create a spreadsheet template for all PHL faculty members to follow. It will indicate the data that they are required to report for each section. Once it is approved, the template will be distributed to all PHL faculty by the Program Coordinator prior to the start of each semester to ensure that data is collected and reported in a format that allows the Philosophy Program to measure its progress. The template will be distributed to the PHL faculty prior to 8/1/09.
Objective
Demonstrate Advanced Understanding Of History Of Philosophy
Well-educated philosophy students must be able to appreciate the arguments and positions of earlier thinkers. Because so much of what is written in philosophy is a reaction to the metaphysical and epistemological presuppositions of earlier thinking, it is the core of well-rounded philosophical education.
Indicator
Pre-test Post-test Response Scores On Locally-Standardized Instruments (364/365)
All students in PHL 364 and PHL 365 will be tested on their knowledge of general concepts in the history of philosophy. All faculty who teach these courses will administer a pre-test and post-test to all students.
All Philosophy BA students are required to take PHL 364 (Ancient and Medieval Philosophy) and PHL 365 (Modern Philosophy). Together, these courses provide students with upper-level instruction covering the history of metaphysics and epistemology.
Following a review of best practices for the teaching of these courses, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a well-rounded understanding of the history of philosophy.
The attached documents provide the assessment instruments for PHL 364 and PHL 365 as well as the credited responses for each.
Criterion
PHL 364 Assessment
At least 75% of students will answer at least 8 of the 10 questions correctly on the post-test.
Finding
PHL 364 Findings
Data for each question is contained in the attached PDF. 75% of students correctly answered 9 of the 10 questions. The one spot of concern was that only 53% of students correctly answered question 1. This was the lowest success rate of all questions and the smallest improvement on any of the questions.
Criterion
PHL 365 Improvement
A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test. In addition, at least 70% of students will correctly answer 70% of the questions on the post-test.
Finding
PHL 365 Findings
In the spring 2009 semester student progress was measured using a pre-test post-test model. On the pre-test, the average student correctly answered 1.63 questions. On the post-test, the average student correctly answered 15.46 of the questions. The improvement of 13.83 items was statistically significant with a p-value less than .05. Data for student responses is contained in the attached files. 70.8% of students correctly answered at least 70% of the post-test questions.
Action
PHL 364 Action
Despite successfully meeting the criterion for success with respect to PHL 364 educating students concerning advanced understanding in history of philosophy, the performance of students on question 1 raised concerns. The question was, "Define soul; and cite a practical, theoretical, or religious consequence of that definition." In response to this, Dr. Bradley, who teaches the course, submitted the following plan of action that was endorsed by the Philosophy Program faculty:
Regarding the part of the question regarding the practical, theoretical, or religious consequences of that definition, I shall emphasize, with more explicit lectures, more compare/contrast elements relative to the parts of the question: i.e., "practically", how Aristotle's inclusion of the body in the definition of soul leads to the definition of moral virtue as the habitual mastery of the passions; "theoretically" how Aristotle's inclusion of the body in the definition of soul suggests an implicit criticism of the seemingly rank dualism of Pythagoras (and perhaps Plato, and some in the modern world, such as Descartes); and "religiously" how Aristotle's inclusion of the body in the definition of soul was one of many points crucial for Aquinas' incorporation, as it were, of Aristotle in Jewish-Christian doctrine, especially regarding the resurrection of the body.
Action
PHL 365 Action
The extent to which PHL 365 exceeded the pre-test/ post-test criterion for success was evidence that the Program set the criterion too low. Although the Program was successful at getting 70% of students to correctly answer 70% of the questions on the post-test, the manner in which the data was reported did not allow for analysis of the particular questions. In contrast, the assessment of PHL 364 provides a model for improving the assessment of PHL 365. During 2009-2010, PHL 365 data will be collected and reported in a manner that allows for a question-by-question analysis to identify areas for improvement. The assessment and subsequent actions for PHL 364 during 2008-2009 will serve as a model for this process. This will allow the Program faculty to identify not only general student learning outcomes, but also provide the data required to identify specific areas for improvement.
Dr. Sanford will create a spreadsheet template for all PHL faculty members to follow. It will indicate the data that they are required to report for each section. Once it is approved, the template will be distributed to all PHL faculty by the Program Coordinator prior to the start of each semester to ensure that data is collected and reported in a format that allows the Philosophy Program to measure its progress. The template will be distributed to the PHL faculty prior to 8/1/09.