Improving Critical Thinking And Analytic Reasoning
Students completing the critical thinking and logic courses in our curriculum will develop a broad-based skills in critical thinking and formal logic.
Objective
Demonstrate Critical Thinking Skills
Critical thinking skills are an essential component of philosophical work. The ability to analyze arguments and draw conclusions from available information serves students in practically every aspect of life.
Indicator
Response Scores On TACTS
All students who take PHL 262 will be tested on their critical thinking skills. All faculty who teach PHL 262 will administer the Texas Assessment of Critical Thinking Skill (TACTS)s, an externally validated test of critical thinking skills, in a pre-test/post-test format. The TACTS is a broad-based assessment of critical thinking skills that goes beyond the current scope of PHL 262. This will allow the faculty to determine areas that may be added to our current curriculum in the future. A copy of the current TACTS is attached. A copy of the credited responses is attached. The Philosophy Program Coordinator, currently Dr. Fair, will be responsible for ensuring that all faculty who teach PHL 262 effectively administer the pre- and post-tests in every section of their course. One faculty member, currently Dr. Sanford, will be responsible for gathering pre- and post-test data from the faculty members who teach PHL 262.
Criterion
Statistically Significant Improvement From The TACTS Pre-test To The TACTS Post-test.
A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. The philosophy program expects to see a statistically significant improvement from the pre-test to the post-test.
Finding
Significant Improvement From Pre-test To Post-test
A paired two-sample t-test on our sample of 387 student scores, for students who took both the pre-test and post-test, demonstrated a statistically significant improvement in their scores. Data and basic analysis are attached.
Criterion
Improvement In Calculating Probabilities
The data from the team that developed the TACTS show that a knowledge gap exists with respect to decision making when an outcome depends upon the conjunction of two probabilistic events. Their data show that less than 20% of those tested correctly answered the following question:
"George is waiting for two of his customers, Fuzzy Logic Computers, Inc. and Stalking Horse Designs, to pay their bills. If either of them pays before the end of the month, then George can pay his supplier. But if neither of them pays, then George will have to take out a bank loan. George estimates that the chance that Fuzzy Logic will pay in time is 70% and the chance Stalking Horse Designs will pay in time is 60%. Assuming that his estimates are correct and that the two events are independent, what is the chance that George will have to take out a bank loan? (a) 12% (b) 40% (c) 65% (d) 42% (e) 88%""
2009-2010 will be the first year that the Philosophy Program will expect all faculty to evaluate this type of reasoning as part of the critical thinking course. We will consider this initial effort successful if there is at least a 50% improvement on this type of question from the pre-test to the post-test.
Finding
Improvement In Calculating Probabilities
The sample is limited to only those students who took both the pre-test and post-test. For students whose pre-test score was reported, 56 out of 293 (19.1%) correctly answered the question. On the post-test, 103 out of 293 (35.6%) correctly answered the question. This represents a 83.9% improvement and remains a challenge to improve even further. The sample for this finding is smaller than the general pre-test/post-test finding, because all faculty reported individual question data from the pre-test and post-test.
Action
Ongoing Improvement In Calculating Probabilities
Although our data showed a substantial improvement in calculating probabilities, it is expected that our faculty can improve on our first-year performance. All Philosophy Program faculty who taught PHL 262 during 2009-2010, will be invited to share their experiences with those who will teach PHL 262 during 2010-2011. In addition, input will be sought from other faculty who have experience teaching probability-based content. The goal will be to identify ways in which pedagogy in this area can be improved without cutting back on the successful core of critical thinking skills we have regularly taught. It is expected that faculty will implement and evaluate new approaches during 2010-2011.
Objective
Demonstrate Competence With Formal Logic
Student's understanding of formal logic provides a foundation for rigorous analysis of arguments. All philosophy majors will have the opportunity to acquire competences in propositional logic and predicate logic.
Indicator
Response Scores On Embedded Logic Questions
All students who take PHL 362 (Introduction to Contemporary Logic) will be tested on knowledge of basic logical concepts, propositional logic, and predicate logic. All faculty who teach PHL 362 will administer a series of embedded questions through the semester and as part of the final examination. These will serve as an indicator of student competence with the subject matter. Following a review of best practices for the teaching of this course, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a an introduction to the concepts of formal logic, propositional logic, and predicate logic. The attached documents provide the sample questions similar to those used in the embedded assessments and the credited responses. Because some of these questions, particularly the logical proofs, have answers that could be memorized if the questions were known in advance, exam security requires that the actual questions not be posted in a public document. The Program faculty will use questions that are essentially similar but sufficiently distinct to limit the potential for academic dishonesty. A group of Program faculty has reviewed the particular questions that will be embedded each semester.
Criterion
Correct Responses On 70% Of Embedded Questions By 70% Of Students
At least 70% of students will correctly answer 70% of the embedded questions.
Finding
PHL 362 Students' Responses To Embedded Questions.
9 out of 12 (75%) of the students for whom data is available correctly answered 70% of the embedded questions. This data represents only the students taking PHL 362 in Fall 2009. A change in instructor for Spring 2010 combined with a failure to communicate the proper assessment strategy prevented data from being reported in Spring 2010.
Action
Ongoing Assessment Of PHL 362
The small enrollment PHL 362 and the failure to gather data for Spring 2010 meant that the sample size for the 2009-2010 evaluation was extremely small. Although the sample indicated that the program appears to be meeting its goal, we must continue to improve our evaluation process to ensure that data is available for identifying areas for future improvement. Additionally, instructors will need to provide more working examples of concepts of formal logic, propositional logic, and predicate logic to ensure the learning outcome.
Goal
Understanding Of General Philosophical Concepts
Students progressing through the introduction and history of philosophy courses in our curriculum will develop a broad-based knowledge of metaphysics, epistemology, and moral theory. These areas form the basis for a general competence in philosophy. Given the small size of the Philosophy Program, it will focus on providing all students with a basic background in each area. In addition, the Program will seek to ensure advanced understanding in the history of metaphysics and epistemology.
Objective
Demonstrate Basic Understanding Of Core Concepts In Philosophy
As students progress through the Philosophy BA, they will acquire a basic understanding of metaphysics, epistemology, and moral theory. This basic information, provided by our introductory courses serves as the foundation for student success in upper-division courses.
Indicator
Statistically Significant Improvement Of Student Scores From Pre-test To Post-test (261/263)
All students in PHL 261 and PHL 263 will be tested on their knowledge of basic concepts in metaphysics, epistemology, and moral theory using a locally standardized pre-test and post-test for each course. Following a review of best practices for the teaching of these courses, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a basic competence in metaphysics, epistemology, and moral theory. The attached documents provide the assessment instruments for PHL 261 and PHL 263 as well as the credited responses for each.
Criterion
Statistically Significant Improvement From The Pre-test To The Post-test (261/263)
A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test.
Finding
PHL 261 Pre-test/Post-test Results
PHL 261 students demonstrated a statistically significant improvement from the pre-test (M=7.9) to the post-test (M=10.1). Although statistically significant, the post-test mean is only 51% with 20 items. See the attached data. Thus, more innovative strategies for teaching basic concepts in metaphysics, epistemology, and moral theory are needed.
Finding
PHL 263 Pre-test/Post-test Results
PHL 263 students demonstrated a statistically significant improvement from the pre-test (M=12.53) to the post-test (15.22). While statiscally significant, the post-test mean is only 62% of 25 items. See the attached data. Hence, basic concepts in metaphysics, epistemology, and moral theory must be addressed in inventive ways so that deep learning versus shallow learning is accomplished.
Action
PHL 261/PHL 263 Evaluation Improvements
We are satisfied that our students are making good general progress in PHL 261 and 263; however, there must be an effort to identify specific areas for improvement. As such, the Philosophy Program will undertake an item analysis of the PHL 261 and PHL 263 post-tests during 2010-2011 for this purpose. This will provide even more specific data. Still, it is obvious that we must be more innovative in our teaching of basic concepts in epistemology, metaphysics, and moral theory.
Objective
Demonstrate Advanced Understanding Of History Of Philosophy
Well-educated philosophy students must be able to appreciate the arguments and positions of earlier thinkers. Because so much of what is written in philosophy is a reaction to the metaphysical and epistemological presuppositions of earlier thinking, it is the core of well-rounded philosophical education.
Indicator
Pre-test Post-test Response Scores On Locally-Standardized Instruments (364/365)
All students in PHL 364 and PHL 365 will be tested on their knowledge of general concepts in the history of philosophy. All faculty who teach these courses will administer a pre-test and post-test to all students. All Philosophy BA students are required to take PHL 364 (Ancient and Medieval Philosophy) and PHL 365 (Modern Philosophy). Together, these courses provide students with upper-level instruction covering the history of metaphysics and epistemology. Following a review of best practices for the teaching of these courses, a group of Program faculty chose the questions for the assessment. The questions cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a well-rounded understanding of the history of philosophy. The attached documents provide the assessment instruments for PHL 364 and PHL 365 as well as the credited responses for each.
Criterion
PHL 365 Assessment
A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test. In addition, at least 70% of students will correctly answer 70% of the questions on the post-test.
Finding
PHL 365 Pre-test/Post-test Results
Students demonstrated a statistically significant improvement from the pre-test (M=0.86) to the post-test (M=14.09). Although statistically significant, post-test mean is only 70% with 20 items. This still represents a weakness in teaching the history of metaphysics and epistemology in modern philosophy.
Finding
General Student Performance In PHL 365
16 out of 22 students (72.7%) correctly answered at least 70% of the questions on the post-test.
Criterion
PHL 364 Assessment
At least 75% of students will answer at least 8 of the 10 questions correctly on the post-test. Additionally, at least 75% of students will correctly respond to the prompt, ""Define soul; and cite a practical, theoretical, or religious consequence of that definition.""
Finding
PHL 364 Pre-test/Post-test Results
Students demonstrated a statistically significant improvement from the pre-test (0.71) to the post-test (17.71). This is a vast improvement with the mean being 89% of the items answered correctly. Still, it is important to maintain this level of learning outcome and even work to increase in pre-modern philosophy.
Finding
General Student Performance In PHL 364
14 out of 16 (87.5%) of the students correctly answered at least 8 of the 10 questions correctly.
Finding
PHL 364 Student Understanding Of The Concept Of Soul Finding
11 out of 16 (68.75%) of students correctly responded to the prompt "Define soul; and cite a practical, theoretical, or religious consequence of that definition." Although this did not reach the stated goal of 75% correct responses, it does represent an improvement from the 53% that correctly answered the question during 2008-2009. Still, though improved, it is still unacceptable and more effort will be made to promote deep learning of defining the soul and the consequences of said definition.
Action
PHL 364 Improvement Target
PHL 364 met two of the three stated criteria during 2009-2010 and showed considerable improvement on the third. Prior to offering the course in Fall 2010, there will be a review of the material and its presentation aimed at continuing the trend toward successful student outcomes regarding the history of pre-modern philosophy.
Action
PHL 365 Improvement Target
PHL 365 met both of the criteria for 2009-2010. Nonetheless, the instructor will undertake a review of the course in an effort to identify ways of ensuring student performance continues to meet the program's goals regarding the history of modern philosophy.