1 2 volunteer departments per institution per As expected, - - PDF document

1

Report from a project funded by the Teagle Foundation to Beloit, Knox, Monmouth, Ripon Marion Fass, Kevin Hastings, Frank Gersich, Robin Woods

 “[R]ather than divide the undergraduate experience into separate camps, general versus specialized education, the curriculum of a college of quality will bring the two together.” Ernest Boyer, 2001  We proposed a collaborative project to examine this complementary relationship between general education and the major, and, in particular, to establish methods of assessment that allow us to learn about, improve, and strengthen the achievement of liberal learning goals by all of our students, irrespective of the major they choose.

 We wanted to develop a project that put

faculty members first in assessment

 Faculty members would define their own

goals

 Develop their own measures of assessment  Maintain autonomy (Free-market perspective)  But within the larger context of institutional

mission

 We know that both students and faculty have

distinct interests and skills

 What happens after General Education courses?  How does course work in majors both support

growth in selected areas AND reinforce all college goals?

 How do faculty members define, teach to and

assess these goals?

Serial model of education Parallel model of liberal arts- is it intentional? Do all graduates have the same skills? Is that our goal?

2

 2 volunteer departments per institution per

year

 How departments address and reinforce all

college goals

• CRITICAL THINKING
• CIVIC ENGAGEMENT
• QUANTITATIVE REASONING

 This process started with defining the

• utcomes

 As expected, quantitative reasoning in

Philosophy differs from that in Mathematics, but what about critical thinking or civic engagement?

• Do we define these in our departments?
• Do we structure our curricula or advising to

reinforce them?

• How do we (do we?) measure what we do?

 We will discuss our experiences, and identify

issues that are important in our ongoing efforts at assessment.

 What do we know about what we do on

each campus?

• NSSE
• Collegiate Learning Assessment

 What do we want to share?

• Start with sharing on campus

 How do these results relate to our

departments? 2 departments per college per year 2 departments per college per year

 Each department developed and carried

• ut a plan, but not all the same

 Process takes more time than anticipated  Strategies

• Focus groups
• Student surveys
• Review of student work and transcripts
• Links to NSSE, CLA
• Embedded assessments

Handout identifies how each department identified and measured progress on goals

Institution Institution Year One Year One Year Two Year Two Beloit Philosophy/ Religion Anthropology Biology Psychology Knox Mathematics English Education Economics Ripon Physics Sociology/ Anthropology French Economics Monmouth Art Biology Communications and Theater Arts Accounting  Faculty “owned” within context of mission

and goals

• Build upon disciplinary definitions
• Encouraged consideration of “out of discipline”

skills, such as Quantitative Literacy for the English major

• Turned focus on curricular connections to

community through civic engagement

 Departments defined engagement and community

 Faculty at dept level identify how to use this

information to improve student academic experiences

3

At the heart of the educational process Critical thinking asks students to engage actively in their

• wn learning

 All see it as a central part of what we do:

• “We believe that by its nature, anthropology is

about critical thinking. Cultural anthropology seeks to make the strange familiar and the familiar strange.” (Beloit Anthropology)

 But possibly because it is so central, some

departments had trouble defining or even focusing on it:

 “This is the area where we found the most assumptions

and took the most for granted, given that we believe this concept to be at the very heart of every single thing toward which the English Department strives.” (Knox)

 Like assessment itself, something we often

prefer to take for granted.

 The challenge is to look at the issue

consciously—that is, critically.

 Most definitions include two essential

components:

• Perception: seeing the subject clearly
• Analysis: questioning the subject

 Models for critical thinking:

• Open model, usually focusing on process
• Discipline-specific model
• “We are committed to the idea that thinking

critically involves being able to recapitulate a line

• f reasoning and engage in dialogue, raising
• bjections and concerns, to see if that reasoning

approaches truth.” (Philosophy Dept, Beloit)

• “Drawing on the work of other scholars….the

sociology department defines critical thinking as thinking that requires students to evaluate and interpret evidence, identify and evaluate relevant arguments, develop and defend a position on an issue, and/or consider all sides of a question or issue without bias.” (Sociology Dept, Ripon)

 “Critical thinking in Physics is centered on

the investigation of physical systems…The student must be able to determine quantitative parameters that describe a system and the accuracy of those parameters.” (Physics, Ripon)

4

 *Direct measures of student learning:

embedded assignments, writing/portfolio analysis

 *Indirect measures: Student surveys and

evaluations, student focus groups, transcript analysis, course evaluations Indirect: Senior exit survey asks students to evaluate how coursework has improved critical thinking skills, then asks about specific critical thinking skills: analyzing arguments, challenging assumptions, etc. Direct: Portfolio analysis from different levels

• f student coursework, reading for

coherence, critical distance, sophistication

 Portfolio review, including student reflection

assignment with prompts that require multiple thinking skills

 Part I: Consider the courses you have taken this year (art and non

• art):

 Explain (i.e. describe and justify) the significant learning /accomplishments  Compare (i.e. describe and analyze) what you did accomplish  Determine (i.e. describe and relate) how ideas/content from your non-art courses can and do influence your art-making.

 Part II: Consider your personal development as an art-maker:

 Compare (i.e. describe and analyze) your struggles with your successes.  Formulate goals and strategies for continuing your own development that build on your strengths, achievements and successes and encounter your challenges.  Students write papers applying two

competing theories to a single case.

• describe, apply, evaluate effectiveness

 Results assessed according to rubric:

• Two theories are discussed
• Discussion draws heavily from text
• Description in own words
• Quotes used sparingly and appropriately
• Theories are

 applied to the case in question  used correctly and not out of context  synthesized or integrated with one another

 Beloit Philosophy Department:

• Mixed results: some get it, some don’t
• Frustration with small number of complete

portfolios

 Resulting action:

• Capstone course doubled in work and credit load
• Students encouraged to meet with project advisor

earlier

• Faculty begin discussing and modeling research

process earlier in the term

 Changes are recent, but students are already

showing improvement.

5

• global competitiveness
• essential to logical thinking
• more information available to

more people; less reliance on experts  Movement grew in the mid-late 1990’s with

falling test scores and calls for accountability

• Innumeracy: Mathematical Illiteracy and its

Consequences, John Allen Paulos (1988)

• Principles and Standards for School Mathematics,

NCTM (2000)

• Mathematics and Democracy: The Case for

Quantitative Literacy, National Council on Education and the Disciplines (2001)

• Current Practices in Quantitative Literacy, MAA

(2006)

From mission statement of National Numeracy Network: “QL is the power and habit of mind to search out quantitative information, critique it, reflect upon it, and apply it in our public, personal, and professional lives.” Refining the definition:

 Interpret mathematical models such as formulas,

graphs, tables, and schematics, and draw inferences from them.

 Represent mathematical information symbolically,

visually, numerically, and verbally.

 Use arithmetical, algebraic, geometric and

statistical methods to solve problems.

 Estimate and check answers to mathematical

problems in order to determine reasonableness, identify alternatives, and select optimal results.

 Recognize that mathematical and statistical

methods have limits.

6

 As long as you abstract from the specific to

the general by representing a system or process symbolically (which is abstracting), setting down rules for it and manipulating that symbol system within those rules, it is

• quantitative. (Chemistry reaction equations?

Logic? Language syntax?)

Examples Examples

• Ripon Physics Department

Ripon Physics Department: Focus on particular skill:

• vectors. Course-based, before and after measures
• n regular quizzes and final
• Monmouth Art Department

Monmouth Art Department: Assignment in Senior Seminar course on Living as an Artist

• Beloit Philosophy Department

Beloit Philosophy Department: Transcript analysis

• f quantitative courses taken, survey on
• advisement. Note connection to logic
• Knox English Department

Knox English Department: Transcript analysis. Possible connection to prosody and structure of language

• Knox Math Department

Knox Math Department: Quantitative Literacy quiz given to liberal arts math, Calculus II, sophomore bridge course, and mathematical finance course

 Most of the departments involved relied on

• ther departments to teach quantitative

skills.

 They did not directly assess student skills.  Exceptions: Monmouth Art, Knox

Mathematics, Beloit Biology, Beloit Psychology, Monmouth Accounting

While there are many career options available for art majors, working as a professional artist is often the first option. How much art do you have to produce to make a living. Part 1 – Supporting Yourself with Your Art

• 1. Would you sell your work through a gallery or an agent?
• 2. Estimate how much a single artwork (or freelance g/d

project) might sell for:

• 3. Subtract 50% for gallery:

Or 20% for agent: This is your gross income per artwork.

• 4. Calculate 15% of the gross for taxes:
• 5. Subtract this amount from the gross:
• 6. Calculate 10% of the gross for materials:
• 7. Subtract this amount from line 5: This equals your net

income per artwork.

• 8. How many artworks will you sell in a month? (Consider this

a goal.)

• 1. One definition of inflation is ``the persistent rise in the

general price level as measured against a standard level of purchasing power.'' In short, because of inflation a dollar tomorrow is generally worth less than a dollar today. One common measure of inflation is the Consumer Price Index (CPI), which measures ``changes in the prices paid by urban consumers for a representative basket of goods and services.'' The table below shows the December-to- December percentage increase in CPI for several recent years: Year Year % Increase in CPI % Increase in CPI 2002 2.4% 2003 1.9% 2004 3.3% 2005 3.4% Suppose that you earned $30,000 per year in the year

• 2002. How much would you need to earn in 2006 to have

the same ``purchasing power'' as you did in 2002? A homeowner needs to replace a concrete driveway whose shape is given below: The volume of concrete to be delivered to a customer is measured in units of cubic yards. (three feet by three feet by three feet). Currently, each cubic yard of concrete costs about $75. If concrete in the driveway is to be poured to a depth of four inches, what will be the cost of the concrete needed to replace the current driveway?

16 feet 30 feet 8 feet 14 feet 12 feet

7

 °

Quantitative Literacy Quiz Quantitative Literacy Quiz (3)

There are serious concerns about the link between exposure to plutonium and cancer incidence rates, so scientists have developed an index which measures a person's exposure to

• plutonium. In a study of cancer rates in communities near

a plutonium processing plant scientists compared this exposure index with cancer mortality rates (in deaths per 100,000 residents) and found the following data:

If you had to predict the cancer mortality rate for an exposure index of 5.0, approximately what it be?

Exposure index Cancer rate

Results – Knox Mathematics Results – Knox Mathematics

• A total of 86 students returned the quiz, including

53 students in Mathematical Ideas, 20 calculus- level students, and 13 major-level students. The average score on the exam was 2.895. The overall distribution of scores among the 86 students was:

Results – by course level Results – by course level

• Liberal Arts Math: average 2.51;
• Calculus average 3.15;
• Mathematics major -average 4.08.

 The exercise of defining quantitative

literacy began the revision of the MATH 121 course, which is still in progress. The quiz was also administered at the end of that course, but little improvement was shown.

 We reviewed question-by-question

• performance. We believe that problem-

solving skills are the principle deficiency. Improvement of real problem solving should be based on application and synthesis.

 A wide campus discussion is necessary to

reinforce the need for this outcome, and to decide what it means. Different departments will have different perspectives.

 This is one of the most measurable

educational goals, so student work samples are necessary.

 The assessment method should go through its

• wn assessment cycle to establish validity.

 The college must identify who is

responsible in order to assure follow-up.

 Will math departments will take leadership?

Improvements won’t be maximized without broader buy-in.

8

Civic Engagement

• Broadly defined: preparing students

to be involved in their community

– Formal curriculum – Co-curriculum – Volunteer activities – Student/Faculty research

• Participating departments “wrestled”

with definition for study

College Goals Include

• Beloit  One of seven curricular goals is: “Active,

responsible citizenship.”

• Knox  One of four curricular goals is: “applying

classroom learning through hands-on experience.”

• Monmouth  General Education Capstone course:

Citizenship (informed, conscientious action)

• Ripon  Communicating Plus Program supports

engagement in issues confronting citizens of the 21st century.

 Level in formal curriculum may be dependent upon

resources

 Departmental commitment may be dependent upon

faculty priorities and campus priorities

 CE outside of formal curriculum is often assessed

by reported participation levels

 CE within formal curriculum may best be assessed

by “guided” reflections

 included CE as part of departmental mission –

focused on community service- 1

 discussed role of preparing students to act

(generally through ethics/values discussions)- 6

 indicated a need to incorporate into future

assessments-2

 collected information about student involvement in

activities outside of departmental courses-1

Ripon Physics Department

Students voluntarily incorporated civic engagement component in class projects assessed by tabulating number Physics Fun Force – student group which travels to local elementary schools assessed by popularity among students and response from schools

9

Beloit Philosophy Department

“Careful and considered” actions vs. actions Assessed by Transcript analysis Senior exit questionnaire Writing Portfolio- critical thinking about civic engagement

Ripon’s Romance and Classical Languages Department

Transcript analysis Electives to provide background for becoming well-informed citizens and active community members

Ripon Physics – require civic engagement component in application projects Beloit Philosophy

• Post-advising debriefings
• Colloquium renovation
• Increase opportunities for informal student-faculty

engagement

Ripon’s Romance and Classical Languages –revised advising process to include discussion of student interests and suggestions for pursuing those interests Monmouth Art – continue to use instruments developed in study and developed Citizenship course “Creating Change Through Art”

Conclusions and recommendations

 We all agree that these are important goals,

but we don’t agree on how they are defined

 There are legitimate differences between

disciplines

Someone who thinks critically as a biologist may differ from someone who thinks critically as a philosopher

• By giving students “majors” on their diplomas we

acknowledge this

 These findings can catalyze important all

college discussions about shared values, essential skills and disciplinary interpretation

10

 Perhaps this isn’t the right question.  We can all agree that we hope to graduate

liberally educated students, but we need to acknowledge that what this means may differ by disciplinary (or interdisciplinary) focus

 Grassroots assessments can be valuable in

aiding faculty to structure and strengthen curriculum

 Departments are at different levels of readiness

• Personalities, changes in discipline, demographics
• Departments recognize need for periodic retreats to

revisit goals and courses articulation- to be more intentional

 Departments that depend on the “kindness of

• thers” to address skills needed by majors need

to sit down and talk about expectations

 Process can be adapted as less intensive

mini-reviews, but still effective (Monmouth)

 Workshops for departmental missions/ goals

(Ripon)

 Annual focus on a particular goal (Beloit)  “Closing the loop” is an ongoing process

• Acknowledge the problem
• Communicate
• Develop courses and procedures

 Smallness doesn’t preclude need to

coordinate curriculum

 “Friendly” departments may not always

discuss educational issues

 Because of desire for flexibility in scheduling,

students don’t always take courses in sequence-

 This increases need for communication about

goals and how they are reinforced in curriculum

Our college graduate with liberal arts skills framed by disciplinary

• rientations

Student Educational Experience

 Thanks to the Teagle Foundation for

supporting this work, to ACM FaCE, Lake Forest College and Rachel Ragland, and to AAC&U

 Thanks to administrators and colleagues at

Beloit, Knox, Monmouth and Ripon Colleges, particularly those colleagues who have reviewed their work and let us share it with you.