Learning Good Employee Skills: Maximizing Internship Program - - PowerPoint PPT Presentation

Learning Good Employee Skills: Maximizing Internship Program Effectiveness

NSEE Annual Conference, September 2016 Abby Trout, Career Center Carol Trosset, Institutional Research and Assessment Carleton College

1 Institutional Research and Assessment

Overview

Carleton College = a highly selective liberal arts college in Minnesota, with 2000 undergraduates.

• Introduce the Carleton College Career Center’s

internship process and its focus on learning goals

• Review Trosset’s research at Bennington College on

which skills are most valued by interns’ employers

• Present new research at Carleton on how students

become aware of and learn these skills

• Discuss implications for Career Center programming

Institutional Research and Assessment 2

Institutional Research and Assessment 3

Career Center mission:

Institutional Research and Assessment 4

Carleton Career Center learning goals:

• Self-assessment
• Career field awareness
• Transferable skills
• Market selves
• Experience
• Job search
• Graduate degrees
• Access to networks
• Effective networking

Institutional Research and Assessment 5

Career Tracks

Institutional Research and Assessment 6

Internships at Carleton

• 488 (33%) of rising Carleton sophomores, juniors, and seniors

did internships in Summer 2016

• 113 of these internships were funded through the Career

Center

• Career Center provides funding to cover expenses (i.e.,

housing, transportation, food)

• $386K awarded for Summer 2016
• Internships located all over the world (all over US and 29

countries) and in a variety of industries

Institutional Research and Assessment 7

What makes for a rewarding internship?

• A strong relationship with a supervisor
• Learning
• Goals
• Self-discovery

At Carleton, students specify learning goals:

Institutional Research and Assessment

Research on Learning Outcomes

Institutional Research and Assessment 9

• What are the actual learning outcomes?
• What are the most important/beneficial outcomes,

that internships should foster?

• How can we assess (a) the degree to which students

achieve these outcomes, and (b) the quality of their performance as interns?

• Can we identify how beneficial learning took place, so

we can design experiences that maximize these effects for other students?

Bennington College Research: Assessing Internship Performance

Liberal arts college in southwest Vermont 700 undergraduates 7-week internship required every winter All students complete 4 internships Research in 2013-14 and 2014-15 Holly McCormack, Dean of Field Work Term

Institutional Research and Assessment 10

What are internships for?

• Students want to explore possible career paths, and to start

building a resume that will lead to jobs after college

• Chronicle of Higher Education and NPR’s Marketplace 2012

study “The Role of Higher Education in Career Development: Employer Perceptions” found that

–Employers place more weight on experience, particularly internships and employment during school vs. academic credentials including GPA and college major when evaluating a recent graduate for employment. –Among all industry segments, an internship is the single most important credential for recent college graduates to have on their resume. –60% of employers practice intern-to-permanent hiring.

Institutional Research and Assessment 11

Liberal Arts Learning vs. Job Training

• AACU 2013 study “It Takes More than a Major: Employer

Priorities for College Learning and Student Success” found that employers endorsed a blended model of liberal and applied learning—practices that involve students in active, effortful work.

• Bennington Field Work Term’s goal: Students will apply their

academic learning in a broader context outside the classroom.

• Same AACU study asked employers whether they want

employees to have the skills liberal arts colleges already value, like writing and critical thinking. Employers said “yes.”

Institutional Research and Assessment 12

But what do interns need to learn?

Supervisors of Bennington students wrote open-ended evaluations, listing interns’ strengths and areas needing improvement. What did they write about without being prompted?

• “She was always on time and willing to step in and help with anything.”
• “He asked great questions and responded to feedback gracefully.”
• “She could pay more attention to her body language during meetings and

while working. At times she seemed a bit distracted and disinterested.”

• “He tended to drift off task to check Facebook.”

Institutional Research and Assessment 13

Categories employers mentioned:

Institutional Research and Assessment 14

• Work ethic
• Engagement with work
• Quality of work
• Organization/efficiency
• Punctuality
• Professional conduct
• Takes direction
• Teamwork
• Interpersonal skills
• Independence/initiative
• Learns quickly
• Confidence
• Creativity
• Writing skills
• Critical thinking skills
• Other job skills

Note: Most of these are neither liberal arts skills nor job-specific skills. Instead, they are qualities of any good employee in any job sector.

Qualities of a good student?

(taken from Bennington Faculty narrative evaluations)

• Work ethic – “works hard,” “diligent,” “spends time texting during class”
• Engagement – “excellent work when focused but not consistently engaged”
• Quality – “does high quality work,” “should focus on producing high-quality work

the first time around”

• Organization – “sometimes unprepared,” “talented but disorganized”
• Punctuality – “late to class several times,” “some absences”
• Takes direction – “responded to feedback,” “needs to learn to follow directions”
• Teamwork – “listens and contributes well in discussion,” “good team member”
• Initiative – “challenges himself,” “does the minimum to get by”
• Learns quickly – “learns from mistakes,” “learns quickly”
• Confidence – “should have more confidence,” “wish would speak up more”

Institutional Research and Assessment 15

Evaluating internship performance

Institutional Research and Assessment

Unacceptable Needs Improvement Good Exceptional Focused work ethic Too laid-back, easily distracted, takes frequent breaks Mostly works hard, but some tendency to lose focus and drift off task Productive and conscientious Works harder than

• thers and is much

more productive Engagement with the work Seems unmotivated, avoids or resists unwanted tasks Engagement and motivation vary with the task Engaged in the work, willing to help as needed Seizes every

• pportunity to

learn as much as possible Organization and Efficiency Inefficient, forgets about assignments, may not follow through to completion Still learning how to manage time and priorities when working alone Good time management, follows up on assigned tasks Unusually efficient, intuitive sense of priorities Takes direction Fails to incorporate feedback into work products; rarely checks in with supervisor Some tendency to put own agenda ahead of others’ needs Checks in regularly, responds appropriately to feedback Frequently seeks

• ut feedback and

uses it to improve the work

16

Benefits of the rubric:

• All employers are asked to comment on a consistent group of skills
• Students have a consistent definition of what the College means by

workplace readiness

• Students can:

–compare their self-rating using the same rubric to that of their employers –track their rubric ratings over four years –use employer evaluations as part of a job application packet

• Employers have asked us for copies of the rubric to use with their staff

Institutional Research and Assessment 17

Challenges for using rubrics with employers:

• Colleges need to compare the students in aggregate, and over

time.

• But students all have different jobs and employers.
• The students start at different stages of development.
• The employers have different expectations.
• We cannot norm the employers to consistent standards.
• The evaluation process cannot be blind or unbiased.

Institutional Research and Assessment 18

Carleton College Research: Employee Skills Metacognition

Focus was on tracking and increasing student awareness of these skills. 90 of the funded interns in Summer 2016 allowed us to use their materials for this research. Materials included: Application essays Blog posts Learning contracts Reflective essays Rachel Leatham, Program Director

Institutional Research and Assessment 19

20

Student Motives and Preparation

Motives

Support the mission – 44% Interest in activity – 39% Personal connection – 19% Learn skills – 17% Explore a career – 13% Apply academics – 9% Broaden my horizons – 6%

Preparation

Coursework – 52% Previous “job” – 39% Specific skills – 33% Studying the organization – 17% Off-campus study – 10% Local knowledge – 10% Extracurricular activity – 9%

Institutional Research and Assessment

Students’ Learning Goals

• Explore a possible career – 73%
• Gain academic learning – 47%
• Learn job-specific skills – 41%
• Improve research skills – 22%
• Improve communication skills – 13%
• Improve time management skills – 13%
• Improve foreign language skills – 12%
• Gain cross-cultural competency – 11%
• Improve interpersonal interaction skills – 11%
• Gain life skills – 7%

32% had only job-specific or academic goals.

Institutional Research and Assessment 21

Learning Outcomes from Final Reflections

Institutional Research and Assessment 22

Goal Outcome Goal Outcome Work ethic 3 Takes direction 1 12 Engagement 6 Teamwork 8 9 Quality of work 9 Interpersonal skills 1 7 Time management 12 10 Initiative 4 Flexibility 1 6 Confidence 6 Professionalism 3 3 Communication skills 12 13

What They Learned about Employee Skills

Work Ethic

• The importance of persistence
• How much work is required to develop a good product
• How to time breaks to be more productive

Engagement

• That feeling engaged with one’s work improves its quality
• That personal growth happens when you commit yourself to

something

Institutional Research and Assessment 23

What They Learned, cont.

Quality of Work

• The importance of doing high-quality work
• Attention to detail
• The value of being well-prepared

Time Management

• Improved time management skills
• Became better organized
• The importance of managing one’s time
• How to set priorities

Institutional Research and Assessment 24

What They Learned, cont.

Flexibility

• The importance of flexibility
• The value of patience
• That being flexible can improve the quality of the work

Taking Direction

• The importance of asking questions
• The importance of taking notes
• How to accept critique

Institutional Research and Assessment 25

What They Learned, cont.

Teamwork

• The importance of teamwork
• The importance of learning from co-workers
• Teamwork and facilitation skills
• Project management and leadership experience

Interpersonal Skills

• The value of interacting and building relationships with

co-workers

Institutional Research and Assessment 26

Those with a goal more likely to learn; Most who learned did not have that goal.

Institutional Research and Assessment 27

0% 10% 20% 30% 40% 50% 60% Learned target skill Learned different skill No employee skills reported With Goal No Goal

What leads to the learning when it’s not motivated by a goal?

• Specific on-the-job events
• The difference between a job and a class
• Directed reflection

Institutional Research and Assessment 28

Specific events

• “One of the biggest lessons I learned early in my internship was to not be afraid to

ask for help, or be inexperienced in any particular skill. The first project I was assigned involved using Excel. Initially I was frustrated and embarrassed that it took me longer than expected to learn how to use that program and that I didn’t know how to use it before. However, as I reflected with my supervisor about this anecdote at the end of my internship, she was surprised that I didn’t ask her for help rather than dealing with it on my own. At the time I didn’t know that was an

• ption, I believed the expectation was to already be equipped to complete the

projects assigned.”

• “Staying organized was something that I struggled with sometimes. My supervisor

would send me an e-mail of something that would be coming up in two weeks and because it was two weeks away I’d say, ‘Well, I’ll read it next week when I have more time.’ I would then lose my e-mail in my constantly overflowing inbox and I’d have to ask my supervisor to resend it. Early on, at meetings I wouldn’t take notes, thinking my memory would be good enough, but when it became clear that the little nuances and wordings mattered just as much as the main idea we were trying to convey I realized that I had to start taking some notes or become lost.”

Institutional Research and Assessment 29

Events, cont.

• “My supervisor was an extremely busy woman – as a result she usually sent tasks

to me and my two fellow interns and left it up to us to work together or divvy up the work as we thought best. This was an important exercise in teamwork: I learned the importance of giving credit where credit was due was essential for a successful team. Even small things like saying ‘Ashley and I edited this report’ rather than ‘here’s the report you wanted edited’ goes a long way in creating a healthy group dynamic.”

• “I’ve learned to keep my collected data clean. Although this project had been

going on for three years now, there was no standardized manner of storing or analyzing the data. Thus, the last two weeks of my internship were largely spent going through thousands of haphazardly sorted photos, essentially doing other interns’ work for a second time. This was incredibly frustrating. Acknowledging that I have been lax in storing my data neatly during past projects, I learned the hard way that it pays to stay organized from the beginning!”

Institutional Research and Assessment 30

Difference between a job and a class

• “Through this project I developed my willingness to pay attention to detail as I

encountered many specific comments of constructive criticism, a type of conversation I rarely have in an academic setting as little of my work ever reaches beyond my professor grading a specific assignment. This experience helped me understand the kind of work put into public education initiatives and the importance of detail-oriented review.”

• “Usually I was very engaged in my work because I had to be extremely attentive

when processing information and working on my projects. In the field that I am interning for, mistakes must be recognized and fixed before it is sent in.”

• “Here, giving up means letting over 40 children down. Giving up means sending

the message to those children that they aren’t worth it. Giving up meant impacting someone else’s life just as much as my own. In school, giving up in a class, for example, only really affects me. I guess in other words I learned a greater sense of responsibility, because I have wanted to just quit so many times.”

Institutional Research and Assessment 31

Directed Reflection via Weekly Blog Prompts

• Observe others’ work ethic
• How you stay engaged with the work
• What you’ve learned by making a mistake
• Standards of punctuality
• Observed nature of professional conduct
• Difference between a supervisor and a professor
• How people work in teams
• Relevance of interpersonal skills
• Working independently and taking initiative
• Accomplishments and skill building
• Observe effective communication

Institutional Research and Assessment 32

Among those who reflected on learning non- goal employee skills, 3/4 wrote about employee skills in response to summer blog prompts.

• “The skill that I’m most proud of is my growing sense of flexibility. I tend to be a

rather stubborn person, and I don’t like to change things or let on when I am lost. This internship, however, has really challenged me to have to change this. Every time I think that I am finally done with creating my stimuli, my supervisor thinks of something I can change. My original plans and expectations for my experiments have certainly changed a lot over the course of the past five weeks, and I think that’s absolutely awesome. Though it’s certainly frustrating to feel as if I’m not making progress at times, it’s really cool seeing just how much of a process experiment design is.”

• “Even doing boring tasks (sharpening pencils, printing tickets, folding programs), I

am surrounded by friendly and engaging people. Many of my bosses are on teams that put on shows here, so seeing their shows serves as motivation and a reminder that grunt work is almost always necessary.”

Institutional Research and Assessment 33

Supervisor Evaluations

Institutional Research and Assessment 34

Positive Improvement Needs Work Work ethic 22 2 Engagement 10 1 Quality work 9 2 1 Time management 6 2 1 Flexibility 5 3 Professionalism 3 1 Take direction 8 2 2 Teamwork 6 7 2 Interpersonal 15 3 4 Initiative 11 5 2 Confidence 2 12 4 Communication 5 10

Differential Salience

• Students start out knowing the importance of time

management and communication skills.

• They become aware of the importance of asking

questions, attention to detail, and relationships with co-workers.

• They remain unaware of most dimensions of

professionalism (other than dress code).

Institutional Research and Assessment 35

Next Steps

• Conduct focus groups with some of the 2016 interns about

what facilitated this learning, and how to get other students to see its importance.

• Summer Interns 2017

–Require one of the three learning goals to be a generalizable skill. – Continue to adjust blog prompts to include features of employee skills we know students are encountering, and that they should think about. – Find ways to expand their awareness of the less salient skills (like professionalism). –Use metrics to show incentive to other interns to register with our

• ffice

Institutional Research and Assessment 36

Next Steps

• Broader application of specific findings and the usage of

structured reflection pre-, during, and post-experience to

• ther preparation programs

–Post-grad fellows –On-campus jobs and interns –Externships

• Structure alumni-student programming

–Build alumni profile questions around employee skills prompts –30 Minutes program questions

Institutional Research and Assessment 37

Questions?