GRADUATE NYC! Academy for Leaders in the Field of College - - PDF document

▶
graduate nyc
SMART_READER_LITE
LIVE PREVIEW

GRADUATE NYC! Academy for Leaders in the Field of College - - PDF document

Nov 09, 2022 583 likes •668 views

Graduate NYC! Academy for Leaders in the Field 5/23/2012 of College Transition GRADUATE NYC! Academy for Leaders in the Field of College Transition Graduate NYC! Academy for Leaders in the Field of College Transition 2 COLLECTING PARTICIPANT

slide-1
SLIDE 1

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 1

GRADUATE NYC!

Academy for Leaders in the Field of College Transition

COLLECTING PARTICIPANT OPINION DATA: SURVEYS, INTERVIEWS, AND FOCUS GROUPS PUBLICLY AVAILABLE DATA SETS

Stefanie Bruno

Coordinator of Research & Evaluation CUNY Collaborative Programs

Adenike Huggins

Policy & Research Analyst CUNY School Support/Research & Evaluation May 24, 2012

2 Graduate NYC! Academy for Leaders in the Field of College Transition

What Is The Best Way to Collect the Information?

3 Graduate NYC! Academy for Leaders in the Field of College Transition

Method Purpose Advantage Challenges Surveys Quickly and/or easily get lots of information from people in a non- threatening way Easy to compare and analyze; can get lots

  • f data; many sample

questions already exist; standardized Might not get careful in-depth feedback; Need to pretest questions; requires ability to read; might be hard to deliver Face-to-Face Interviews Fully understand an individual’s opinions or experiences, or learn more about their answers to surveys Get full range and depth of information; can be flexible Time-consuming; hard to analyze and compare data; can be costly; interviewer can bias responses Focus Groups Explore a topic in depth through group discussion Efficient way to get a range and depth of information in a short time; new perspective may emerge Need skilled moderator; have to listen to a lot of different/overlapping voices; responses bias; expensive

slide-2
SLIDE 2

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 2 What Is a Survey?

  • A method of gathering information from a number of

individuals, a “sample”, in order to learn something about the larger population from which the sample has been drawn.

  • Provide a speedy and cost effective means of learning

about people’s knowledge, attitudes, beliefs, and behavior.

4 Graduate NYC! Academy for Leaders in the Field of College Transition

Pros and Cons of Conducting Surveys

Surveys: Advantages

  • Easy to administer to a wide sample
  • Could assess changes in attitudes and behavior over time
  • Provide an efficient summary of opinions (if the population is

sampled appropriately)

  • Highly structured, which reduces bias
  • Easy to analyze

Surveys: Disadvantages

  • Yields mostly surface-level reactions
  • Difficult to ensure a representative sample respond

5 Graduate NYC! Academy for Leaders in the Field of College Transition

Data That Can and Cannot be Obtained From Surveys

Data that CAN be obtained:

  • Demographics
  • Desires
  • Expectations
  • Opinions
  • Perceived knowledge
  • Practices
  • Reactions

Data that CANNOT be obtained:

  • Actual behavior and knowledge

6 Graduate NYC! Academy for Leaders in the Field of College Transition

slide-3
SLIDE 3

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 3

Things to Consider Before Conducting a Survey

  • What is the purpose of the survey, what are you interested in

learning?

  • Who is the best source for the data?
  • What is the best way to reach your target participants?
  • Who will be administering the survey?
  • Who will be affected by the results of your survey?
  • Who is the audience?
  • How will the results be used?

7 Graduate NYC! Academy for Leaders in the Field of College Transition

Survey Creation

  • Include an introduction or cover letter and instructions
  • Start with an interesting/important question
  • Logical flow of questions:
  • Group similar questions
  • Smooth transitions
  • Ideal length
  • Leave white space
  • Demographics
  • Closing

8 Graduate NYC! Academy for Leaders in the Field of College Transition

Survey Creation

  • Question Writing Tips
  • Include relevant decision-makers in the question design process
  • Keep the survey and individual questions short
  • Make sure that all of the questions are relevant
  • Use simple and direct language
  • Be neutral
  • Ask for one piece of information at a time
  • Balance rating scales
  • Put the response options in a logical order
  • If possible, pre-test the survey

9 Graduate NYC! Academy for Leaders in the Field of College Transition

slide-4
SLIDE 4

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 4 What Is Wrong With This Question?

1.

I got along well with the teacher and students in my class.

2.

Have you heard back yet from your top choice college?

3.

I have a strong understanding of the physical sciences.

4.

Do you think the AHC math class is better than the other high school math classes you have taken?

5.

What is your current age?

a)

15-18

b)

18-20

c)

20-22

10 Graduate NYC! Academy for Leaders in the Field of College Transition

Target Sample Size

  • How many survey respondents do I need?
  • Survey purpose
  • Desired confidence level
  • Margin of error (confidence interval)
  • Total population size
  • http://www.custominsight.com/articles/random-sample-

calculator.asp

11 Graduate NYC! Academy for Leaders in the Field of College Transition

Target Sample Size

Example Survey Question: Please check whether you agree or disagree with the following statement: “I am aware of various college entrance requirements.” __ Agree __ Disagree Let’s say 75% of those asked said “agree.” Analysis at confidence interval 5: We can say with 95% certainty that between 70 and 80 % (75 plus/minus 5) of the population in the school agree that they are aware of various college entrance requirements. Analysis at confidence interval 2: We can say with 95% certainty that between 73 and 77 % (75 plus/minus 2) of the school agree that they are aware of various college entrance requirements.

Graduate NYC! Academy for Leaders in the Field of College Transition 12

slide-5
SLIDE 5

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 5 Maximizing Response Rate

  • Build a relationship with respondents
  • Include a personalized cover letter
  • Provide survey instructions
  • Interesting/relevant survey topic
  • Logical survey design
  • Send reminders
  • Offer a gift or incentive
  • Offer to share a summary of the results
  • Include your contact information
  • Use a professional reply email address
  • Indicate how long the survey takes to complete and a cutoff date
  • Timing

13 Graduate NYC! Academy for Leaders in the Field of College Transition

Types of Surveys and Administration Methods

  • Who administers the survey?
  • How could this impact the results?
  • How much staff will be needed?

14 Graduate NYC! Academy for Leaders in the Field of College Transition

Type Administration Tools Paper-and-pencil Mail, face-to-face, classroom, doorstep MS Word, Remark OMR Electronic Email, website link, classroom Survey Monkey, Poll Everywhere Verbal Telephone, face-to- face MS Word, CATI (Computer-Assisted Telephone Interviewing) Software

Next Workshop: Survey Analysis

  • E.g. of what will be covered

15 Graduate NYC! Academy for Leaders in the Field of College Transition

34% 52% 56% 43% 58% 85% 0% 20% 40% 60% 80% 100%

% of Students Who Claim to Know About How to Get Financial Aid for College

2011 2012 Current 11th Grade Cohort Current 10th Grade Cohort n=504 n=607 n=1099 n=908 n=607 n=556 Current 12th Grade Cohort

slide-6
SLIDE 6

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 6 Interviews

Activity: Pair up with someone else in the room and interview each other.

  • Interview topic: college-readiness
  • Time allotted for activity: 10 minutes

16 Graduate NYC! Academy for Leaders in the Field of College Transition

Interviews

Tips

  • Start with a non-threatening question to build rapport
  • Stay silent to give the respondent time to think
  • Try to avoid questions that lead to one word responses
  • Break down a complex question into multiple questions
  • Ask questions about the intensity of a feeling
  • Ensure you strike a balance of positive/negative

questions

  • Actively listen to the respondent

Graduate NYC! Academy for Leaders in the Field of College Transition 17

Focus Groups

Selecting Participants

  • Groups should consist of people with similar

experiences so that they feel comfortable talking with

  • ne another
  • Strive for 6-10 participants per group
  • It is ok to select based on the purpose of the groups

rather than strive for a representative sample

18 Graduate NYC! Academy for Leaders in the Field of College Transition

slide-7
SLIDE 7

Graduate NYC! Academy for Leaders in the Field

  • f College Transition

5/23/2012 7 Focus Groups

Deciding how many participants are needed:

  • Budget
  • Variety of relevant characteristics
  • Conduct as many groups as necessary until no new

information is revealed

Graduate NYC! Academy for Leaders in the Field of College Transition 19

Focus Groups

Conducting the session

  • Schedule 1-11/2 hours
  • Try to map out how much time you want to spend on each

question

  • Write 8-12 main questions with probes
  • Use a skilled moderator and assistant
  • Select a comfortable setting that is easily accessible
  • Audio record (after obtaining the proper permission)

Graduate NYC! Academy for Leaders in the Field of College Transition 20