Social Science Research Tyler Portis Design Considerations Meg - - PowerPoint PPT Presentation

▶
social science research
SMART_READER_LITE
LIVE PREVIEW

Social Science Research Tyler Portis Design Considerations Meg - - PowerPoint PPT Presentation

Sep 29, 2022 129 likes •346 views

Office of Institutional Research Social Science Research Tyler Portis Design Considerations Meg Nyce Jimmy Huynh Larry Long Alexander Yin Social Science Research Design 1. De ve lop your Re se ar c h Que stion(s) 2. Se le c t Re se ar

slide-1
SLIDE 1

Social Science Research Design Considerations

Office of Institutional Research Tyler Portis Meg Nyce Jimmy Huynh Larry Long Alexander Yin

slide-2
SLIDE 2

Social Science Research Design

1. De ve lop your Re se ar c h Que stion(s) 2. Se le c t Re se ar c h Appr

  • ac h

3. De te r mine Re c r uitme nt / Sampling Str ate gy 4. De ve lop Instr ume nts & Pr

  • toc ol

5. Conside r E thic al Implic ations & Biase s 6. Colle c t Data 7. Analyze and Communic ate Re sults

2

slide-3
SLIDE 3

Research Questions Research Approach Collecting Data Analyzing Data

Social Science Research Design

All re se a rc h sta rts with re se a rc h q ue stio ns tha t se t the me tho do lo g y a nd a na lysis. Se le c t a ppro pria te me tho do lo g y to a nswe r q ue stio ns (i.e . surve ys, fo c us g ro ups, inte rvie ws, me ta -a na lysis). E xe c ute re se a rc h a ppro a c h to c o lle c t da ta . Ana lyze re sults to dra w c o nc lusio ns tha t a nswe rs the re se a rc h q ue stio ns.

3

slide-4
SLIDE 4

Developing Research Questions

  • T

he re se a rc h q ue stio n is the first ste p a nd ne e ds to b e de ve lo pe d a t the b e g inning o f the pro c e ss. T he da ta yo u c o lle c t sho uld b e dire c tly re la te d to yo ur re se a rc h q ue stio n.

  • Re se a rc h q ue stio ns sho uld b e c le a r, fo c use d, a nd inte re sting .
  • Yo ur re se a rc h q ue stio n will dic ta te yo ur re se a rc h a ppro a c h (i.e . fo c us g ro ups, surve ys,

a rtic le a na lysis).

  • 6 Go lde n Rule s fo r Re se a rc h Ob je c tive s (https:/ / www.lipma nhe a rne .c o m/ ho w-to -d e fine -

g o o d-re se a rc h-o b je c tive s/ )

4

slide-5
SLIDE 5

Examples

  • Wha t is the e xpe r

ie nc e o f Ho no rs Co lle g e stude nts? (inte rvie ws, fo c us g ro ups)

  • Wha t is the diffe r

e nc e in pe rso na l de ve lo pme nt skills o f intra mura l spo rts pa rtic ipa nts a nd

no n-pa rtic ipa nts? (surve y)

  • Wha t is the r

e lationship b e twe e n se x a nd hig h-risk drinking ? (surve y, inte rvie w, fo c us

g ro up)

5

slide-6
SLIDE 6

Interviews vs Focus Group vs Surveys

Interviews Focus Groups Surveys Pros

  • Provide structure through guided

facilitation and one-on-one time

  • Removes potential bias from
  • ther participants that may occur

within a focus group

  • Are flexible in method (i.e.

telephone, in-person, online)

  • Provide qualitative data
  • Give flexibility to dive deeper into

participant responses

  • Provide quantitative data
  • May provide more candid

responses since respondents complete survey in a more private, anonymous setting

  • Reach more people

simultaneously Cons

  • Can be susceptible to open-

endedness (and lack of generalizability in analysis)

  • Require careful consideration and

training of observers to avoid biasing participants

  • Are time consuming
  • Lack of anonymity within group

may cause participants to not speak up on issues

  • May provide skewed data due to
  • utspoken participants who

dominate group discussion

  • Require more investment of time

and effort in planning, recruiting, and facilitating groups

  • Cannot ask follow-up questions

to responses

  • Do not always give respondents a

way to make comments on matters not directly asked

  • Can yield low response rates (i.e.

survey fatigue) - thus requires proper planning to recruit participants 6

slide-7
SLIDE 7

Determine Recruitment / Sampling Strategy

T hing s to c o nside r o nc e yo u’ ve se le c te d a re se a rc h a ppro a c h (thing s yo u ha ve to c o nside r fo r I RB):

  • Ho w will yo u re c ruit pe o ple / g e t pe o ple to pa rtic ipa te ?
  • Ho w will yo u g e t e ma ils if yo u ne e d to surve y pe o ple ?
  • Ho w a re yo u g o ing to pro te c t c o nfide ntia lity / a no nymity?
  • Do yo u ne e d multiple pe o ple to run a fo c us g ro up? And ho w will yo u e limina te / minimize b ia s?
  • Ho w muc h mo ne y a re yo u willing to spe nd to re c ruit pe o ple ?
  • Ho w / whe re will yo u sto re yo ur da ta ?

7

slide-8
SLIDE 8

Why send survey to only a sample size instead of population?

  • Po pula tio n re fe rs to a ll me mb e rs in a g ro up o f inte re st in a study
  • Sa mple re fe rs to a sma lle r sub se t o f the po pula tio n tha t the study will b e imple me nte d o n
  • Sa mpling :
  • Re duc e s c o st o f da ta c o lle c tio n (Po pula tio n studie s re q uire o ne to g e t a re spo nse fro m

e ve ryo ne in the de fine d po pula tio n)

  • Must b e c a re ful o f b ia s (i.e . a re the pe o ple re spo nding to the surve y re pre se nta tive o f the

po pula tio n? )

  • T

his vide o pro vide s mo re info rma tio n a b o ut po pula tio n a nd sa mple s

  • https:/ / yo utu.b e / e I

ZD1BF fw8E

8

slide-9
SLIDE 9

Sample Types & Sizes

Sample T ype s

Clic k the b utto n to g e t a n o nline re so urc e to c a lc ula te sa mple size . Note , not

e ve r yone will r e spond to your sur ve y, so you may ne e d to adjust your sample size to ac c ount for r e sponse r ate s.

Sample Size s

https:/ / yo utu.b e / pT uj57uXWlk

9

slide-10
SLIDE 10

Good Survey Design/Questions – Likert Questions

Likert questions enable for respondent to rate their

  • pinion about something as
  • pposed to eliminating it to a

black and white answer

10

slide-11
SLIDE 11

Good Survey Design/Questions – Skip Logic

Skip logic is when the designed survey skips survey questions that do not apply to the respondent. If we have a set of questions about living on campus, we want students who live

  • ff-campus to not have to answer them.

11

slide-12
SLIDE 12

Good Survey Design/Questions – Avoid Leading Questions

Image source: https://aea365.org/blog/objection-thats-a- leading-survey-question-by-carla-hillerns-by-carla-hillerns/.

A leading question is a question that suggests the answer the research is looking to have

  • confirmed. Avoid these questions

as they can create untrustworthy data.

12

slide-13
SLIDE 13

Survey Research Resource (Considered the Source for Survey Research)

13

slide-14
SLIDE 14

Preparing for Observations, Interviews, and Focus Groups

  • Ma ke sure yo u ha ve the pro pe r supplie s (i.e . no te pa d, pe ns, ta pe re c o rde r, c o nse nt fo rms,

inte rvie w pro to c o l)

  • Pla n yo ur inte rvie w q ue stio ns a nd de te rmine the struc ture o f yo ur inte rvie w
  • De te rmine if yo ur inte rvie w is fully-struc ture d o r se mi-struc ture d
  • F

ully-struc ture d inte rvie ws a re stric t a nd fo llo w a sc ript

  • Se mi-struc ture d inte rvie ws a llo w fo r fo llo w-up q ue stio ns o r a dditio na l pro mpts b ut still fo llo w

so me fo rm o f struc ture

  • Whe n writing inte rvie w q ue stio ns, g o fo r q ue stio ns tha t e lic it a tho ro ug h re spo nse

(i.e . a vo id ye s/ no re spo nse s)

14

slide-15
SLIDE 15

Consider Ethical Implications & Bias

  • Pro te c t the c o nfide ntia lity o f yo ur pa rtic ipa nts (i.e . use pse udo nyms fo r q ua lita tive studie s a nd a g g re g a te

da ta fo r q ua ntita tive studie s)

  • Cra ft yo ur q ue stio ns a nd da ta c o lle c tio n struc ture in a wa y tha t pa rtic ipa nts c a n a nswe r ho ne stly a nd
  • pe nly
  • I

f yo ur to pic is b a nkruptc y, c o nside r ho w yo u will g e t yo ur pa rtic ipa nts to fe e l c o mfo rta b le a nswe ring yo ur q ue stio ns.

  • Co lle c t yo ur da ta in a wa y tha t is se c ure a nd do e s no t ide ntify yo ur pa rtic ipa nts
  • Co nside r if yo u a re re q uire d to ha ve a c o unse lo r a t ha nd de pe nding o n yo ur inte rvie w q ue stio ns o r to pic
  • Be c o nsiste nt with ho w yo u c o lle c t yo ur da ta to a vo id b ia s.
  • I

f yo u ha ve two inte rvie we rs, ho w a re yo u ta king no te s fo r e a c h a nswe r? Are yo u tra nsc rib ing o r inte rpre ting a nswe rs?

15

slide-16
SLIDE 16

Strategies to Increase Response Rates

  • Ma ke yo ur a udie nc e fe e l spe c ia l b y e xpre ssing g e nuine a ppre c ia tio n fo r the ir

pa rtic ipa tio n.

  • “We wa nt to kno w wha t yo u think.”
  • Pro vide inc e ntive s whe n po ssib le
  • Gift c a rds, e ntry into ra ffle s, a nd o the r prize s a re e ffe c tive in g e tting re spo nde nts to c o mple te

surve ys.

  • Ma ke yo ur surve y a s sho rt a s po ssib le

16

slide-17
SLIDE 17

Collecting Data

  • F
  • llo w yo ur e sta b lishe d pro to c o l fo r c o lle c ting da ta .

17

slide-18
SLIDE 18

Analyzing Survey Data

  • Be fo re vie wing re sults, g o b a c k to yo ur ma jo r re se a rc h o b je c tive s a nd write a n o utline fo r

yo ur a na lysis to a nswe r tho se q ue stio ns.

  • Cle a n a nd tra nsfo rm the da ta a s ne c e ssa ry
  • Use da ta te c hno lo g y like Mic ro so ft E

xc e l, SPSS, o r SAS to a nswe r the q ue stio ns yo u ha ve .

  • E

.g . Ca lc ula te me a ns, c ro ss-ta b ula te da ta , a nd c o nduc t t-te sts b a se d o n yo ur re se a rc h q ue stio n

  • Dra w c o nc lusio ns a b o ut the da ta .
  • Mo re info rma tio n o n a na lyzing re sults c a n b e fo und a t

https:/ / g e tthe ma tic .c o m/ insig hts/ a na lyze -surve y-da ta -surve y-a na lysis/ .

18

slide-19
SLIDE 19

Analyzing Interview / Focus Group Data

  • Be fo re vie wing re sults, g o b a c k to yo ur ma jo r re se a rc h o b je c tive s a nd write a n o utline fo r

yo ur a na lysis to a nswe r tho se q ue stio ns.

  • T

ra nsc rib e the inte rvie w re spo nse s

  • I

de ntify the me s within the re spo nse s (Co de re spo nse s in so ftwa re suc h a s NVI VO, De duc e , e tc .)

  • Dra w c o nc lusio ns a b o ut the da ta .
  • Cro ss-c he c k (tria ng ula te ) yo ur finding s with o the r so urc e s (i.e . e xisting re se a rc h, fa c ulty

a dviso rs, e tc .)

19

slide-20
SLIDE 20

Institutional Review Board (IRB) Approval

I RB a ppro va l is ne e de d fo r re se a rc h studie s. F

  • r the se purpo se s,

re se a rc h is de fine d a s "a syste ma tic inve stig a tio n, inc luding re se a rc h de ve lo pme nt, te sting a nd e va lua tio n, de sig ne d to de ve lo p o r c o ntrib ute to g e ne ra liza b le kno wle dg e .

Use UVM’s Se lf- De te r mination T

  • ol to assist in de te r

mining if the pr

  • je c t r

e quir e s IRB r e vie w. If you ne e d appr

  • val, the we bsite will

dir e c t you on how to c omple te the ne c e ssar y IRB pape r wor k.

20

slide-21
SLIDE 21

Note about IRB Approval:

  • I

RB is a b o ut pro te c ting the rig hts o f huma n pa rtic ipa nts; ho we ve r e ve n if yo u ha ve a n institutio n’ s I RB a ppro va l, it do e s no t me a n tha t the institutio n will suppo rt the re se a rc h.

  • I

f yo u a re re q ue sting institutio na l da ta tha t is no t pub lic ly a c c e ssib le (e .g ., e ma ils fo r surve ys), yo u sho uld c o nta c t the Offic e o f I nstitutio na l Re se a rc h (https:/ / www.uvm.e du/ o ir/ c o nta c t-us) fo r c o nsulta tio n.

21