E ssa y Que stio n De ve lo pme nt & Asse ssme nt Wor kshops - - PowerPoint PPT Presentation

e ssa y que stio n de ve lo pme nt
SMART_READER_LITE
LIVE PREVIEW

E ssa y Que stio n De ve lo pme nt & Asse ssme nt Wor kshops - - PowerPoint PPT Presentation

10/9/2014 E ssa y Que stio n De ve lo pme nt & Asse ssme nt Wor kshops - Oc t. 10, 2014 E ast AM Ana lytic T hinking 10 a m - no o n E ssa y Que stio n De ve lo pme nt - 8-144 (2 PD) We st PM 3 pm - 5 pm E ssa y Que stio n De ve


slide-1
SLIDE 1

10/9/2014 1

E ssa y Que stio n De ve lo pme nt & Ana lytic T hinking

L AURA BL ASI , PH.D., DIRE CT OR OF INST I T UT I ONAL ASSE SSME NT OCT . 10, 2014

Asse ssme nt Wor kshops - Oc t. 10, 2014 E ast AM

10 a m - no o n E ssa y Que stio n De ve lo pme nt - 8-144 (2 PD)

We st PM

3 pm - 5 pm E ssa y Que stio n De ve lo pme nt - 11-216 (2 PD)

* I ntro duc to ry Writing a nd Disc ussio n….

 Ple a se write o ne e ssa y q ue stio n spe c ific to yo ur

disc ipline – o f no mo re tha n fo ur se nte nc e s (yo u will no t ha ve to turn it in.)

 Whic h kind o f e xa m q ue stio ns do yo u pre fe r to c re a te

a nd why? (F

  • r e xa mple , e ssa y q ue stio ns, multiple

c ho ic e q ue stio ns MCQ, e tc .) Wha t do yo u ho pe to g e t

  • ut o f this wo rksho p….

A b it o f b a c kg ro und: c o mpo sitio n the o ry, pe rfo rma nc e

a sse ssme nt, c o mpute rize d g ra ding , g a me de sig n, UCF g ra dua te stude nts….

Purpo se

 T

his wo rksho p ha s b e e n de sig ne d to sha re re se a rc h finding s a nd e ffe c tive pra c tic e s fo r writing a nd g ra ding e ssa y e xa m q ue stio ns.

 We will pra c tic e c ritiq uing q ue stio ns a nd de ve lo ping o ur o wn.

We will a lso lo o k a t c o ntro ve rsia l issue s re la te d to this fo rm o f a sse ssme nt.

 T

he wo rksho p ha s b e e n de sig ne d a ro und the b lue b o o k Analytic T hinking b y Dr. L inda E lde r a nd Dr. Ric ha rd Pa ul (2010.) Pa rtic ipa nts sho uld b e a b le to a pply the sta nda rds the y pro vide to (1) de ve lo p e ffe c tive e ssa y q ue stio ns; (2) c ritiq ue e ssa y q ue stio ns; a nd (3) c re a te le a rning a c tivitie s to pre pa re stude nts fo r this kind o f a sse ssme nt.

slide-2
SLIDE 2

10/9/2014 2 L e a rning Outc o me s Pa rtic ipa nts sho uld b e a b le to ….

 Apply sta nda rds fo r a na lytic thinking to :

 Cre a te yo ur o wn e ssa y q ue stio ns  Critiq ue yo ur o wn e ssa y q ue stio ns  T

e a c h stude nts ho w to a nswe r e ssa y q ue stio ns  c ite ke y re se a rc h finding s re g a rding the impa c t o f e ssa y q ue stio ns o n stude nts’ thinking skills.  disting uish b e twe e n e ffe c tive a nd ine ffe c tive q ue stio ns.  de ve lo p a n a c tivity to te a c h stude nts ho w to ta c kle e ssa y q ue stio ns.  sha re five tips fo r mo re re lia b le a nd e ffe c tive g ra ding o f e ssa ys.  disc uss c urre nt inno va tio ns suc h a s c o mpute rize d g ra ding fo r e ssa y e xa ms.

L e a rning Outc o me s Pa rtic ipa nts sho uld b e a b le to ….

 Apply sta nda rds fo r a na lytic thinking to :

 Cre a te yo ur o wn e ssa y q ue stio ns  Critiq ue yo ur o wn e ssa y q ue stio ns  T

e a c h stude nts ho w to a nswe r e ssa y q ue stio ns  c ite ke y re se a rc h finding s re g a rding the impa c t o f e ssa y q ue stio ns o n stude nts’ thinking skills.  disting uish b e twe e n e ffe c tive a nd ine ffe c tive q ue stio ns.  de ve lo p a n a c tivity to te a c h stude nts ho w to ta c kle e ssa y q ue stio ns.  sha re se ve n tips fo r mo re re lia b le a nd e ffe c tive g ra ding o f e ssa ys.  disc uss c urre nt inno va tio ns suc h a s c o mpute rize d g ra ding fo r e ssa y e xa ms.

Co nstruc te d Re spo nse (E ssa ys) Que stio ns Re stric te d + E xte nde d Re spo nse

http:/ / www.e dc a n.o rg / pdf/ E dCa nF a c tShe e tE ssa ys.pdf

R e str ic te d r e sponse e ssa ys limit b o th

c o nte nt a nd re spo nse a s indic a te d within the q ue stio n. T he re stric te d re spo nse e ssa y a ddre sse s a limite d sa mple o f the c urric ulum o r le a rning

  • utc o me s.

Re stric te d Re spo nse

Ma y c o mmo nly b e kno wn a s:

Pr

  • ble m solving e xe r

c ise s fo c us o n so lving a pro b le m a nd de c isio n ma king

pro c e sse s. I n nursing c o nte xts, c linic a l da ta is pre se nte d a nd the le a rne r must de mo nstra te the ir a b ility to a sse ss, a na lyse , pla n, imple me nt a nd e va lua te .

Case studie s pre se nt c e rta in, b ut no t a ll, ke y e le me nts o f a c linic a l situa tio n

re se mb ling re a l-life sc e na rio s. T he le a rne r must c o mb ine the se s e le me nts with info rma tio n a c q uire d fro m pre vio us e duc a tio na l e xpe rie nc e s. With a dditio na l re so urc e s the le a rne r wo rks thro ug h a se q ue nc e o f inc re a sing ly c o mple x a c tivitie s. Othe r te rms ma y b e use d to de sc rib e the se a sse ssme nt me tho ds, the y inc lude sc e na rio -b a se d a c tivitie s o r c o nte xt- de pe nde nt ite m se ts.

slide-3
SLIDE 3

10/9/2014 3 E xte nde d re spo nse e ssa ys c a n b e c o nside re d a fo rm o f pe rfo rma nc e a sse ssme nt o r c o mple x a c hie ve me nt.

T he se e ssa ys pro vide the fre e do m o f re spo nse to a q ue stio n a nd a sse ss the a b ility to re se a rc h a to pic , c re a tive ly o rg a nize , inte g ra te a nd e va lua te ide a s, a nd c o nstruc t a n a rg ume nt.

Quiz….. * Bra insto rm…. Ho w to de c ide …

Wha t a re q ue stio ns we c a n a sk…. Do yo u ha ve a la rg e g ro up o f stude nts? Do yo u pla n to re use the e xa m? Are yo u c o nfide nt in yo ur a b ility to o b je c tive ly g ra de e ssa ys? Are yo u c o nfide nt in yo ur a b ility to write c le a r MCQ ite ms? Wha t a re yo ur inte nde d le a rning o utc o me s….

L e a rning Outc o me s + Blo o m’ s T a xo no my F rie nds o f the E ssa y Que stio n Write r

slide-4
SLIDE 4

10/9/2014 4

*A c he c klist o f q ue stio ns yo u

c a n a sk to ta rg e t inte lle c tua l sta nda rds… (T

he Blue Bo o k, p. 9)

1.

T hink o f ho w yo u wo uld use this list to de ve lo p e ssa y q ue stio ns with the g o a l

  • f a sking stude nts to a pply the ir

a na lytic a l thinking skills….

2.

L

  • o king a t yo ur prio r re fle c tive writing ,

se e if yo u c a n mo dify it b y a dding so me o f the q ue stio ns fro m this list.

Wha t if MCQs we re the o nly o ptio n…?

Wha t is g a ine d a nd wha t is lo st in a wo rld whe re ,

a s a stude nt, yo u c a n o nly ta ke MCQ e xa ms?

Wha t is g a ine d a nd wha t is lo st in a wo rld whe re ,

a s a n instruc to r , yo u c a n o nly c re a te MCQ?

Sa mple Pro mpt 1

Wha t c o uld we le a rn whe n re a ding the ir

re spo nse s [in te rms o f the ir a c a de mic skills]?

Wha t e lse wo uld yo u wa nt to se e de mo nstra te d

tha t this q ue stio n do e s no t g e t a t…

Wha t prio r [c la ssro o m] e xpe rie nc e s did the y

ne e d to ha ve in o rde r to b e a b le to re spo nd?

Wha t va lue s a re we c o mmunic a ting to the

stude nts thro ug h the kinds o f e ssa y q ue stio ns we a sk?

Sa mple Pro mpt 2

 Co mpa re a nd

c o ntra st with the prio r e ssa y q ue stio n

slide-5
SLIDE 5

10/9/2014 5

T he Re de sig ne d SAT 2016 – E ssa y Que stio ns

T

  • e va lua te

thinking we must a pply inte lle c tua l sta nda rds

(T he Blue Bo o k, p. 8)

We c a n a pply the se sta nda rds a s we e va lua te

  • ur o wn e ssa y q ue stio ns…

Pra c tic a l De sig n Co nside ra tio ns E xa mple Se nt to Me 1

Que stion:

Bo th Oe dipus a nd Othe llo sho w le a de rship q ua litie s, a g re a t se nse o f re spo nsib ility to the ir c o mmunity o r c o nstitue nts, a nd a n inc re dib le se nse o f g uilt fo r a c tio ns the y ma y no t ultima te ly ha ve b e e n re spo nsib le fo r. Write a n e ssa y a rg uing whic h o f the two wa s the mo re no b le c ha ra c te r. Use re fe re nc e s fro m te xt to suppo rt the a rg ume nt yo u ma ke . T ype yo ur e ssa y in the spa c e pro vide d. Yo u ha ve 4 ho urs to do the e xa m.

slide-6
SLIDE 6

10/9/2014 6 Whic h is b e tte r a nd why? Co mpa ra tive E xa mple s o f E ssa y Que stio ns - 1 Co mpa ra tive E xa mple s o f E ssa y Que stio ns - 2 Co mpa ra tive E xa mple s o f E ssa y Que stio ns - 3 E xa mple Se nt to Me 2

Hote l R

  • om R

ate s

 T

he Ho te l Orla ndo ha s 115 ro o ms, whic h a re ra re ly a ll full during the o ff se a so n. Ho te l re c o rds sho w tha t a t $46 pe r nig ht the y a ve ra g e 81 o c c upie d ro o ms during the o ff se a so n; a t $54 pe r nig ht 69 ro o ms a re o c c upie d, while a t $58 o nly 63 ro o ms a re fille d. While in a ma na g e r's me e ting , the o wne rs a sk yo u to a na lyze this da ta to de te rmine ho w ma ny ro o ms wo uld fill if the y c ha rg e d $40 pe r nig ht.

 Pa rt I Instruc tio ns: Using the da ta g ive n c re a te a ma the ma tic a l mo de l, a n

e q ua tio n tha t yo u c a n use to pro vide the o wne rs with the info rma tio n the y ha ve re q ue ste d. Be sure to a nswe r the ir q ue stio n a nd to c le a rly indic a te ho w yo u a rrive d a t tha t a nswe r. Sho w yo ur wo rk! (*E ve n if using a c a lc ula to r, sho w ho w a nd why yo u to o k the ste ps yo u did, no t just wha t b utto ns yo u pushe d.) Ma the ma tic a l mo de l: Numb e r o f ro o ms fille d if the y c ha rg e $40:

slide-7
SLIDE 7

10/9/2014 7 Disc ipline Spe c ific Co nside ra tio ns: Ma the ma tic s

 http :/ / www.e d uc a tionald e signe r.o rg/ e d /volume 1/issue2/ artic le6/ima ge s/figure_1_large .p ng

CRE SST Re se a rc he rs T a p into Stude nt T hinking – Ma th

1997 CSE

  • T

E CH493

 F

  • r mo re info rma tio n se e …

 CRE

SST Re po rt Mc Ne il Ma th Que stio ns T E CH449

Disc ipline Spe c ific Co nside ra tio ns: T he He a lth Sc ie nc e s

 Mo difie d e ssa y q ue stio n (ME

Q) is a use ful “pa pe r a nd pe nc il” instrume nt de sig ne d to a sse ss stude nts’ c linic a l re a so ning skill, a b ilitie s in pro b le m so lving , de c isio n ma king a nd unde rsta nding o f pro b le m-re la te d c o nte nt fro m the b a sic a nd c linic a l sc ie nc e s. Pro b le ms pre se nte d a re b a se d

  • n a n a c tua l c a se o r pro fe ssio na l inte ra c tio n situa tio n.

L e kha kula ,A. a nd Pina ikul, S. (ND)

http:/ / me d-e d.psu.ac .th/ we b/ me u/ re se arc h/ pdf/ ME Q.pdf

slide-8
SLIDE 8

10/9/2014 8 Do n’ t b e drive n b y the q ue stio n – b e g in with the c o mpe te nc ie s first… Wha t is a “ta b le o f spe c ific a tio ns? ”

“A T OS, so me time s c a lle d a te st b lue print, is a ta b le tha t he lps te a c he rs a lig n o b je c tive s, instruc tio n, a nd a sse ssme nt (e .g ., No ta r, Zue lke , Wilso n, & Yunke r, 2004). T his stra te g y c a n b e use d fo r a va rie ty o f a sse ssme nt me tho ds b ut is mo st c o mmo nly a sso c ia te d with c o nstruc ting tra ditio na l summa tive te sts.”

F ive s, He le nro se & DiDo na to -Ba rne s, Nic o le (2013). Cla ssro o m T e st Co nstruc tio n: T he Po we r o f a T a b le o f Spe c ific a tio ns. Pra c tic a l Asse ssme nt, Re se a rc h & E va lua tio n, 18(3). Ava ila b le o nline : http:/ / pa re o nline .ne t/ g e tvn.a sp? v=18&n=3

* T

a ke “Ana lyzing Pro b le ms” a nd a pply it a s a mo de l fo r writing a n e ssa y q ue stio n in yo ur disc ipline o r fie ld (T

he Blue Bo o k, p. 26)

F

  • r e xa mple o n tha t pa g e ta ke “… so me

impo rta nt a ssumptio ns” a nd a da pt the phra se to a sk yo ur stude nt “Wha t a re so me

  • f the a ssumptio ns tha t yo u…”
slide-9
SLIDE 9

10/9/2014 9

* T

e a c hing yo ur stude nts to “q ue stio n a n issue o r a c e ntra l pro b le m” p. 44

  • 1. Ho w c o uld this pa g e b e use d to te a c h stude nts

stra te g ie s fo r a nswe ring e ssa y q ue stio ns?

  • 2. Pro vide o ne e xa mple o f a n a c tivity o r

a ssig nme nt yo u mig ht de ve lo p fo r yo ur stude nts.

Re se a rc h: Stude nts Be ne fit fro m Pra c tic e Writing Que stio ns….

T he E ffe c ts of Stude nt-Wr itte n Que stions on Stude nt T e st Pe r for manc e

Pa ul W. F

  • o s F

lo rida Inte rna tio nal Unive rsity (1989) T he e ffe c ts o f stude nt-writte n te st q ue stio ns o n stude nt te st pe rfo rma nc e we re e xa mine d in a n Intro duc to ry Psyc ho lo g y c la ss. Be fo re e a c h o f thre e te sts, ra ndo mly a ssig ne d stude nts wro te e ssa y q ue stio ns, multiple c ho ic e q ue stio ns, o r no q ue stio ns. All te sts c o nta ine d e ssa y a nd multiple -c ho ic e ite ms b ut no q ue stio ns writte n b y stude nts. Que stio n write rs pe rfo rme d sig nific a ntly b e tte r tha n no n-write rs o n the first two te sts; the diffe re nc e o n the third te st wa s ma rg ina lly sig nific a nt. No diffe re nc e s we re fo und b e twe e n stude nts who wro te e ssa y a nd tho se who wro te multiple -c ho ic e q ue stio ns.

Que stion wr iting appe ar s to be an e ffe c tive study te c hnique .

Re se a rc h: Mo re De e p L e a rning Whe n Stude nts I nte g ra te T e xts in E ssa y Writing

T he E ffe c ts of T asks on Inte gr ating Infor mation F r

  • m Multiple Doc ume nts (2008)

Ra q ue l Ce rda ´ a nd E dua rdo Vida l-Ab a rc a , Ca tho lic Unive rsity o f Va le nc ia T he a utho rs e xa mine 2 issue s: (a ) ho w stude nts inte g ra te info rma tio n fro m multiple sc ie ntific do c ume nts to de sc rib e a nd e xpla in a physic a l phe no me no n tha t re pre se nts a sub se t o f the info rma tio n in the do c ume nts; a nd (b ) the ro le o f 2 so rts o f ta sks to a c hie ve this type o f inte g ra tio n, e ithe r writing a n e ssa y o n a q ue stio n re q uiring inte g ra tio n a c ro ss te xts o r a nswe ring sho rte r intra -te xt q ue stio ns tha t re q uire stude nts to inte g ra te info rma tio n within a sing le te xt, while supe rfic ia l a nd de e p c o mpre he nsio n me a sure me nts a re o b ta ine d.

Re sults showe d that the inte gr ation que stion inc r e ase d inte gr ation and de c r e ase d the pr

  • c e ssing of isolate d units of infor

mation, whic h e nhanc e d de e p le ar ning, whe re a s no

diffe re nc e s b e twe e n the 2 so rts o f ta sks o n me mo ry re c a ll we re a ppa re nt.

Ove ra ll E ssa y Writing Sho wn to Ha ve a n I mpa c t

An Inve stigation of the E ffe c ts of E xam E ssay Que stions

  • n Stude nt L

e ar ning in Unite d State s Histor y Sur ve y Classe s

Sa ra Bro o ks Sundb e rg Unive rsity o f Ce ntra l Misso uri T hre e hundre d sixty-five stude nts, who b o th the pre -te st a nd po st-te st. T he to ta l numb e r o f stude nts e nro lle d in the c o urse during the c o urse o f the study wa s five hundre d a nd two stude nts. T he thirte e n se c tio ns o f the c o urse tha t to o k b o th the pre te st a nd the po st-te st ra ng e d in o ve ra ll size fro m thirty-thre e stude nts to fo rty- nine stude nts in a se c tio n with a n a ve ra g e o f thirty-nine stude nts pe r se c tio n.'‘

E ssay wr iting on e xams made a diffe r e nc e in stude nts' sc or e s, and pote ntially the ir unde r standing, as r e fle c te d in the gr e ate r ne t gain of the post-te st ove r pr e -te st sc or e s for the nine se c tions in the fir st thr e e se me ste r s whe n e ssa y q ue stio ns we re

pa rt o f the e xa ms. During the se se me ste rs a ve ra g e sc o re s inc re a se d b e twe e n twe nty a nd thirty pe rc e nt. T he se re sults suppo rt the a sse rtio n tha t e ssa y writing impro ve s histo ry le a rning ; the g a in wa s sta tistic a lly sig nific a nt.

slide-10
SLIDE 10

10/9/2014 10 Ho w c a n we impro ve o ur q ue stio n de ve lo pme nt a nd imple me nta tio n?

* Write the Bo o k

Po o ling o ur c o lle c tive wisdo m…. … if we we re to write a b o o k to he lp

  • the rs g ra de e ssa y e xa ms wha t a re

five ke y po ints o r tips yo u wo uld wa nt to inc lude .

Guide line s fo r Gra ding E ssa y Que stio ns Mo re Re lia b ly – Be fo re Yo u Be g in

 Be fo re g ra ding , list the ma in po ints yo u

e xpe c t a g o o d a nswe r to c o ve r.

 De c ide in a dva nc e ho w yo u will ha ndle

fa c to rs suc h a s spe lling a nd g ra mma r, a nd a pply the rule s c o nsiste ntly.

 Be fo re g ra ding , re a d thro ug h a fe w sa mple

stude nt a nswe rs to g e t a g e ne ra l ide a o f the q ua lity le ve l.

Ste ps to T a ke ….

 T

  • c o unte ra c t the “ha lo ” e ffe c t, try to g ra de

a nswe rs witho ut kno wing the stude nt’ s ide ntity.

 Gra de o ne q ue stio n fo r a ll stude nts b e fo re

g o ing o n to the ne xt q ue stio n.

 I

f po ssib le , re a d e a c h a nswe r twic e , shuffling the o rde r the se c o nd time thro ug h.

 Re shuffle the pa pe rs a fte r c o mple ting e a c h

ite m.

slide-11
SLIDE 11

10/9/2014 11 Adde d Ste ps to T a ke ….

 So rt pa pe rs into “hig h,” “me dium,” a nd

“lo w” sta c ks b e fo re a ssig ning fina l g ra de s.

 Write c o mme nts so tha t stude nts unde rsta nd

why a nswe rs we re g o o d o r po o r.

 Mo re de ta ile d re a ding a b o ut the use o f rub ric s a nd

sc o ring re la te d to re lia b ility a nd va lidity a re a va ila b le

  • utside o f the sc o pe o f this wo rksho p Se e : E

xa mining De sig n a nd Inte r-Ra te r Re lia b ility o f a Rub ric Me a suring Re se a rc h Qua lity a c ro ss Multiple Disc ipline s: http:/ / pa re o nline .ne t/ pdf/ v14n12.pdf .)

Curre nt I ssue s a nd Po ints o f Co ntro ve rsy

 Co mpute rize d g ra ding  Ne w po ssib ilitie s within vide o g a me s  Bla c kBo a rd to o ls pro vide mo re o ptio ns

F lunk the ro b o -g ra de rs. L e s Pe re lma n April 30, 2014, T he B

  • sto n Glo be

Co nside ra tio ns

 I

s the E ssa y Que stio n the b e st o ptio n fo r yo u?

 Wha t c o nside ra tio ns a re impo rta nt in yo ur fie ld?  Wha t do yo u ne e d to c o nside r a b o ut yo ur stude nts?  Wha t will the y le a rn fro m yo u a s the y ta ke yo ur te st?  Wha t ste ps c a n yo u ta ke to stre ng the n va lidity a nd re lia b ility?

slide-12
SLIDE 12

10/9/2014 12 F ina l Re fle c tio n

 One T

a ke -Awa y

 One Surprise  One Are a to E

xplo re F urthe r….