Ma rc ia L . Zuc ke r, Ph.D. ZI VD L L C 1 I nte rpre t sta - PowerPoint PPT Presentation

Ma rc ia L . Zuc ke r, Ph.D. ZI VD L L C 1

 I nte rpre t sta tistic a l a na lyse s a s re po rte d b y c o mme rc ia l pro g ra ms  I de ntify the sta tistic a l a na lyse s re le va nt to the q ue stio n b e ing a ske d  Critic a lly e va lua te da ta pre se nte d in pa c ka g e inse rts fo r misuse d sta tistic s 2

 De finitio n o f Sta tistic s: T he sc ie nc e o f pro duc ing unre lia b le fa c ts fro m re lia b le fig ure s.  E va n E sa r  Be a b le to a na lyze sta tistic s, whic h c a n b e use d to suppo rt o r unde rc ut a lmo st a ny a rg ume nt.  Ma rilyn vo s Sa va nt  Sta tistic : a func tio n o f a se t o f o b se rva tio ns fro m a ra ndo m va ria b le .  CL SI Ha rmo nize d Da ta b a se 3

 A ne w POCT is to b e imple me nte d › Multiple re plic a te s o f c o ntro ls run › Run side b y side pa tie nt sa mple s with c urre nt me tho d › Da ta is:  E nte re d into E P E va lua to r OR  E nte re d into Sta tisPro OR  Se nt to ma nufa c ture r › Re po rt re turne d with lo ts o f sta tistic s  Re po rt ma y indic a te pa ss/ fa il to unc le a r spe c ific a tio ns  Ma nufa c ture r re p e xpla ins it is a ll g o o d  Ho w do I kno w it is OK ? 4

 Adva nc e fo r Administra to rs o f the L a b o ra to ry  We b ina r o n sta tistic s b y Da vid Pla ut  E xc e l T e mpla te s fo r:  L ine a rity  5 sa mple s; 2-4 re plic a te s e a c h  Re pro duc ib ility  20 va lue e va lua tio n 4 sa mple c o mpa riso n b e twe e n syste ms  Me tho d Va lida tio n  35 sa mple s  80 sa mple s   F re e do wnlo a da b le b o o k “Unde rsta nding L a b o ra to ry Sta tistic s”  http:/ / la b o ra to ry-ma na g e r.a d va nc e we b .c o m/ We b ina r/ E d ito ria l- We b ina rs/ Ma king -Se nse -o f-L a b o ra to ry-Sta tistic s.a sp x 5

 Qua ntita tive Me tho ds › Sta tistic s we use a ssume a norma l distribution SD 6

 Me a sure o f the va ria b ility o f the syste m › Ho w c lo se a re multiple re plic a te s?  Hig he r numb e r o f re plic a te s a llo ws b e tte r e stima te o f pre c isio n  Outlie rs a ffe c t sma ll numb e rs muc h mo re sig nific a ntly  Ca lc ula tio ns a ssume a No rma l Distrib utio n › F re q ue ntly untrue a ssumptio n, b ut use d a nywa y. 7

8

6 7 6 N=10 5 N=20 5 4 Frequency Frequency 4 3 3 2 2 1 1 0 0 3 3.25 3.5 3.75 4 4.25 3 3.25 3.5 3.75 4 4.25 4.5 4.75 Result Result 45 40 N=100 35 30 Frequency 25 20 15 10 5 0 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 9 Result

 Me a n – c e ntra l te nde nc y o f the da ta › Pe a k o f the b e ll c urve (Ave ra g e use d in pra c tic e )  Me dia n › Va lue whe re 50% o f sa mple s a re lo we r & 50% hig he r  Sta nda rd de via tio n (SD) – me a sure o f va ria b ility › Width o f the b e ll c urve › Re la te s to diffe re nc e b e twe e n individua l re sults a nd the me a n  Sta nda rd e rro r (SE ) – me a sure o f SD o f the me a n › Ca lc ula te d fro m va ria nc e (SD 2 ) & N  95% Co nfide nc e inte rva l › E stima te o f “truth” fro m da ta c o lle c te d › 95% pro b a b ility tha t the “true ” va lue is within the inte rva l de fine d 10

N=10 N=20 N=100 Sta tistic Me a n 3.90 4.17 4.22 95% CI me a n 3.65 – 4.14 4.00 – 4.35 4.14 – 4.27 SE 0.11 0.08 0.02 SD 0.34 0.38 0.24 𝑁𝑁𝑁𝑁 𝑇𝑇 ) ∗ 100 CV = ( 8.7% 9.1% 5.7% Me d ia n 3.99 4.21 4.25 95% CI me d ia n 3.45 – 4.20 4.01 – 4.44 4.19 – 4.29 11

2.5 N=8 2 Frequency 1.5 1 0.5 0 3.85 3.9 3.95 4 4.05 4.1 4.15 4.2 4.25 Result 25 N=98 20 Frequency 15 10 5 0 3.8 3.9 4 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 Result 12

Statistic N=10 N=8 N=100 N=98 Me a n 3.90 4.04 4.22 4.24 95% CI me a n 3.65 – 4.14 3.92 – 4.16 4.14 – 4.27 4.20 – 4.28 SE 0.11 0.05 0.02 0.02 SD 0.34 0.14 0.24 0.20 𝑁𝑁𝑁𝑁 𝑇𝑇 ) ∗ 100 CV = ( 8.7% 3.5% 5.7% 4.8% Me d ia n 3.99 4.05 4.25 4.25 95% CI me d ia n 3.45 – 4.20 3.86 – 4.23 4.19 – 4.29 4.20 – 4.30 13

 Sta tistic s o fte n lo o k b e tte r a t hig he r me a n va lue s › If me a n is 0.1 a n SD o f 0.05 is 50% CV › If me a n is 100 a n SD o f 5.0 is 5% CV  E va lua te va lue s re po rte d in inse rts › Sho uld b e ne a r c linic a l de c isio n po ints › Re q uire d fo r ne we r pro duc ts › F o r o lde r pro duc ts e xpe c t to se e mo re va ria b ility in e nd-use r re sults 14

 Co mpa riso n to “truth” › T ruth usua lly de fine d a s c urre nt syste m › T ruth a myth fo r ma ny a na lyte s  No ta b ly c o a g ula tio n, tro po nin I, o the r no n- sta nda rdize d a na lyte s  Ho w c lo se do e s POCT c o me to la b re sult › Co rre la tio n using pa tie nt sa mple s 15

16

 Da ta po ints › E a c h split sa mple g e ne ra te s o ne po int › Ho rizo nta l (X) a xis is L a b (c urre nt syste m) › Ve rtic a l (Y) a xis is po int o f c a re (ne w) de vic e  Re g re ssio n line › Ma the ma tic a l pre dic tio n o f re la tio nship b e twe e n two de vic e s

1200 Re g re ssio n 1000 line 800 Da ta W NE po ints 600 ACT 400 Re g re ssio n y = 1.03x + 3.6 200 R = 0.965 e q ua tio n 0 0 200 400 600 800 1000 ACT OL D

 Re g re ssio n e q ua tio n › 3 pa rts: Y = mX + b (y = 1.03x + 3.6)  Y = POC (ne w) re sult; X = la b (c urre nt) re sult  m = slo pe - pe rfe c t c o rre la tio n m = 1.0  b = inte rc e pt - pe rfe c t c o rre la tio n b = 0.0 › r va lue - c o rre la tio n c o e ffic ie nt  NOT 2 r  De sc rib e s ho w muc h o f the c ha ng e in Y va lue is due to the c ha ng e in the X va lue  r = 0.91 me a n 91% c o rre la tio n

Glucose 150 140 130 120 110 POC 100 y = 1.08x + 5.53 90 80 R = 0.906 70 60 50 50 70 90 110 130 150 Lab  Ca nno t judg e › All va lue s c lo se to no rma l ra ng e › No thing a b o ve 150  E va lua te the a xe s whe n lo o king a t c o rre la tio n g ra phs

400 y = 1.01x - 9.86  Assa y ra ng e to 500, so 350 R = 0.980 300 spre a d se e ms OK 250 › Iso la te d va lue drive s 200 150 c o rre la tio n 100  Orig ina l da ta se t sho we d 50 0 o ut o f ra ng e va lue s 0 100 200 300 400 › T he se must b e e xc lude d 180 y = 0.94x - 1.90 160 b e fo re re g re ssio n run R = 0.937 140 120  Re vise d da ta ha s sa me 100 issue s a s prio r g luc o se 80 60 re sults 40 20 0 0 50 100 150 200 21

 Da ta ne e d to spa n the c linic a lly impo rta nt ra ng e › Sing le e xtre me va lue s sho uld b e o mitte d › Out o f ra ng e va lue s must b e o mitte d 22

 Diffe re nc e plo t › Bla nd Altma n a na lysis › Plo t e ithe r re fe re nc e re sult o r a ve ra g e o f two me tho ds a s X  Re fe re nc e re sult use d whe n c o nside re d “truth”  e .g ., POC e le c tro lyte s ve rsus la b  Ave ra g e use d whe n “truth” is unc e rta in  e .g ., ACT c o mpa riso ns › Plo t diffe re nc e b e twe e n two re sults a s Y 23

 L o o k fo r b ia s › Co nsta nt o r va ria b le ? › Clinic a lly sig nific a nt? 6.0 6.0 5.0 5.0 4.0 Current INR - New INR 4.0 Current INR - New INR 3.0 3.0 2.0 2.0 1.0 1.0 0.0 0.0 0.0 2.0 4.0 6.0 8.0 10.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 -1.0 -1.0 -2.0 -2.0 -3.0 -3.0 -4.0 -4.0 -5.0 -5.0 -6.0 -6.0 Mean INR Mean INRs

 Cha ng e o f c linic a l de c isio n limit c a n ma inta in c urre nt pra c tic e sta nda rds 900 800 y = 1.09x - 7.53 700 R = 0.915 T a rg e t T ime 600 c ha ng e fro m Ne w ACT 500 480 to 520 400 se c o nds 300 200 100 0 0 200 400 600 800 1000 Cur r e nt ACT 25

LAB POC A >0.1 <0.1 >0.1 28 1 PPV 97% <0.1 2 9 NPV 82% Sensitivity Specificity Concordance 93% 90% 93% LAB POC B >0.1 <0.1 >0.1 18 0 PPV 100% <0.1 12 10 NPV 45% Sensitivity Specificity Concordance 60% 100% 70% 26

 Se nsitivity › a b ility o f a n a ssa y to ide ntify pa tie nts with a spe c ific c o nditio n ( true po sitive s )  Spe c ific ity › a b ility o f a n a ssa y to ide ntify pa tie nts witho ut a spe c ific c o nditio n ( true ne g ative s )  Po sitive pre dic tive va lue › like liho o d tha t a pa tie nt with a po sitive re sult (o r a b o ve the c ut-o ff) truly ha s the c o nditio n  Ne g a tive pre dic tive va lue › like liho o d tha t a pa tie nt with a ne g a tive re sult (o r b e lo w the c ut-o ff) is truly no rma l 27