Course Business New dataset on CourseWeb: bpd.csv Midterm project - - PowerPoint PPT Presentation
Course Business New dataset on CourseWeb: bpd.csv Midterm project due today Today & next week : Specialized designs Today: Signal detection theoryfor categorical judgments Next week: Longitudinal designs Week 9:
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects
l Your colleague Arpad, who studies insomnia, ran a
InsomniaModel <- glmer(SleptThroughNight ~ 1 + HoursExercise + MgCaffeine + (1|Subject), data=sleep, family=binomial)
l Arpad would like help interpreting his R output. l Describe how hours of exercise affected
l Your colleague Arpad, who studies insomnia, ran a
InsomniaModel <- glmer(SleptThroughNight ~ 1 + HoursExercise + MgCaffeine + (1|Subject), data=sleep, family=binomial)
l Arpad would like help interpreting his R output. l Describe how hours of exercise affected
l Every hour of exercise increased the odds of
l Sleep data from one subject wasn’t properly
l Since there is no reason to think this subject
(a) sleep$HoursSleep <- ifelse(is.na(sleep$HoursSleep), 0, sleep$HoursSleep) (b) sleep <- subset(sleep, is.na(sleep$HoursSleep) == FALSE) (c) sleep <- sleep[is.na(sleep$HoursSleep) == FALSE, ] (d) sleep <- na.omit(sleep)
l Sleep data from one subject wasn’t properly
l Since there is no reason to think this subject
(a) sleep$HoursSleep <- ifelse(is.na(sleep$HoursSleep), 0, sleep$HoursSleep)
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects
las gatos (1) Grammatical (4) Ungrammatical
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects
l Imagine asking groups of second-language
l Imagine asking groups of second-language
ACCURACY
SAID “GRAMMATICAL”
l Imagine asking groups of second-language
ACCURACY SAID “GRAMMATICAL”
l Proportion accuracy would be misleading l We want an analysis that tests both subjects’
ACCURACY
SAID “GRAMMATICAL”
l Comparison to “chance” get at a similar idea
l
l Many experiments do balance frequency of
l But even so, bias can differ for many reasons
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects
l Traditional logistic regression model: l Accuracy confounds sensitivity and
l Traditional logistic regression model: l Signal detection model:
l Traditional logistic regression model: l More generally:
l SDT model:
l SDT model:
Results
l More generally:
l bpd.csv
l Clinical trainees evaluating learning to diagnose
l Potentially relevant columns:
l JudgedBPD: Trainees’ judgment of BPD (1 yes, 0 no) l HasBPD: Whether the person in the case actually has
l Accuracy: Was the trainees’ judgment correct? (1
l If our memory experiment SDT analysis involved
l Can you run a SDT model on the bpd data?
l Tip 1: Apply effects coding (-0.5 and 0.5) to the
l Tip 2: Should this be an lmer model or a glmer
l If our memory experiment SDT analysis involved
l Can you run a SDT model on the bpd data?
l contrasts(bpd$HasBPD) <- c(-0.5, 0.5) l model1 <- glmer(JudgedBPD ~
l model2 <- glmer(JudgedBPD ~
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects
l Signal detection model with another
l More generally…
l SDT model:
l SDT model:
Results
l More generally…
l SDT model:
l SDT model:
Results
l We’re concerned that there may be a Gender
l Can you test whether Gender affects response
l Don’t forget to apply effects coding (-0.5 and 0.5) to
l Which gender do we think will get more BPD
l We’re concerned that there may be a Gender
l Can you test whether Gender affects response
l contrasts(bpd$Gender) <- c(0.5, -0.5) l model3 <- glmer(JudgedBPD ~
l Summary:
l No overall response bias to judge people as having
l Trainees have some ability to discern which people
l Overall bias to diagnosis more women with BPD, but
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects
Area under curve from -∞ up to this point
l Signal Detection Theory
l Why Do We Need SDT? l Sensitivity vs. Response Bias l Implementation l SDT & Other Independent Variables l Logit vs. Probit
l Discuss Midterm Projects