Evaluating an Alternative CS1 for Students with Prior Programming - - PowerPoint PPT Presentation

▶
evaluating an alternative cs1 for students with prior
SMART_READER_LITE
LIVE PREVIEW

Evaluating an Alternative CS1 for Students with Prior Programming - - PowerPoint PPT Presentation

May 02, 2023 7 likes •211 views

Evaluating an Alternative CS1 for Students with Prior Programming Experience Michael S. Kirkpatrick Chris Mayfield SIGCSE Technical Symposium March 2017 Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017

slide-1
SLIDE 1

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Evaluating an Alternative CS1 for Students with Prior Programming Experience

Michael S. Kirkpatrick Chris Mayfield SIGCSE Technical Symposium March 2017

slide-2
SLIDE 2

CS2 CS1

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

JMU Introductory Sequence

CS 139 (4 cr.) CS 239 (4 cr.) CS 240 (3 cr.) MA 205 (3 cr.) Applied Calculus Not Java Java Java

  • Development process
  • Control structures
  • Variables and expressions
  • Functions
  • Arrays
  • Classes/objects
  • Console I/O
  • Classes/objects
  • OOP concepts
  • Packages
  • References
  • Recursion
  • Exceptions
  • Basic file I/O
  • Design and testing
  • Collections
  • New language
  • Recursion
  • Asymptotics
  • Searching
  • Sorting
  • Hashing
  • Trees
slide-3
SLIDE 3

CS2 CS1 Goals and benefits

  • No prior exposure required
  • Emphasis on algorithmic thinking
  • Maintain positive climate
  • Small class sizes
  • Exposure to two languages

CS2 CS1

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

JMU Introductory Sequence

CS 139 (4 cr.) CS 239 (4 cr.) CS 240 (3 cr.) MA 205 (3 cr.) CS 139 (4 cr.) CS 239 (4 cr.) CS 240 (3 cr.) MA 205 (3 cr.)

slide-4
SLIDE 4

CS1 CS2 CS1

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

JMU Introductory Sequence

CS 139 (4 cr.) CS 239 (4 cr.) CS 240 (3 cr.) MA 205 (3 cr.)

Fall 2013: Accelerated Java

CS 159 (3 cr.) CS 149 (3 cr.) Students with prior exposure Prereq: C C students just left later... Not enough for Big-O analysis MA 235 (3 cr.) Prereq: B-

Fall 2014: Prereq + Calculus

slide-5
SLIDE 5

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Key (Research) Questions

Effects on retention

  • Did either change affect retention into CS 159?
  • Did either change affect retention beyond CS 159?

Effects on successful progression

  • How did they do in CS 240?

Effects on underrepresented groups

  • Do overall effects extend to women and underrepresented

minority students?

slide-6
SLIDE 6

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

slide-7
SLIDE 7

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Research Hypotheses

CS 159 and CS 240 retention

  • The split sections improve retention.
  • The Calculus/B- change led to drop in retention.

CS 159 and CS 240 grades

  • CS 139/149 has no effect on CS 159 or CS 240 grades.
  • Skip CS 139/149 with AP 4 yields difference in CS 159.
  • AP 4 + CS 139/149 no difference in CS 159 or CS 240

relative to AP 5.

Effect on women and URM students

  • Split sections improve women/URM retention.
  • Calculus/B- had disproportionate effect on women/URM.
slide-8
SLIDE 8

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

CS 139/149 Demographics

0% 20% 40% 60% 80% 100% Baseline 2013-14 2014-15 White Asian URM Male Female

slide-9
SLIDE 9

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

CS 139/149 → CS 159/239 Retention

0% 20% 40% 60% 80% 100%

CS1a only X-sq = 1.66, p = 0.20 CS1 only X-sq = 4.83, p = 0.03* CS1/1a combined X-sq = 1.81, p = 0.18 Baseline (pre-2013) 2013-14

0% 20% 40% 60% 80% 100% 2014-15 X-sq = 0.04, p = 0.85 2013-14 2014-15 X-sq < 0.01, p = 0.95 2013-14 2014-15 X-sq = 0.02, p = 0.90 2013-14 CS1a CS1 CS1/1a combined

Retention Attrition

No overall effect after splitting sections No effect after adding B- prerequisite Difference between CS 139 and previous combined

slide-10
SLIDE 10

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

CS 159 → 240 Retention

0% 20% 40% 60% 80% 100%

CS1a only X-sq = 0.02, p = 0.88 CS1 only X-sq = 3.33, p = 0.07 CS1/1a combined X-sq = 1.97, p = 0.16 Baseline (pre-2013) 2013-14

0% 20% 40% 60% 80% 100%

2014-15 X-sq = 0.18, p = 0.67 2013-14 2014-15 X-sq = 6.89, p < 0.01** 2013-14 2014-15 X-sq = 5.44, p = 0.02* 2013-14 CS1a CS1 CS1/1a combined

No effect after accelerated split Significant difference after Calculus/B-, but...

slide-11
SLIDE 11

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Research Hypotheses

CS 159 and CS 240 retention

  • The split sections improve retention.
  • The Calculus/B- change led to drop in retention.

CS 159 and CS 240 grades

  • CS 139/149 has no effect on CS 159 or CS 240 grades.
  • Skip CS 139/149 with AP 4 yields difference in CS 159.
  • AP 4 + CS 139/149 no difference in CS 159 or CS 240

relative to AP 5.

Effect on women and URM students

  • Split sections improve women/URM retention.
  • Calculus/B- had disproportionate effect on women/URM.

✓ / ✕ ✕

slide-12
SLIDE 12

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

How were their grades?

CS1.5 Grade Factor Est. SE t value Pr(>|t|) (Intercept) 0.745 0.436 1.708 0.089 . CS1/1a

  • 0.319

0.126

  • 2.525

0.012 * CS1X Grade 0.834 0.096 8.685 < 4e-16 *** CS1X Attempts

  • 0.600

0.224

  • 2.675

0.008 ** Residual standard error: 0.917 on 270 degrees of freedom Multiple R-squared: 0.2633, Adjusted R-squared: 0.2552 F-statistic: 32.17 on 3 and 270 DF, p-value: < 2.2e-16

Linear regression on CS 159 grades

  • Choice of CS 139 vs CS 149? Significant (p < 0.05)
  • Grade in CS 139/149? Very significant (p < 0.001)
  • # of attempts in CS 139/149? Very significant (p < 0.01)

CS1.5 grade factor Spearman’s rank coefficient CS1 ρ = −0.158, p = 0.009 ** CS1/1a Grade ρ = 0.492, p < 3e-16 *** CS1/1a Attempts ρ = −0.182, p = 0.002 **

slide-13
SLIDE 13

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

How were their grades?

Linear regression on CS 240 grades

  • Choice of CS 139 vs CS 149? NOT significant
  • Grade in CS 159? Very significant (p < 0.001)
  • # of attempts in CS 139/149? NOT significant

CS2 Grade Factor Est. SE t value Pr(>|t|) (Intercept) 1.187 0.234 5.079 < 1e-06 *** CS1.5 Grade 0.512 0.075 6.788 < 2e-10 *** Residual standard error: 0.723 on 177 degrees of freedom Multiple R-squared: 0.2066, Adjusted R-squared: 0.2021 F-statistic: 46.08 on 1 and 177 DF, p-value: 1.651e-10

− CS2 grade factor CS1 ρ = 0.052, p = 0.489 CS1.5a Grade ρ = 0.467, p < 5e-11 ***

slide-14
SLIDE 14

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

What about AP students?

Average grades in CS 159/240

  • AP 4 who skipped CS 139/149 did worse in CS 159

Course Mean Samples Result CS1.5/1.5a AP 5, µ = 3.4 t = 2.4, d f = 56.2 AP 4 (skip), µ = 2.8 p = 0.02 * CS1.5/1.5a AP 5, µ = 3.4 t = 1.9, d f = 53.9 AP 4 (no skip), µ = 3.0 p = 0.06 CS1.5/1.5a AP 4 (skip), µ = 2.8 t = −0.7, d f = 55.2 AP 4 (no skip), µ = 3.0 p = 0.47 CS2 AP 5, µ = 2.7 t = 1.6, d f = 44.6 AP 4 (skip), µ = 2.2 p = 0.12 CS2 AP 5, µ = 2.7 t = 0.2, d f = 39.0 AP 4 (no skip), µ = 2.7 p = 0.82 CS2 AP 4 (skip), µ = 2.2 t = −1.5, d f = 42.2 AP 4 (no skip), µ = 2.7 p = 0.15

slide-15
SLIDE 15

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Research Hypotheses

CS 159 and CS 240 retention

  • The split sections improve retention.
  • The Calculus/B- change led to drop in retention.

CS 159 and CS 240 grades

  • CS 139/149 has no effect on CS 159 or CS 240 grades.
  • Skip CS 139/149 with AP 4 yields difference in CS 159.
  • AP 4 + CS 139/149 no difference in CS 159 or CS 240

relative to AP 5.

Effect on women and URM students

  • Split sections improve women/URM retention.
  • Calculus/B- had disproportionate effect on women/URM.

✕ ✕ / ✓ ✓ ✓ ✓ / ✕

slide-16
SLIDE 16

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

CS 139/149 → CS 159/239 Retention

0% 20% 40% 60% 80% 100%

2013-14 CS1 X-sq = 0.01, p = 0.91 2013-14 CS1/1a X-sq < 2e-30, p ≈ 1 Baseline (pre-2013) 2013-14 CS1 X-sq < 6e-31, p ≈ 1 2013-14 CS1/1a X-sq < 0.01, p = 0.98 Baseline (pre-2013) Women URM

0% 20% 40% 60% 80% 100%

2014-15 X-sq = 0, p ≈ 1 2013-14 2014-15 X-sq = 0, p ≈ 1 2013-14 2014-15 X-sq < 0.01, p = 0.96 2013-14 2014-15 X-sq = 0.11, p = 0.74 2013-14 Women CS1 only Women CS1/1a URM CS1 only URM CS1/1a

No effect after accelerated split No effect after increasing to B-

slide-17
SLIDE 17

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

CS 159 → CS 240 Retention

0% 20% 40% 60% 80% 100%

2014-15 CS1/1a combined X-sq = 1.15, p = 0.29 2013-14 CS1/1a combined 2014-15 CS1/1a (vs. 2013-14) X-sq = 5.11, p = 0.02* 2013-14 CS1/1a combined X-sq = 0, p ≈ 1 Baseline CS1 (pre-2013) 2013-15 CS1/1a combined X-sq < 8e-31, p ≈ 1 Baseline CS1 (pre-2013) Men Women URM

CS1.5/1.5a Retention After Calculus Change

Significant drop in women retention after increasing Calculus requirement...

slide-18
SLIDE 18

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Research Hypotheses

CS 159 and CS 240 retention

  • The split sections improve retention.
  • The Calculus/B- change led to drop in retention.

CS 159 and CS 240 grades

  • CS 139/149 has no effect on CS 159 or CS 240 grades.
  • Skip CS 139/149 with AP 4 yields difference in CS 159.
  • AP 4 + CS 139/149 no difference in CS 159 or CS 240

relative to AP 5.

Effect on women and URM students

  • Split sections improve women/URM retention.
  • Calculus/B- had disproportionate effect on women/URM.

✕ ✕ / ✓ ✓ ✓ ✕ ? / ✕ ✓ / ✕

slide-19
SLIDE 19

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Key Takeaways

Extended intro helps level playing field

  • No difference by the end of CS2

Extended intro helps students with AP 4

  • No difference by the end of CS2

Extended intro helps for fair enrollment mgmt

  • No evidence of prior exposure bias
slide-20
SLIDE 20

Evaluating an Alternative CS1 for Students with Prior Programming Experience SIGCSE 2017 • Kirkpatrick and Mayfield

Thank you

kirkpams@jmu.edu mayfiecs@jmu.edu