Global Perceptions on the Use of WeBWorK as an Online Tutor for - - PowerPoint PPT Presentation

in CS

Global Perceptions on the Use of WeBWorK as an Online Tutor for Computer Science

Olly Gotel, Christelle Scharff, Andrew Wildenberg, Mamadou Bousso, Chim Bunthoeurn, Phal Des, Vidya Kulkarni, Srisupa Palakvangsa Na Ayudhya, Cheikh Sarr, Thanwadee Sunetnanta Presented by: Olly Gotel (ogotel@pace.edu) Pace University, New York, USA [FIE 2008]

in CS

Outline

• The Problem with CS and Teaching

Programming … A Personal Perspective …

• Systems for Automated Assessment of

Programming Assignments

• WeBWorK and WeBWorK-JAG
• A Global Experiment - PERCEPTIONS
• Conclusions

in CS Systems for Automated Assessment of Programming Assignments

• Web-based systems to encourage practice

(with feedback), and improve and reinforce students’ understanding of concepts

• Types of questions

– True / false, short answer, multiple-choice, programming

• Grading programs

– Correctness + quality + authenticity

in CS

Some Existing Systems

• BOSS (http://www.dcs.warwick.ac.uk/boss, Open Source, Computer Science)
• CodeLab (http://www.turingscraft.com, Commercial, Computer Science)
• CourseMarker (http://www.cs.nott.ac.uk/coursemarker, Commercial, Computer Science)
• DevSquare (http://www.devsquare.com, Commercial, Computer Science)
• Educosoft (https://www.educosoft.com/ecf, Commercial, Mathematics)
• Gradiance (http://www.gradiance.com, Commercial, Computer Science)
• JavaBat (http://javabat.com, Free, Computer Science)
• MathXL (http://www.mathxl.com, Commercial, Mathematics)
• MyCodeMate (http://www.mycodemate.com, Commercial , Computer Science)
• MyMathTest (http://www.mymathtest.com, Commercial, Mathematics)
• OWL (http://owl.course.com, Commercial , Computer Science)
• Quiz PACK (http://www.sis.pitt.edu/~taler/QuizPACK.html, Open Source, Computer Science)
• Viope (http://www.viope.com, Commercial, Computer Science)
• WebAssign (http://www.webassign.net, Commercial, Mathematics)
• WebCAT (http://web-cat.cs.vt.edu, Open Source, Computer Science)
• WeBWorK (http://webwork.rochester.edu and http://atlantis.seidenberg.pace.edu/~scharff/webwork, Open Source,

Mathematics and Computer Science)

• WileyPLUS (http://he-cda.wiley.com/WileyCDA/Section.rdr?id=101003, Commercial, Mathematics)
• ASSYST, CFX, CodeWitz, Marmoset, Praktomat, RoboProf, SQL-Tutor, TRAKLA2…

Not an exhaustive list!

in CS

WeBWorK: About

• http://webwork.rochester.edu
• Project funded by NSF
• Free, open-source and web-based
• Automated problem delivery and grading
• Initial development and applications in the fields
• f mathematics and physics
• Currently in use at more than 60 colleges and

universities (for maths)

in CS

WeBWorK: Underneath

• Problems are written in the Problem Generating

macro language (PG)‏ – Text, HTML, Latex, Perl

• Underlying engine dedicated to dealing with

mathematical formulae – x+1 = (x^2-1)/(x-1) = x+sin(x)^2+cos(x)^2

• Individualized and parameterized versions of

problems

in CS

WeBWorK: for Programming Fundamentals

• Extension of WeBWorK for use in programming

fundamentals - Pace and Cornell (since 2005)

• True / false, short answer and multiple choice problems for

Java, Python and SML

• Evaluation of Java program fragments in real time by

interfacing WeBWorK with JUnit [www.junit.org] – WeBWorK-JAG = WeBWorK + Java Auto-Grader

• http://atlantis.seidenberg.pace.edu/webwork2

in CS

A Global Experiment

• Global Perceptions on the Use of WeBWorK as an Online

Tutor for Computer Science

– Students and Professors – Perceived value and use (class size, integration) – Implications for uptake (US-centric)

• 5 Country Experiment

– 10 Java/C problems (common questions) – Time-boxed, unlimited attempts

• Questionnaires

– Students: Background info, technology access, time spent coding per week, opinions of use, preference for question types, etc – Professors: Background info, teaching style, class sizes, homework, time spent on tasks, contact hours, technology use in classroom, value

• f question types, etc

in CS

10 Cornell College 8 University of Delhi 8 Mahidol University 10 Royal University

• f Phnom Penh

8 University of Thies

in CS

Problem Sets

in CS

Problems

in CS

Multiple-Choice Questions

in CS

Fill-in-the-Blanks Questions

in CS

You Can Build up Interesting / Typical Questions

in CS

Randomize / Parameterize

in CS

JAG - For Open Coding

in CS

Typical Feedback - Correct

in CS

Typical Feedback - Incorrect

in CS

How Did They Do?

in CS

… & With Respect to the Others?

in CS

Results

• Not designed to compare abilities!
• Very few students got all the questions

correct

• Average number of attempts per question

was 1 to 5 (except US where > 20)

– Prior exposure – Driven to achieve 100%

• Background data
• Logistics - Internet

in CS

Student Perceptions - Qs

• Like best? Instant feedback
• Like least? Clarity of questions/language
• Suggestions? Live help-desk links
• Most valuable type of questions? Open coding

(MC in Senegal)

• Easiest type of question to answer are least

valuable

• Why use? Practice
• Use again? 72.5%

in CS

Student Perceptions - Free-Form

• “WebWorK is good but needs the user to have the Internet.

So if we don't have internet we can't do it.” [Cambodia.]

• “Though it was my first experience at WeBWorK, I found it

quite simple to use and really encouraging to remind me about Java, as I haven't did any java practice for the last 3-4

• months. Its interface was nice.” [India.]
• “I liked the concept of WeBWorK. It is a useful tool for

testing students’ level at programming.” [Senegal.]

• “WeBWorK make it convenient to do the test by

automatically checking the answer and responding back to the user.” [Thai.]

• “It was a fun experience to go through the questions and find

the answers. I enjoyed reviewing, and verifying that I know how to write code.” [US.]

Implications? For uptake and future work?

in CS

Professor Perceptions - Qs

• All seek help/relief from compiling/marking

homework

• Prepared to invest time to learn -- how

much?

• Like most? Instant feedback to students so

learn from mistakes

• Most valuable question type? Open coding

in CS

Professor Perceptions - Free-Form

• “I think the problem should be clear - should [be]

basic Java and OOP concept in Java. Internet was too slow to have all students complete the exercises.” [Cambodia.]

• “I will be happy to see how it will test a large

programming assignment with multiple classes.” [India.]

• “Internet use which expands collaborations.”

[Senegal.]

• “…limited programming library; expand problem

library…” [US.]

Implications? For uptake and future work?

in CS

Conclusions

• Supports an overloaded professor and can tide students
• ver in-between classes
• These systems open up opportunities for professors and

students around the globe to transfer and share knowledge, and extend practice, but some local adaptations are necessary (Intranet, question formulation, integration)

• Development of a novel pedagogy, encouraging

students to contribute their own questions to the WeBWorK library, introducing them to crucial practices of software engineering

• Aim to create a community of contributors to monitor

quality, share work and extend the WeBWorK library

in CS

Acknowledgements

• NSF CCLI AI Grants “Collaborative Research: Adapting

and Extending WeBWorK for Use in the Computer Science Curriculum” #0511385 and #0511391

• Students:

– The many computer science students in the US, Cambodia, India, Senegal and Thailand – Jacqueline Baldwin, Nathan Baur (JUnit extension) – Sophal Chiv (input of problems and help desk) – Eileen Crupi, Tabitha Estrellado (input of problems) – Allyson Ortiz, Veronica Portas (existing systems) – Yue Ma (user acceptance testing)