Introduction to Vocareum Virtual Labs Sanjay Srivastava David Lin - - PowerPoint PPT Presentation

Introduction to Vocareum Virtual Labs

March 26, 2020

Webinar

Sanjay Srivastava

CEO

David Lin

Business Development

David Joyner, PhD

Georgia Tech

Agenda

• Vocareum Introduction - Sanjay Srivastava - 10 mins
• Teaching Computer Science at Scale - David Joyner - 30 mins
• Q&A

Mission

Deliver virtual labs and assessments securely, on budget, and at scale

( incl. skills-based exams) (infrastructure, tools, content) (user roles, policies, spend) (~ 500K learners)

Academic Partners

… over 600 institutions using Vocareum Labs today …

University of Illinois RMIT University Indiana University Delft University

Supporting broad range of subject areas

Data Science

Fully hosted notebook solutions; Pre-installed packages

Programming

Broad range of languages; Standard IDE, spyder, eclipse.

Engineering

Stream desktop & applications; License flexibility, Raspberry Pi

Big Data

Hadoop processing; Run MapR on Clusters

Machine Learning

Any ML framework; Dedicated GPU

Cloud Computing

Policy controlled, budgeted; Aggressive resource mgmt

Database

MySQL, PostgreSQL; Neo4j, DynamoDB

Full Stack

Front- and back-end services; Business logic; Databases

DevOps & Cyber Security

Range of tools and infrastructure; Linux; Sudo/Root/Admin

C/C++

Assessment Options

Manual Grading Automated Grading In-Context Feedback Peer Review Mastery Learning Competition Leaderboard

Using diverse architecture to support labs

• Web Applications
• Streaming Linux Applications
• Batch Compute Clusters
• Cloud Infrastructure
• BYO Applications + Infrastructure
• Network of Docker Containers
• Network of Virtual Machines
• AppStream

Open source web applications/frameworks

• Jupyter Notebook
• R Studio
• Terminal
• MySQL
• MongoDB
• Neo4J
• Postgres

Supporting a variety of languages:

• C, C++, Python, R, Java, Scala, Go, PHP

, etc.

• Hadoop applications,

including Spark

• ML frameworks - TensorFlow,

PyTorch

• Eclipse Theia
• Cloud 9
• Node

Open source desktop applications

• MySQL Workbench
• Eclipse
• Rattle (for R)
• Spyder
• Firefox browser
• pgAdmin

Others can be easily added upon request

Learning Management & Assessment

• Enrollment
• Hosting Course Material
• Grading
• LTI integrations

Supporting Exams

• Synchronous exams with 500 students
• Asynchronous exams (90 minutes over 3 days)
• Scheduled proctored certification tests
• Exams with a passcode provided by proctor
• Check whether learners came from a specific IP

range

• Deliver random tests from a bank of problems
• Built in plagiarism detection tests.

“I can see how it is very challenging to scale up the system so all students can access and work

• n the exam at the same time.

In previous years, we used our own online platform to handle students’ submissions. But, we found Vocareum to be much more maintainable and has better features. Using Vocareum really helps us: we know who submitted, at what time they submitted, and then we can run an auto-grading job immediately.”

• Dr. Oka Kurniawan, SUTD

CONFIDENTIAL

Student Lab Interfaces

Supporting Advanced Coursework & Research

• Long running jobs
• Large data sets
• Additional resources
• Cloud - AWS
• Databases - MySQL, Mongo, Neo4j
• Aggregated labs + tools

Deploying Vocareum

Hands-on Labs / MOOC

Labs for learners to use tools, technology and services in managed environment (policy, budget, etc)

Skills-based Exams

Administer proctored skills-based certification exams, with automatic scoring

Training / Bootcamps / Hack-a-Thons

Course material + lab environments to support instructional training events

Research

Team-based lab environment with data security, grant management, etc.

Syndication

Course material + labs packaged and distributed to Universities, partners, etc.

Virtual Campus Labs

Upgrade on-prem computer labs with purpose-built cloud labs and tools

Security

Public Subnet Private Subnet servers distributed file system

• bject storage

proxy server internet gateway NAT gateway Customer VPC

REST API

CONFIDENTIAL

Thank you!

Vocareum Webinar Series David Joyner May 28, 2020

CS1 Online & At Scale

Goal: Scale up the availability of intro-level computer science education. Solution: Georgia Tech's online CS1301: Introduction to Computing in Python course, available as both a massive open online course (MOOC) and a for-credit offering.

CS1301 Introduction to Computing CS1301 Introduction to Computing On Online

Course Videos Integrated Exercises Programming Exercises Adaptive Textbook Problem Sets Proctored Exams Recitations & TAs

Course Videos Integrated Exercises Programming Exercises Adaptive Textbook Proctored Exams Recitations & TAs ~500 2-minute videos ~500 integrated multiple choice and text entry exercises ~200 integrated programming exercises with live feedback Textbook with ~1000 multiple choice and text entry exercises ~150 programming and ~500 multiple choice/ text entry problems Proctored exams occurring within a browser-based IDE Traditional in-person recitations, graders, and teaching assistants Problem Sets

Course Videos Integrated Exercises Programming Exercises Adaptive Textbook Proctored Exams Recitations & TAs Free MOOC Verified MOOC Certificate MOOC Series Georgia Tech Course Problem Sets

Free MOOC Verified MOOC Certificate MOOC Series Georgia Tech Course

Opened February 2017, Free Opened February 2017, $100 Opened July 2018, $400 Opened January 2017, Tuition

Honors Program Course Dual Enrollment Course

Opened August 2017, Tuition

Free MOOC Series

Opened July 2018, Free Opened January 2018, Free*

Video Course Platform Assignment Platform Proctoring Platform Adaptive Textbook Platform Course Content

Partnerships

20% Headshot 80% Screencapture

All Course Content Available at bit.ly/CS1301PC

Video Content

Programming Problems Multiple Choice & Text Entry

All Course Content Available at bit.ly/CS1301PC

Exercises

Integrated Exercises

All Course Content Available at bit.ly/CS1301PC

Random Test Cases

All Course Content Available at bit.ly/CS1301PC

Sample Answers

All Course Content Available at bit.ly/CS1301PC

Timed, Proctored Tests

All Course Content Available at bit.ly/CS1301PC

Total Volume: Over 500 coding problems

All Course Content Available at bit.ly/CS1301PC

Automated Evaluation

All Course Content Available at bit.ly/CS1301PC

Phineas (Global Autograder) Student Code Ferb (Local Autograder) Grade Feedback Results

2017 2019 J F M A M J J A S O N D J F M A M J J A S O N D J F M 2018

Georgia Tech Course Free MOOC Free MOOC (Series) Honors Program Course DE Course Verified MOOC Certificate MOOC (Series)

A M J J A S O N D

200 400 600 800 1000 1200 1400 1600 1800

Enrollment Over Time (Georgia Tech)

2017 2018 2019

50000 100000 150000 200000 250000

Enrollment Over Time (All Versions)

Georgia Tech Free MOOC Verified MOOC Free MOOC Series Verified MOOC Series

2017 2018 2019

Georgia Tech, 2069 Free MOOC, 678 Verified MOOC, 239 Free MOOC Series, 3300 Certificate MOOC Series, 5564

Total Course Completers by Version

Research Findings

Students in CS1301 online class pe perf rform rm as well as1 or better than2 students in a traditional version of CS1301 in a pre- test/post-test setup.

1: Joyner, D. A. (2018). Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. London, United Kingdom. ACM Press. 2: Joyner, D. A. & McDaniel, M. (2019). Replicating and Unraveling Performance and Behavioral Differences between an Online and a Traditional CS Course. In Proceedings of the ACM Global Computing Education Conference (CompEd). Chengdu, China. ACM Press.

Research Findings

Students who previously failed or withdrew from a CS class pre prefer the online version, and perform better in the online version than the traditional version1.

1: Joyner, D. A. (2018). Intelligent Evaluation and Feedback in Support of a Credit-Bearing MOOC. In Proceedings of the 19th International Conference on Artificial Intelligence in Education. London, United

• Kingdom. Springer.

Research Findings

Students rate the online version of CS1301 as higher qu quality than other college courses they have taken1.

1: Joyner, D. A. (2018). Toward CS1 at Scale: Building and Testing a MOOC-for-Credit Candidate. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale. London, United Kingdom. ACM Press.

Awards

Bronze Award: Computing & IT Discipline 2019 Reimagine Education Awards Class Central Top 100 Course of All-Time (2019)

For more: bit.ly/CS1301PC (the whole course in MOOC format) LucyLabs.gatech.edu (research on the course) DavidJoyner.net (me!)

Questions?