University of Arkansas at Little Rock SaaS Migration Debrief - PowerPoint PPT Presentation

University of Arkansas at Little Rock SaaS Migration Debrief

Project SaaS : Goals and Vision Enhanced Quality Rapid Innovation Fixes & maintenance more quickly Enhancements & features more quickly Improved Experience Milestone to Analytics for Learn SaaS migration required in order to leverage A4L Zero downtime updates Easier Change Management Gateway to ULTRA Option to leverage newer interface in future Smaller, continuous delivery updates As part of a special program Blackboard has in place with the Board of Trustees of the University of Arkansas, many participating institutions, including UALR, had the opportunity to switch Learn deployment models from Self-Hosting or Managed Hosting to Software-as-a-Service or SaaS. At this time about half of the UA campuses have migrated to Learn SaaS. 2 2

Summary of Technical Events Date Issue Impact 10 JAN Storage throughput not sized properly. Performance degradation 12 JAN Both storage throughput and database services undersized. 2 hours of severe application slowness. 17 JAN NAT Gateway – AWS environmental service - failure 1 hour 40 min outage. impacted all SaaS customers. 06 FEB Database lock System outage 12 FEB Database Lock System outage 13 Feb Failed version upgrade 2 hour system outage. Next upgrade attempt successful 28 Feb AWS East outage Full system outage, 4 hours. Cumulative Effect January 99.507168% (220 Mins / 3.66 Hours) February 98.405258% (643 Mins / 10.71 Hours) 3

Other Challenges that Limited Faculty & Student Success Speed of restores Malformed Course ID’s after SIS Integration during SaaS migration restores of merged courses Enrollments and Drops were not being SIS feed run on 3 Jan created shells that Identified problem on 27 Dec. Affected processed properly. Students confused caused the failure of restores. We had approximately 120 courses in the current by seeing courses in which they were to pause restores until we could delete term. no longer enrolled. the course shells that were created by Blackboard worked with UALR to manually the feed. correct the courses. Fully resolved on 7 Blackboard worked with UALR team to Jan, before the start of the spring term. manually help them process drops Historic courses back to the beginning directly in the database. of 2015 were restoring at a slow rate. Professor’s use the old courses to pull Fully resolved the issue after Assessment Issue content in the current term. approximately 3 weeks Reported 28 Jan and escalated to product Process completed by Blackboard Learn development immediately. Ops to speed up the remaining restores 21- 22 Jan. Resolved on 13 Feb with the deployment of release 3100.9 (2) to production 4

Continuous Improvement & Risk Mitigation Upgraded our NAT Gateway Revised storage and database Resolved 75 support tickets services with AWS for a more sizing guidelines for future since 01 Jan. Currently two robust solution. operations. open severity 3 tickets. 73% of tickets closed in under 7 days. Daily email summary of project Four version upgrade Ongoing work to complete a with all support ticket status. deployments in two months. comparison report of prior Managed Hosting environment Daily check-in call with full Three completed with to new SaaS UALR project team. no issue. Environment. 5

Questions and Feedback • Please share your name, department and role • Feel free to ask a question, raise an issue, or voice an observation. • We will respond with candor and transparency. 6