Organizers: Tom Mller, Hana Rudov, and Zuzana Mllerov Outline - - PowerPoint PPT Presentation

Organizers: Tomáš Müller, Hana Rudová, and Zuzana Müllerová

Outline

1

Introduction

2

Competition problem and data instances

3

Presentations of finalists

1

Efstratios Rappos, Eric Thiémard, Stephan Robert Jean-François Hêche, HEIG-VD, Switzerland

2

Karim Er-rhaimini Ministère de l’éducation nationale, France

3

Alexandre Lemos, Pedro T Monteiro, Inês Lynce INESC-ID / IST, Universidade de Lisboa, Portugal

4

Dennis S. Holm, Rasmus Ø. Mikkelsen, Matias Sørensen, Thomas R. Stidsen, Technical University of Denmark, Denmark

5

Edon Gashi, Kadri Sylejmani University of Prishtina, Kosovo

4

Results of the competition

5

Now & future: benchmarks, PATAT 2021

6

Questions: chat in Zoom

International Timetabling Competition 2019 2

Competitions supported by PATAT

Conference on the Practice and Theory of Automated Timetabling The First International Timetabling Competition (ITC 2002) The Second International Timetabling Competition (ITC 2007) The First International Nurse Rostering Competition (2010) The Third International Timetabling Competition (ITC 2011) Cross-domain Heuristic Search Challenge (CHeSC 2011) The Second International Nurse Rostering Competition (2014-2016) The Fourth International Timetabling Competition (ITC 2019) See http://patatconference.org/communityService.html

International Timetabling Competition 2019 3

International Timetabling Competitions

ITC 2002

events, rooms, students enrollment-based timetabling students in events cannot have any overlap

ITC 2007

examination timetabling post enrolment-based course timetabling ITC 2002 extension curriculum-based course timetabling based on real-world instances from University of Udine

ITC 2011

high-school timetabling real-world instances

International Timetabling Competition 2019 4

ITC 2019: University course timetabling

Enrollment-based timetabling

students enroll in courses

Course structure how to split course into events/classes

students sectioning often needed

Rooms with travel times and unavailabilities Events not meeting every week Distribution constraints on set of courses: 19 types

DifferentTime, SameRoom, MinGap, MaxBreaks, ...

Assignment of times and rooms to classes Optimization by minimizing penalties of time and room assignments, violated soft constraints, student conflicts

• T. Müller, H. Rudová, and Z. Müllerová, University course timetabling and

International Timetabling Competition 2019. In PATAT 2018 – Proceedings of the 12th International Conference on the Practice and Theory of Automated Timetabling (PATAT 2018), pages 5–31, 2018.

International Timetabling Competition 2019 5

Organization of the competition

August 30, 2018: Announcement, sample data instances November 15, 2018: Early instances published February 1, 2019: 1st milestone June 1, 2019: 2nd milestone September 18, 2019: Middle instances published November 8, 2019: Late instances published November 18, 2019: Deadline January 1, 2020: Open source solvers publication September 2, 2020: Award ceremony August 2021: PATAT 2021 special track

International Timetabling Competition 2019 6

Real-world data instances

Data from the UniTime educational scheduling system Diverse characteristics 10 institutions from 5 continents

Purdue University Masaryk University AGH University of Science and Technology Lahore University of Management Sciences ˙ Istanbul Kültür University Bethlehem University Universidad Yachay Tech Turkish-German University University of Nairobi Maryville University

International Timetabling Competition 2019 7

Differences among data instances

Size of the problem

• ne school or faculty: 500 classes, 2,000 students, 50 rooms

(part of) university: 8,800 classes, 38,000 students, 770 rooms Room utilization may happen: problem complexity difficult – not size Student course demands diverse pre-enrollments created based on curricula students by distribution constraints (e.g., SameAttendees) Course structure lecture, seminar, laboratory, ... student sectioning complex, e.g., introductory Biology for most freshmen simple: course = lecture

International Timetabling Competition 2019 8

Differences among data instances

Times Europe: class once a week USA: class several times a week at the same time and room even/odd weeks, some weeks distance learning, e.g., irregular classes on Fridays Travel times

• ne building vs. campus

Distribution constraints use of constraints for instructors (e.g., WorkDay, MaxBlock)

International Timetabling Competition 2019 9

UniTime → ITC 2019

Data taken from the course timetabling module of UniTime anonymized and simplified format & model goal: remove features that

do not make the problem easier to solve just easier to model

Examples: removed/simplified features

rooms a class may have multiple rooms, classes may share a room more complex room sharing distribution constraints: some removed or changed students reservations keeping some students together student conflicts may be prohibited instructors using same attendees in the competition instructors may require shorter travel times

• ptimization: additional penalizations

International Timetabling Competition 2019 10

Statistics about data instances

Basics about courses, classes, rooms, and students Statistics about distribution constraints Statistics about domains Statistics about date and time patterns Statistics about utilization See https://www.itc2019.org/early-instances ...

International Timetabling Competition 2019 11

Basic information about Early instances

Basic information Students Instance Size Courses Classes fixed Rooms

• St. St.courses St.classes

agh-fis-spr17 14.55 340 1,239 543 80 1,641 8.17 16.20 agh-ggis-spr17 5.82 272 1,852 332 44 2,116 6.98 29.92 bet-fal17 3.88 353 983 79 62 3,018 6.24 9.08 iku-fal17 12.60 1,206 2,641 530 214 – – mary-spr17 2.94 544 882 63 90 3,666 2.88 2.90 muni-fi-spr16 1.41 228 575 128 35 1,543 6.24 10.06 muni-fsps-spr17 1.48 226 561 191 44 865 7.76 11.60 muni-pdf-spr16c 15.92 1,089 2,526 1,132 70 2,938 8.72 17.35 pu-llr-spr17 4.69 687 1,001 318 75 27,018 3.02 3.40 tg-fal17 1.94 36 711 74 15 – –

International Timetabling Competition 2019 12

Date and time patterns, utilization

Date & time patterns Utilization per semester Instance Weeks Minutes Days Weeks Minutes Minutes Minutes per mtg. per class per class per class per room per student agh-fis-spr17 16 104.46 1.01 10.93 1,135.0 9,352.1 17,441.4 agh-ggis-spr17 16 124.46 1.00 4.18 450.2 16,111.4 12,428.7 bet-fal17 16 93.38 1.38 15.81 1,840.3 21,724.3 15,936.5 iku-fal17 14 123.70 1.00 14.00 1,734.4 19,060.9 – mary-spr17 16 141.88 1.51 13.83 2,606.1 20,069.3 7,252.5 muni-fi-spr16 15 121.45 1.00 10.89 1,188.3 17,295.7 12,593.5 muni-fsps-spr17 19 90.97 1.00 9.30 731.6 6,008.9 7,728.2 muni-pdf-spr16c 13 140.50 1.00 5.90 527.0 16,985.6 5,194.6 pu-llr-spr17 16 63.52 1.90 14.33 1,659.0 21,666.7 5,906.6 tg-fal17 14 132.53 1.00 1.16 159.8 7,308.4 – Minutes per class = |Weeks| * |Days a Week| * |Minutes per mtg.|

International Timetabling Competition 2019 13

Current status

November 18, 2019

5 teams participated in the final competition 7 (15) teams uploaded one or more solutions (including sample instances) 23 users successfully validated one or more solutions

Now

13 (20) teams uploaded one or more solutions (including sample instances) 29 users successfully validated one or more solutions

International Timetabling Competition 2019 14

Current status

263 registered users from 57 countries

International Timetabling Competition 2019 15

Acknowledgements

Conference on the Practice and Theory

• f Automated Timetabling (PATAT)

ORTEC: optimization software and analytics solutions Apereo Foundation: supporting open-source software for higher education EURO working group on Automated Timetabling (EWG PATAT) UniTime educational scheduling system Faculty of Informatics, Masaryk University

International Timetabling Competition 2019 16

Presentations of the finalists

1

Efstratios Rappos, Eric Thiémard, Stephan Robert, Jean-François Hêche

HEIG-VD, Switzerland

2

Karim Er-rhaimini

Ministère de l’éducation nationale, France

3

Alexandre Lemos, Pedro T Monteiro, Inês Lynce

INESC-ID / IST, Universidade de Lisboa, Portugal

4

Dennis S. Holm, Rasmus Ø. Mikkelsen, Matias Sørensen, Thomas R. Stidsen

MaCom / Technical University of Denmark, Denmark

5

Edon Gashi, Kadri Sylejmani

University of Prishtina, Kosovo

International Timetabling Competition 2019 17

Ordering of competitiors

Ranking based on the computation of points in the F1 championship

Points awarded per instance Position Early Middle Late 1. 10 15 25 2. 7 11 18 3. 5 8 15 4. 3 6 12 5. 2 4 10 6. 1 3 8 7. – 2 6 8. – 1 4 9. – – 2 10. – – 1

International Timetabling Competition 2019 18

1st milestone: Early instances

Early instances published: November 15, 2018 1st milestone: February 1, 2019 Prices by PATAT: 300, 200, 100 EUR Winners 1st place (94 points)

Edon Gashi, Kadri Sylejmani University of Prishtina, Kosovo

2nd place (69 points)

Karim Er-rhaimini Ministère de l’éducation nationale, France

3rd place (32 points)

Marlúcio A. Pires, Haroldo G. Santos, Túlio Ângelo M. Toffolo Universidade Federal de Ouro Preto, Brasil

International Timetabling Competition 2019 19

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

1st Place in the 1st Milestone

for the Team of

Edon Gashi Kadri Sylejmani

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

2nd Place in the 1st Milestone

for

Karim Er-rhaimini

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

3rd Place in the 1st Milestone

for the Team of

Marlúcio Alves Pires Haroldo Gambini Santos Túlio Ângelo Machado Toffolo

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

1st milestone results

Gashi et al. Er-rhaimini Pires et al. Instance penalty points penalty points penalty points agh-fis-spr17 7,419 10 22,488 7 – agh-ggis-spr17 75,123 10 174,405 7 180,641 5 bet-fal17 324,294 10 383,254 7 – iku-fal17 74,335 10 151,599 7 – mary-spr17 26,745 10 33,217 7 55,847 5 muni-fi-spr16 7,865 7 6,918 10 – muni-fsps-spr17 25,526 10 33,760 7 179,616 5 muni-pdf-spr16c 97,898 10 485,753 7 – pu-llr-spr17 34,962 10 67,635 5 64,130 7 tg-fal17 10,570 7 20,705 5 8,990 10 Total points 94 69 32

International Timetabling Competition 2019 20

2nd milestone: Early instances

Early instances published: November 15, 2018 1st milestone: February 1, 2019 2nd milestone: June 1, 2019 Prices by PATAT: 300, 200, 100 EUR Winners 1st place (73 points)

Edon Gashi, Kadri Sylejmani University of Prishtina, Kosovo

2nd place (69 points)

Dennis S. Holm, Rasmus Ø. Mikkelsen, Matias Sørensen, Thomas R. Stidsen MaCom, Denmark

3rd place (61 points)

Karim Er-rhaimini Ministère de l’éducation nationale, France

International Timetabling Competition 2019 21

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

1st Place in the 2nd Milestone

for the Team of

Edon Gashi Kadri Sylejmani

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

2nd Place in the 2nd Milestone

for the Team of

Dennis S. Holm Rasmus Ø. Mikkelsen Matias Sørensen Thomas R. Stidsen

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

3rd Place in the 2nd Milestone

for

Karim Er-rhaimini

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

2nd milestone results

Gashi ... Holm ... Er-rhaimini Pires ... Rappos ... Instance

• penal. po.
• penal. po.
• penal. po.
• penal. po.
• penal. po.

agh-fis-spr17 6,030 10 22,689 5 7,563 7 – – agh-ggis-spr17 64,703 7 49,901 10 79,695 5 180,641 3 – bet-fal17 301,725 10 – 383,254 7 – – iku-fal17 61,505 7 19,080 10 68,525 5 153,008 3 – mary-spr17 18,131 7 14,927 10 23,733 5 55,847 3 – muni-fi-spr16 5,973 5 4,112 10 5,729 7 16,902 3 – muni-fsps-spr17 9,316 5 6,676 7 5,601 10 179,616 2 119,879 3 muni-pdf-spr16c 74,186 10 – 108,007 7 – – pu-llr-spr17 23,364 7 10,046 10 23,579 5 64,130 3 – tg-fal17 8,093 5 4,890 7 10,618 3 4,215 10 – Total points 73 69 61 27 3

International Timetabling Competition 2019 22

Ongoing results: Early instances

Instance 1st 2nd Competition Now Date Feb 1, 2019 June 1, 2019 Nov 18, 2019 Sep 2, 2020 agh-fis-spr17 9,259 7,270 3,081 3,039 agh-ggis-spr17 98,868 49,901 35,808 34,285 bet-fal17 327,048 303,399 290,086 289,965 iku-fal17 74,335 19,080 18,968 ← mary-spr17 26,825 14,927 14,910 ← muni-fi-spr16 6,918 4,112 3,756 ← muni-fsps-spr17 33,760 5,601 868 ← muni-pdf-spr16c 125,938 85,248 36,487 33,724 pu-llr-spr17 34,962 10,046 10,038 ← tg-fal17 8,990 4,215 ← ← Published: November 15, 2018

International Timetabling Competition 2019 23

Competition results: Early instances

agh-fis-spr17 agh-ggis-spr17 bet-fal17 iku-fal17 mary-spr17 muni-fi-spr16 muni-fsps-spr17 muni-pdf-spr16c pu-llr-spr17 tg-fal17 Total points Holm 3,081 35,808 290,086 18,968 14,910 3,756 868 36,487 10,038 4,215 et al. 10 10 10 10 10 10 10 10 10 9 99 Rappos 4,557 36,616 295,427 26,840 15,021 3,844 883 37,487 13,385 4,215 et al. 7 7 7 7 7 7 7 7 7 9 72 Gashi 6,799 77,932 299,205 50,613 15,894 5,006 1,938 5,8206 16,874 8,044 et al. 3 3 5 3 5 5 5 5 5 2 41 Er-rhaimini 5,709 56,755 313,812 44,482 16,698 5,207 4,135 77,573 19,231 7,358 5 5 3 5 3 3 3 3 3 3 36 Lemos 35,139 194,138 – – 51,147 19,314 211,142 – 68,003 6,774 et al. 2 2 2 2 2 2 5 17

International Timetabling Competition 2019 24

Ongoing results: Middle instances

Instance Competition Now Date Nov 18, 2019 Sep 2, 2020 agh-ggos-spr17 3,055 2,864 agh-h-spr17 23,502 22,175 lums-spr18 95 ← muni-fi-spr17 3,825 ← muni-fsps-spr17c 2,596 ← muni-pdf-spr16 18,151 17,208 nbi-spr18 18,014 ← pu-d5-spr17 15,910 15,204 pu-proj-fal19 148,016 117,425 yach-fal17 1,239 1,074 Published: September 18, 2019

International Timetabling Competition 2019 25

Competition results: Middle instances

agh-ggos-spr17 agh-h-spr17 lums-spr18 muni-fi-spr17 muni-fsps-spr17c muni-pdf-spr16 nbi-spr18 pu-d5-spr17 pu-proj-fal19 yach-fal17 Total points Holm 3,055 23,502 95 3,825 2,596 18,151 18,014 15,910 148,016 1,239 et al. 15 15 15 15 15 15 15 15 15 15 150 Rappos 6,320 26,159 114 4,289 3,303 24,318 19,055 18,813 561,194 1,844 et al. 11 6 8 11 11 11 11 11 6 8 94 Gashi 9,666 25,081 107 4,692 9,222 40,074 26,517 19,440 237,909 1,727 et al. 6 11 11 8 8 6 8 8 8 11 85 Er-rhaimini 7,725 25,745 178 5,433 23,520 38,826 30,309 20,242 176,039 3,181 8 8 6 6 6 8 6 6 11 6 71 Lemos 79,745 55,887 820 18,080 618,217 310,994 49,924 – – 32,198 et al. 4 4 4 4 4 4 4 4 32

International Timetabling Competition 2019 26

Ongoing results: Late instances

Instance Competition Now Date Nov 18, 2019 Sep 2, 2020 agh-fal17 153,236 118,038 bet-spr18 348,589 348,524 iku-spr18 25,878 ← lums-fal17 349 ← mary-fal18 4,423 4,422 muni-fi-fal17 2,999 ← muni-fspsx-fal17 17,074 10,123 muni-pdfx-fal17 117,412 98,373 pu-d9-fal19 43,006 39,942 tg-spr18 12,704 ← Published: November 8, 2019

International Timetabling Competition 2019 27

Competition results: Late instances

ORTEC: 150 EUR per each best solution for the Late instances

agh-fal17 bet-spr18 ku-spr18 lums-fal17 mary-fal18 muni-fi-fal17 muni-fspsx-fal17 muni-pdfx-fal17 pu-d9-fal19 tg-spr18 Total points Holm 186,200 348,589 25,878 349 4,423 2,999 17,074 117,412 43,006 12,704 et al. 15 25 25 25 25 25 25 25 25 25 240 Rappos – 360,057 36,711 386 5,637 3,794 33,001 151,464 134,009 12,856 et al. 18 18 18 18 18 18 18 12 18 156 Gashi 184,030 360,437 85,969 486 7,199 4,712 44,059 170,061 82,757 15,992 et al. 18 15 12 15 12 15 15 15 15 15 147 Er-rhai. 153,236 373,039 70,932 558 6,944 4,820 104,625 191,887 70,450 19,738 25 12 15 12 15 12 12 12 18 12 145 Lemos – – – 1,151 44,097 – – – – 31,900 et al. 10 10 10 30

International Timetabling Competition 2019 28

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

Winners for 9 Late instances

for the Team of

Dennis S. Holm Rasmus Ø. Mikkelsen Matias Sørensen Thomas R. Stidsen

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

Winner for 1 Late instance

for

Karim Er-rhaimini

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Competition results!

Position Team Early Middle Late Total points 1. 2. 3. 4. 5. Lemos et al. 17 32 30 79

International Timetabling Competition 2019 29

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

Finalist

Nomination for

Alexandre Lemos Pedro T Monteiro Inês Lynce

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Competition results!

Position Team Early Middle Late Total points 1. 2. 3. 4. Er-rhaimini 36 71 145 252 5. Lemos et al. 17 32 30 79

International Timetabling Competition 2019 30

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

Finalist

Nomination for

Karim Er-rhaimini

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Competition results!

Position Team Early Middle Late Total points 1. 2. 3. Gashi et al. 41 85 147 273 4. Er-rhaimini 36 71 145 252 5. Lemos et al. 17 32 30 79

International Timetabling Competition 2019 31

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

3rd Place

for the Team of

Edon Gashi Kadri Sylejmani

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Competition results!

Position Team Early Middle Late Total points 1. 2. Rappos et al. 72 94 156 322 3. Gashi et al. 41 85 147 273 4. Er-rhaimini 36 71 145 252 5. Lemos et al. 17 32 30 79

International Timetabling Competition 2019 32

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

2nd Place

for the Team of

Efstratios Rappos Eric Thiémard Stephan Robert Jean-François Hêche

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Competition results!

Position Team Early Middle Late Total points 1. Holm et al. 99 150 240 489 2. Rappos et al. 72 94 156 322 3. Gashi et al. 41 85 147 273 4. Er-rhaimini 36 71 145 252 5. Lemos et al. 17 32 30 79

International Timetabling Competition 2019 33

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

1st Place

for the Team of

Dennis S. Holm Rasmus Ø. Mikkelsen Matias Sørensen Thomas R. Stidsen

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Open-source price

500 USD for the winner by Apereo Foundation Gashi et al. https://github.com/edongashi/itc-2019 Other competitor Lemos et al. https://github.com/ADDALemos/MPPTimetables

International Timetabling Competition 2019 34

International Timetabling Competition

Supported by the conference on the Practice and Theory of Automated Timetabling (PATAT)

Best Open-Source Solver

for the Team of

Edon Gashi Kadri Sylejmani

September 2, 2020 Brno, Czech Republic Competition organizers Tomáš Müller Hana Rudová Zuzana Müllerová

Current results

Position Author Early Middle Late Total points 1. Dennis Holm 97 129 226 452 2. Tomáš Müller 64 123 195 382 3. Efstratios Rappos 58 64 126 248 4. Edon Gashi 22 57 104 183 5. Karim Er-rhaimini 19 47 102 168 6. Alexandre Lemos 11 27 80 118 7. Jason C.H 3 18 60 81 8. Marlúcio Alves Pires 9 23 41 73 9. Eduardo Flores 6 10 16 10. I Gusti Agung Premananda 2 8 10 11. Quentin Peña 4 4 12. Jerry Wang 1 1

International Timetabling Competition 2019 35

Now & future

https://www.itc2019.org Score based on F1 championship

competition current score

Results

upload new solutions associate papers with solutions

PATAT 2021 special track

full presentations live award ceremony new papers

Video & slides at the website Questions? Chat?

International Timetabling Competition 2019 36