Matthew Flaschen Software Engineer, Collaboration T eam, - - PowerPoint PPT Presentation

matthew flaschen
SMART_READER_LITE
LIVE PREVIEW

Matthew Flaschen Software Engineer, Collaboration T eam, - - PowerPoint PPT Presentation

Matthew Flaschen Software Engineer, Collaboration T eam, Wikimedia Foundation Converting LiquidThreads to Flow Or: How I learned to stop worrying and love the batch But First A Long Introduction Wikipedia Matthew Flaschen Software


slide-1
SLIDE 1

Matthew Flaschen

Software Engineer, Collaboration T eam, Wikimedia Foundation

slide-2
SLIDE 2

Converting LiquidThreads to Flow

Or: How I learned to stop worrying and love the batch

slide-3
SLIDE 3

But First

A Long Introduction

slide-4
SLIDE 4

Wikipedia

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11

Matthew Flaschen

Software Engineer, Collaboration T eam, Wikimedia Foundation

slide-12
SLIDE 12

Discussion!

slide-13
SLIDE 13
slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18

LiquidThreads

slide-19
SLIDE 19
slide-20
SLIDE 20

Flow

slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23

Storage model

slide-24
SLIDE 24

Conversion Goals

slide-25
SLIDE 25

T est with real data locally

slide-26
SLIDE 26

Future fmexibility in import source

slide-27
SLIDE 27

Resumability

slide-28
SLIDE 28

Preserve attribution and history

slide-29
SLIDE 29

Conversion Architecture

slide-30
SLIDE 30

Abstract API handling

slide-31
SLIDE 31

Source store

slide-32
SLIDE 32

Data mapping

slide-33
SLIDE 33

Map similar Thread → T

  • pic

Summary → Summary Page → Description Post → Post

slide-34
SLIDE 34

Mock dissimilar Moved posts Hidden user names

slide-35
SLIDE 35

Converting LiquidThreads to Flow

(on offjcewiki)

slide-36
SLIDE 36

Converting LiquidThreads to Flow

(on MediaWiki.org)

slide-37
SLIDE 37

MediaWiki.org

slide-38
SLIDE 38

Conversion Rollout

slide-39
SLIDE 39

2 pages 5 pages

slide-40
SLIDE 40

2 pages 5 pages 24 pages

slide-41
SLIDE 41

2 pages 5 pages 24 pages

slide-42
SLIDE 42

2 pages 5 pages 24 pages

slide-43
SLIDE 43

Don't try this at home.

slide-44
SLIDE 44

"/tmp/${page//\//-}.log";

slide-45
SLIDE 45

250 pages, without user talk

slide-46
SLIDE 46

All remaining except the two hard ones

slide-47
SLIDE 47

Is it that bad, sir? Looks like it's pretty hairy.

slide-48
SLIDE 48

A classic

slide-49
SLIDE 49

T alk:Wikimedia engineering 20% policy

slide-50
SLIDE 50

__proto__

slide-51
SLIDE 51

A tale of two syntaxes

slide-52
SLIDE 52

alias = '__proto__'; this.magicWords[alias]

slide-53
SLIDE 53

hasOwnProperty (or ECMAScript maps)

slide-54
SLIDE 54
slide-55
SLIDE 55

A tale of one annoying syntax

slide-56
SLIDE 56



slide-57
SLIDE 57

Not valid in HTML5

slide-58
SLIDE 58
slide-59
SLIDE 59

User not fully renamed

slide-60
SLIDE 60

Almost done

slide-61
SLIDE 61

page='Talk:Groups/Proposals/Ciudad de México'; echo "/tmp/${page//\//-}.log"; /tmp/Talk:Groups/Proposals/Ciudad de México.log

slide-62
SLIDE 62

page='Talk:Groups/Proposals/Ciudad de Mexico'; echo "/tmp/${page//\//-}.log"; /tmp/Talk:Groups-Proposals-Ciudad de Mexico.log

slide-63
SLIDE 63

That's what sed's for. ;)

slide-64
SLIDE 64

$(echo "$page"|sed 's#/#-#g')

slide-65
SLIDE 65

Performance

slide-66
SLIDE 66

T echnique: Install XDebug to profjle memory usage

slide-67
SLIDE 67

T echnique: Difgerent cache handling for batch

slide-68
SLIDE 68

The Last Page

slide-69
SLIDE 69

Fatal error: Maximum function nesting level of '100' reached, aborting! in /srv/mediawiki/php- 1.26wmf18/vendor/zordius/lightncandy/src/lightn candy.php on line 2150 Call Stack: 0.0003 663088 1. {main}() /srv/mediawiki/multiversion/MWScript.php:0 0.0179 1097424 2. require_once('/srv/mediawiki/php- 1.26wmf18/extensions/Flow/maintenance/convertLq tPageOnLocalWiki.php') /srv/mediawiki/multiversion/MWScript.php:97 0.0199 1753856 3. require_once('/srv/mediawiki/php- 1.26wmf18/maintenance/doMaintenance.php') /srv/mediawiki/php- 1.26wmf18/extensions/Flow/maintenance/convertLq tPageOnLocalWiki.php:91 ...

slide-70
SLIDE 70

Credit

slide-71
SLIDE 71

David McCabe

For writing LiquidThreads

slide-72
SLIDE 72

Andrew Garrett

For major long-term new work

  • n LiquidThreads
slide-73
SLIDE 73

Danny Horn Nick Wilson

For project management and community coordination in a shifting timeline

slide-74
SLIDE 74

Andrew Garrett Erik Bernhardson Matthias Mullie Matthew Flaschen

For implementation of LiquidThreads->Flow conversion

slide-75
SLIDE 75

LiquidThreads and Flow contributors

slide-76
SLIDE 76

The Collaboration T eam

For ongoing new work on opt-in, conversion, and just awesome stufg all around

slide-77
SLIDE 77

It is the stated position(not really) of the U.S. Air Force that their safeguards would prevent the

  • ccurrence of such events as

are depicted in this presentation.

slide-78
SLIDE 78

Culture

Stanley Kubrick – How I learned to stop worrying and love the bomb James T aylor - Mexico

slide-79
SLIDE 79

Questions