Automating Software Development with Deep Learning Emil Wallner, - - PowerPoint PPT Presentation

▶
automating software development with deep learning
SMART_READER_LITE
LIVE PREVIEW

Automating Software Development with Deep Learning Emil Wallner, - - PowerPoint PPT Presentation

Apr 17, 2023 662 likes •1.21k views

Automating Software Development with Deep Learning Emil Wallner, 42, QCon So Paulo, 2019-05-7 01001001001010101010100111110110101010101000001010111010101010111101010111

slide-1
SLIDE 1

Automating Software Development with Deep Learning

Emil Wallner, 42, QCon São Paulo, 2019-05-7

slide-2
SLIDE 2
slide-3
SLIDE 3

Software Development as Data

01001001001010101010100111110110101010101000001010111010101010111101010111 00011100011011001100101010010101000101010100101010010101010100101010101010 10001010101010000000001111111010101111101000101010101010010010010010101010 10100111110110101010101000001010111010101010111101010111000111000110110011 00101010010101000101010100101010010101010100101010101010100010101010100000 00001111111010101111101000101010101010010010010010101010101001111101101010 10101000001010111010101010111101010111000111000110110011001010100101010001 01010100101010010101010100101010101010100010101010100000000011111110101011 11101000101010101010010010010010101010101001111101101010101010000010101110 10101010111101010111000111000110110011001010100101010001010101001010100101 01010100101010101010100010101010100000000011111110101011111010001010101010 10010010010010101010101001111101101010101010000010101110101010101111010101 11000111000110110011001010100101010001010101001010100101010101001010101010 10100010101010100000000011111110101011111010001010101010100100100100101010 10101001111101101010101010000010101110101010101111010101110001110001101100 11001010100101010001010101001010100101010101001010101010101000101010101000 00000011111110101011111010001010101010100100100100101010101010011111011010 10101010000010101110101010101111010101110001110001101100110010101001010100 01010101001010100101010101001010101010101000101010101000000000111111101010 11111010001010101010100100100100101010101010011111011010101010100000101011 10101010101111010101110001110001101100110010101001010100010101010010101001 01010101001010101010101000101010101000000000111111101010111110100010101010 10100100100100101010101010011111011010101010100000101011101010101011110101 01110001110001101100110010101001010100010101010010101001010101010010101010 10101000101010101000000000111111101010111110100010101010101001001001001010 10101010011111011010101010100000101011101010101011110101011100011100011011 00110010101001010100010101010010101001010101010010101010101010001010101010 00000000111111101010111110100010101010101001001001001010101010100111110110 10101010100000101011101010101011110101011100011100011011001100101010010101 00010101010010101001010101010010101010101010001010101010000000001111111010 10111110100010101010101001001001001010101010100111110110101010101000001010 11101010101011110101011100011100011011001100101010010101000101010100101010 01010101010010101010101010001010101010000000001111111010101111101000101010 10101001001001001010101010100111110110101010101000001010111010101010111101 01011100011100011011001100101010010101000101010100101010010101010100101010 10101010001010101010000000001111111010101111101000101010101010010010010010 10101010100111110110101010101000001010111010101010111101010111000111000110
slide-4
SLIDE 4

Software 1.0 - Traditional Software 2.0 - Deep learning Software 2.5 - Blended models

slide-5
SLIDE 5

1.0

slide-6
SLIDE 6

2.0

slide-7
SLIDE 7

Source: Andrej Karpathy on Software 2.0

slide-8
SLIDE 8
slide-9
SLIDE 9

Software engineers are becoming data scientists

slide-10
SLIDE 10

Can we automate software development 1.0 with 2.0 software?

slide-11
SLIDE 11
slide-12
SLIDE 12

2.5

slide-13
SLIDE 13

Researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation.

  • Richard S. Sutton
slide-14
SLIDE 14

1.0

2.0

2.5

Complexity

slide-15
SLIDE 15

Assisted Software Development A u t

  • m

a t i

  • n
  • f

c l

  • s

e d s y s t e m s Automation of integrated systems

Novel Manifolds/Scalability Time

Today

slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18

Understand an image Generate semantically correct text Correlate objects in images with text

slide-19
SLIDE 19

Convolutional Neural Networks Recurrent Neural Networks Image Captioning Models

slide-20
SLIDE 20
slide-21
SLIDE 21
slide-22
SLIDE 22

Start <HTML> <center> <H1> Hello World! </H1> </center> </HTML> End

slide-23
SLIDE 23
slide-24
SLIDE 24

Core structure: Pix2code (Tony Beltramelli, 2017)

slide-25
SLIDE 25
slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28
slide-29
SLIDE 29
slide-30
SLIDE 30
slide-31
SLIDE 31

GPT-2

slide-32
SLIDE 32
slide-33
SLIDE 33
slide-34
SLIDE 34

Source: Thomas Leiterman using UIzard

slide-35
SLIDE 35

Software Development as Data

01001001001010101010100111110110101010101000001010111010101010111101010111 00011100011011001100101010010101000101010100101010010101010100101010101010 10001010101010000000001111111010101111101000101010101010010010010010101010 10100111110110101010101000001010111010101010111101010111000111000110110011 00101010010101000101010100101010010101010100101010101010100010101010100000 00001111111010101111101000101010101010010010010010101010101001111101101010 10101000001010111010101010111101010111000111000110110011001010100101010001 01010100101010010101010100101010101010100010101010100000000011111110101011 11101000101010101010010010010010101010101001111101101010101010000010101110 10101010111101010111000111000110110011001010100101010001010101001010100101 01010100101010101010100010101010100000000011111110101011111010001010101010 10010010010010101010101001111101101010101010000010101110101010101111010101 11000111000110110011001010100101010001010101001010100101010101001010101010 10100010101010100000000011111110101011111010001010101010100100100100101010 10101001111101101010101010000010101110101010101111010101110001110001101100 11001010100101010001010101001010100101010101001010101010101000101010101000 00000011111110101011111010001010101010100100100100101010101010011111011010 10101010000010101110101010101111010101110001110001101100110010101001010100 01010101001010100101010101001010101010101000101010101000000000111111101010 11111010001010101010100100100100101010101010011111011010101010100000101011 10101010101111010101110001110001101100110010101001010100010101010010101001 01010101001010101010101000101010101000000000111111101010111110100010101010 10100100100100101010101010011111011010101010100000101011101010101011110101 01110001110001101100110010101001010100010101010010101001010101010010101010 10101000101010101000000000111111101010111110100010101010101001001001001010 10101010011111011010101010100000101011101010101011110101011100011100011011 00110010101001010100010101010010101001010101010010101010101010001010101010 00000000111111101010111110100010101010101001001001001010101010100111110110 10101010100000101011101010101011110101011100011100011011001100101010010101 00010101010010101001010101010010101010101010001010101010000000001111111010 10111110100010101010101001001001001010101010100111110110101010101000001010 11101010101011110101011100011100011011001100101010010101000101010100101010 01010101010010101010101010001010101010000000001111111010101111101000101010 10101001001001001010101010100111110110101010101000001010111010101010111101 01011100011100011011001100101010010101000101010100101010010101010100101010 10101010001010101010000000001111111010101111101000101010101010010010010010 10101010100111110110101010101000001010111010101010111101010111000111000110
slide-36
SLIDE 36

Progress Time

slide-37
SLIDE 37

Image recognition Image generation Voice recognition Board games Multiplayer games Text understanding Translation Self-driving Cars Medical Image Analysis Protein Folding Handwriting Fraud detection

slide-38
SLIDE 38

Progress Time

slide-39
SLIDE 39

Assisted Software Development A u t

  • m

a t i

  • n
  • f

c l

  • s

e d s y s t e m s Automation of integrated systems

Novel Manifolds/Scalability Time

Today

slide-40
SLIDE 40

Complexity Empathy Creativity Critical thinking

Undefined Tasks Novel Manifolds Scalability

slide-41
SLIDE 41
slide-42
SLIDE 42

1.0

2.0

2.5

Complexity

slide-43
SLIDE 43

Novel Manifolds Scalability

1.0 2.0 2.5

slide-44
SLIDE 44

Augmented IDE Bank API Social Media App

slide-45
SLIDE 45

Refactoring Autocomplete Code reviewing User testing GUI prototyping Semantic code search Security issues Monitoring

slide-46
SLIDE 46

Novel Manifolds Scalable

slide-47
SLIDE 47

Dynamic pages Linking pages Using Variables Security Rules Integrating databases

slide-48
SLIDE 48

Novel Manifolds Scalable

slide-49
SLIDE 49

API roles Transactions ATM integrations Communication World Model Deep Security Troubleshooting

slide-50
SLIDE 50

Novel Manifolds Scalable

slide-51
SLIDE 51

Functions Paragraphs Static GUI Short Conversations Systems of Programs Documents Advanced GUIs Expert-level Dialogs A program A document Dynamic GUI Conversations with context

slide-52
SLIDE 52

Assisted Software Development A u t

  • m

a t i

  • n
  • f

c l

  • s

e d s y s t e m s Automation of integrated systems

Novel Manifolds/Scalability Time

Today

slide-53
SLIDE 53

emilwallner (github/twitter) w@llner.co