How How Thinking Thinking in in Python Made Me a Better Python - - PowerPoint PPT Presentation

▶
how how thinking thinking in in python made me a better
SMART_READER_LITE
LIVE PREVIEW

How How Thinking Thinking in in Python Made Me a Better Python - - PowerPoint PPT Presentation

Dec 10, 2022 233 likes •1.35k views

How How Thinking Thinking in in Python Made Me a Better Python Made Me a Better Soware Engineer Soware Engineer EuroPython 2019 Johnny Dude bit.ly/ThinkPy Hi, I'm Johnny Dude Hi, I'm Johnny Dude Soware Engineer at TogaNetworks

slide-1
SLIDE 1

How How Thinking Thinking in in Python Made Me a Better Python Made Me a Better Soware Engineer Soware Engineer

EuroPython 2019 Johnny Dude

bit.ly/ThinkPy

slide-2
SLIDE 2

Hi, I'm Johnny Dude Hi, I'm Johnny Dude

Soware Engineer at Using Python at work since 2005 I use Python for prototyping Responsible for c++ production code This is my first EuroPython talk TogaNetworks

bit.ly/ThinkPy

slide-3
SLIDE 3

Outline Outline

  • 1. Psychological concepts
  • 2. Relation to the development process
  • 3. Experiment
slide-4
SLIDE 4

Psychological Concepts Psychological Concepts

slide-5
SLIDE 5
slide-6
SLIDE 6

Trying not to think about something, makes thinking about it more likely

* Ironic Process Theory

slide-7
SLIDE 7

The number of objects an average human can hold in working memory is 7 ± 2

* The Magical Number Seven, Plus or Minus Two

slide-8
SLIDE 8

Anything that occupies your working memory reduces your ability to think

* Thinking Fast and Slow

slide-9
SLIDE 9

Capital of France

slide-10
SLIDE 10

Priming is a technique whereby exposure to one stimulus influences a response to a subsequent stimulus, without conscious guidance or intention

* Thinking Fast and Slow * The Priming Effect

slide-11
SLIDE 11

You cannot prevent it

slide-12
SLIDE 12

Task switching reduces your productivity time

* Executive Control of Cognitive Processes in Task Switching

slide-13
SLIDE 13

Fluency is the ability to do an activity with little, or no conscious effort

slide-14
SLIDE 14
slide-15
SLIDE 15

Relation to the Relation to the Development Process Development Process

slide-16
SLIDE 16

Immediate Feedback Immediate Feedback

slide-17
SLIDE 17
slide-18
SLIDE 18

def get_biggest_files(n, path='.'): lines = system(f'du -a {path}').splitlines() pairs = [line.split('\t') for line in lines] return [name for size, name in nlargest(n, pairs)]

slide-19
SLIDE 19

>>> n, path = 2, 'small folder' >>> lines = system(f'du -a {path}').splitlines() >>> lines[:2] ['8\t./darker.css', '32\t./index.html']

slide-20
SLIDE 20

>>> n, path = 2, 'small folder' >>> lines = system(f'du -a {path}').splitlines() >>> lines[:2] ['8\t./darker.css', '32\t./index.html'] >>> pairs = [line.split('\t') for line in lines] >>> pairs[:2] [['8', './darker.css'], ['32', './index.html']]

slide-21
SLIDE 21

>>> n, path = 2, 'small folder' >>> lines = system(f'du -a {path}').splitlines() >>> lines[:2] ['8\t./darker.css', '32\t./index.html'] >>> pairs = [line.split('\t') for line in lines] >>> pairs[:2] [['8', './darker.css'], ['32', './index.html']] >>> nlargest(n, pairs) [['96', './.git/objects/d1/31af6800b725b05b...'], ['912', './images/repr.jpg'], ['900', './.git/objects/20']]

slide-22
SLIDE 22

Learn faster

slide-23
SLIDE 23

Catching bugs earlier reduces task switching

slide-24
SLIDE 24

Confidence that your code works,

slide-25
SLIDE 25

Confidence that your code works, without concious effort

slide-26
SLIDE 26
slide-27
SLIDE 27

Standard Representation Standard Representation

slide-28
SLIDE 28

[ { "name": "Tyler Durden", "age": 35, "sibling":[]}, { "name": "Brad Pitt", "age": 56, "sibling":["Doug", "Julie"]}, { "name": "Mia Wallace", "age": 25, "sibling":[]}, { "name": "Uma Thurman", "age": 49, "sibling":[ "Dechen", "Taya", "Ganden", "Mipam"]}, ]

slide-29
SLIDE 29

A list of strings, optimized for filtering items matching a regular expression

slide-30
SLIDE 30

A list of strings, optimized for filtering items matching a regular expression

"Tyler Durden\nBrad Pitt\nMia Wallace\nUma Thurman\n"

slide-31
SLIDE 31

A list of strings, optimized for filtering items matching a regular expression

"Tyler Durden\nBrad Pitt\nMia Wallace\nUma Thurman\n" ["Tyler Durden", "Brad Pitt", "Mia Wallace", "Uma Thurman"]

slide-32
SLIDE 32

A dictionary with keys that can be searched by regular expression.

"Brad Pitt\nMia Wallace\nTyler Durden\nUma Thurman\n" { 22: "Dead", 35: "Alive", 0: "Alive", 10: "Alive" }

slide-33
SLIDE 33

A dictionary with keys that can be searched by regular expression.

"Brad Pitt\nMia Wallace\nTyler Durden\nUma Thurman\n" { 22: "Dead", 35: "Alive", 0: "Alive", 10: "Alive" } { "Tyler Durden": "Dead", "Uma Thurman": "Alive", "Brad Pitt": "Alive", "Mia Wallace": "Alive" }

slide-34
SLIDE 34

(gdb) p my_dict $1 = { keys = "Brad Pitt\nMia Wallace\nTyler Durden\nUma T hurman\n", hash_table = std::unordered_map with 4 elements = { [10] = PersonState::alive, [0] = PersonState::alive, [35] = PersonState::alive, [22] = PersonState::dead}}

slide-35
SLIDE 35

{ "Tyler Durden": <PersonState object at 0x7fd2622fbd60>, "Uma Thurman": <PersonState object at 0x7fd2622fbc40>, "Brad Pitt": <PersonState object at 0x7fd26231a160>, "Mia Wallace": <PersonState object at 0x7fd26231a100> }

slide-36
SLIDE 36

{ "Tyler Durden": <PersonState object at 0x7fd2622fbd60>, "Uma Thurman": <PersonState object at 0x7fd2622fbc40>, "Brad Pitt": <PersonState object at 0x7fd26231a160>, "Mia Wallace": <PersonState object at 0x7fd26231a100> } { "Tyler Durden": PersonState(0), "Uma Thurman": PersonState(1), "Brad Pitt": PersonState(1), "Mia Wallace": PersonState(1) }

slide-37
SLIDE 37

If you can read it then you can visualize it, think about it, and discuss it with other developers

slide-38
SLIDE 38

Standard API Standard API

slide-39
SLIDE 39

Counter({ "Walking Dead": 19, "Alive": 7, "Dead": 2, "Not Born": 1, })

slide-40
SLIDE 40

{ } { }

slide-41
SLIDE 41

I want to store something in a dictionary...

slide-42
SLIDE 42

I want to store something in a std::map...

slide-43
SLIDE 43

Composability Composability

slide-44
SLIDE 44

def f(nums): return [str(n) for n in sorted(nums) if valid(n)]

slide-45
SLIDE 45

def f(nums): return [str(n) for n in sorted(nums) if valid(n)] def f(nums): xs = list(nums) sort(xs) ys = filter(valid, xs) zs = map(str, ys) return zs

slide-46
SLIDE 46

Mix the ingredients in a bowl. Pour the bowl’s contents into a mould. Bake the mould along with its content.

slide-47
SLIDE 47

Mix the ingredients in a bowl. Pour the bowl’s contents into a mould. Bake the mould along with its content. Bake the mixed ingredients.

slide-48
SLIDE 48

def f(nums): return [str(n) for n in sorted(nums) if valid(n)] def f(nums): xs = list(nums) sort(xs) ys = filter(valid, xs) zs = map(str, ys) return zs

slide-49
SLIDE 49

vector<string> f(const vector<int>& nums) { vector<int> xs = nums; sort(xs.begin(), xs.end()); vector<int> ys; copy_if( xs.begin(), xs.end(), back_inserter(ys), valid ); vector<string> zs; transform( ys.begin(), ys.end(), back_inserter(zs), [](int n){ return to_string(n); } ); return zs; }

slide-50
SLIDE 50

It is easy to think with composable tools

slide-51
SLIDE 51

Simple is better than Complicated Simple is better than Complicated

slide-52
SLIDE 52

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer

slide-53
SLIDE 53

Simple is better than complex Simple is better than complex

slide-54
SLIDE 54

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer

slide-55
SLIDE 55

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer

void f(shared_ptr<Object> or unique_ptr<Object> obj) // ??

slide-56
SLIDE 56

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer void f(shared_ptr<Object> or unique_ptr<Object> obj) // ??

void f(const Object* obj) // object is immutable void f(Object* const obj) // pointer is immutable void f(const Object* const obj) // both are immutable

slide-57
SLIDE 57

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer void f(shared_ptr<Object> or unique_ptr<Object> obj) // ?? void f(const Object* obj) // object is immutable void f(Object* const obj) // pointer is immutable void f(const Object* const obj) // both are immutable

void f(Object const* obj) // what is immutable?

slide-58
SLIDE 58

Complex is better than complicated Complex is better than complicated

slide-59
SLIDE 59

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer void f(shared_ptr<Object> or unique_ptr<Object> obj) // ?? void f(const Object* obj) // object is immutable void f(Object* const obj) // pointer is immutable void f(const Object* const obj) // both are immutable void f(Object const* obj) // what is immutable?

slide-60
SLIDE 60

void f(shared_ptr<Object>& obj) // pass shared pointer // by reference

void f(Object obj) // pass by value void f(Object& obj) // pass by reference void f(Object* obj) // pass by raw pointer void f(Object&& obj) // pass by rvalue void f(shared_ptr<Object> obj) // pass by shared pointer void f(unique_ptr<Object> obj) // pass by unique pointer void f(shared_ptr<Object> or unique_ptr<Object> obj) // ?? void f(const Object* obj) // object is immutable void f(Object* const obj) // pointer is immutable void f(const Object* const obj) // both are immutable void f(Object const* obj) // what is immutable?

slide-61
SLIDE 61

We can use shared pointers everywhere But, we cannot stop thinking about...

slide-62
SLIDE 62

Type Hints Type Hints

slide-63
SLIDE 63
slide-64
SLIDE 64

Do we realy want to define types and structures before understanding the problem and the solution?

slide-65
SLIDE 65

Constantly task switching between: Coding and Type-defining

slide-66
SLIDE 66

How many bits would I like this integer to have?

slide-67
SLIDE 67

What happens when you are wrong?

slide-68
SLIDE 68

Lets just use int, and deal with it later.

slide-69
SLIDE 69
slide-70
SLIDE 70

Prototyping Prototyping

slide-71
SLIDE 71

Prototype is a model built to test a concept, and to be learned from

slide-72
SLIDE 72

You write it once, gaining experience in both understanding the problem, and understanding a solution

slide-73
SLIDE 73

You write it again, with less things to worry about and attention to finer details

slide-74
SLIDE 74

Improved Readability Improved Maintainability Fewer Bugs

slide-75
SLIDE 75

Coding Prototyping Coding

slide-76
SLIDE 76

Coding Prototyping Coding Future Tasks Future Tasks

slide-77
SLIDE 77

Some things you can do

  • nly in Python
slide-78
SLIDE 78

Use a dictionary Define a function

slide-79
SLIDE 79

Think in the language you write

slide-80
SLIDE 80

Handle type checking, seperately

slide-81
SLIDE 81

Along with many other reasons.

slide-82
SLIDE 82

c++ Coding c++ Coding

Python Prototyping

Future Tasks Future Tasks

An empirical comparison of c, c++, java, perl, python, ...

slide-83
SLIDE 83

How much of the speedup do we get from thinking faster?

slide-84
SLIDE 84

My Experiment My Experiment

slide-85
SLIDE 85

+--+--+--+--+--+--+--+--+--+ +--+--+--+--+--+--+--+--+--+ |S | |Sooooooooooooooooooo | + +--+--+--+--+ + +--+ + + +--+--+--+--+ + o+--+ + | | | | | | | | | | ooo | | +--+ + + + +--+--+ + + +--+ + + + +--+--+o + + | | | | | | | | | | oooooo | | + +--+ +--+--+ +--+--+ + + +--+ +--+--+ o+--+--+ + | | | | | | | | | oooo| | +--+ +--+--+ +--+ + +--+ +--+ +--+--+ o+--+ +--+--+ | | | | | | ooo | | + +--+ +--+--+ +--+--+ + + +--+ +--+--+o +--+--+ + | |E | | |E | +--+--+--+--+--+--+--+--+--+ +--+--+--+--+--+--+--+--+--+

slide-86
SLIDE 86

Python c++ Python c++ Python

# 1 # 2 # 3 # 4 # 5

slide-87
SLIDE 87

Python c++ Python c++ Python

# 1 # 2 # 3 # 4 # 5 22 42 14 30 151

Work Time in Minutes

slide-88
SLIDE 88

Python c++ Python c++ Python

#1 #2 #3 #4 #5 22 42 14 30 151 64 50 54 74 108

Work Time in Minutes Source Lines of Code

Excluding: comment, empty lines, bracelets

slide-89
SLIDE 89

c++ Python

#4 #5 14 30 64 50

Work Time in Minutes Source Lines of Code

Excluding: comment, empty lines, bracelets

slide-90
SLIDE 90

Algorithm Data Types Funtions Names

14 30 64 50

Work Time in Minutes Source Lines of Code

Excluding: comment, empty lines, bracelets

Both version have exactly the same

slide-91
SLIDE 91

Algorithm Data Types Funtions Names

14 30 64 50

Work Time in Minutes Source Lines of Code

Excluding: comment, empty lines, bracelets

Both version have exactly the same

28

excluding extra time for typing

slide-92
SLIDE 92

set<Point> calc_path(map<Point, Point> prevs, Point point) { set<Point> results; point = prevs[point]; while (prevs.find(point) != prevs.end()) { results.insert(point); point = prevs[point]; } return results; } auto points = calc_path(prevs, end_point); def calc_path(prevs, point): point = prevs[point] while point in prevs: yield point point = prevs[point] points = set(calc_path(prevs, end_point))

slide-93
SLIDE 93

Why? Why?

slide-94
SLIDE 94

What was I Thinking About?

slide-95
SLIDE 95
slide-96
SLIDE 96

Experiment, Experiment, it's fun it's fun

slide-97
SLIDE 97

Python c++ Python c++ Python

#1 #2 #3 #4 #5 22 42 14 30 151 64 50 54 74 108

Work Time in Minutes Source Lines of Code

Excluding: comment, empty lines, bracelets

slide-98
SLIDE 98

Summary Summary

Immediate Feedback Standard Representation & API Composability Prototype in Python

slide-99
SLIDE 99

Summary Summary

Standard Representation & API Composability Prototype in Python

Immediate Feedback

slide-100
SLIDE 100

Summary Summary

Composability Prototype in Python

Immediate Feedback

Standard Representation & API

slide-101
SLIDE 101

Summary Summary

Prototype in Python

Immediate Feedback Standard Representation & API

Composability

slide-102
SLIDE 102

Summary Summary

Immediate Feedback Standard Representation & API Composability

Prototype in Python

slide-103
SLIDE 103

Think about the way you think Think in Python Experiment, it's fun!

slide-104
SLIDE 104

Think in Python Experiment, it's fun!

Think about the way you think

slide-105
SLIDE 105

Experiment, it's fun!

Think about the way you think

Think in Python

slide-106
SLIDE 106

Think about the way you think Think in Python

Experiment, it's fun!

slide-107
SLIDE 107

Think in Python

slide-108
SLIDE 108

Thank You Thank You

Twitter: @DudeJohnny1219 email: johnny.dude@gmail.com

bit.ly/ThinkPy

slide-109
SLIDE 109

Many thanks to those who made this talk possible, Many thanks to those who made this talk possible, without thier help this talk might not be worth listening to. without thier help this talk might not be worth listening to.

Nobuko Sano (佐野信⼦), Elizabeth Firman,

Michael Hirsch, Aharon Broduch, Eran Galon, Kobi and Suzi Lidershnider,

Boris Liberman, Ariel Weinstein, Omer Anson, Eran Galon, Aviv Kuvent, Eddy

Duer, Meital Bar-Kana, Yaron Mor

slide-110
SLIDE 110

References and Inspirations References and Inspirations

Bret Victor gave an amazing talk ("Inventing on Principle") in which he mentioned immidiate reaction. I first learned about the magical number 7 from the famous post of Glyph about threading module complexities. Alan Kay have many talks explaining science, human, machines, learning, teaching, and combining it all together. I actually started reading "Thinking fast and slow" of Daniel Kahneman last month and decided to take a couple ofcouple of very good points from the first half

  • f this book.

One of the papers about task switching I happend to find. It talks about many interesting experiments on task switiching Rubinstein, J. S., Meyer, D. E. & Evans, J. E. (2001). Executive Control of Cognitive Processes in Task Switching. Journal of Experimental Psychology: Human Perception and Performance, 27, 763-797. The empirical research I like, it shows a lot of measurements comparing programing languages, it is old, but I believe nothing major changed since then. There are other reseach that shows similar results in different domains, and with different methods. (like analyzing github repositories.) Prechelt, Lutz. (2000). An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program. https://vimeo.com/36579366 https://glyph.twistedmatrix.com/2014/02/unyielding.html https://www.apa.org/pubs/journals/releases/xhp274763.pdf https://page.mi.fu-berlin.de/prechelt/Biblio/jccpprtTR.pdf