ctypes Direct access to happiness.dll Mike C. Fletcher VRPlumber - - PowerPoint PPT Presentation

▶
ctypes
SMART_READER_LITE
LIVE PREVIEW

ctypes Direct access to happiness.dll Mike C. Fletcher VRPlumber - - PowerPoint PPT Presentation

May 06, 2023 29 likes •719 views

ctypes Direct access to happiness.dll Mike C. Fletcher VRPlumber Consulting Inc. Who is this guy PyOpenGL lead developer (other stuff too) OpenGLContext, SimpleParse, StarPy, TTFQuery, BasicProperty, PyDispatcher (and a consultant on

slide-1
SLIDE 1

Mike C. Fletcher – VRPlumber Consulting Inc.

ctypes

Direct access to happiness.dll

slide-2
SLIDE 2

Mike C. Fletcher – VRPlumber Consulting Inc.

Who is this guy

PyOpenGL lead developer (other stuff too) OpenGLContext, SimpleParse, StarPy, TTFQuery, BasicProperty, PyDispatcher (and a consultant on VoIP and other stuff)

slide-3
SLIDE 3

Mike C. Fletcher – VRPlumber Consulting Inc.

What I've done lately

Rewrote PyOpenGL from SWIG to ctypes (Why?)

slide-4
SLIDE 4

Mike C. Fletcher – VRPlumber Consulting Inc.

Python will be faster

slide-5
SLIDE 5

Mike C. Fletcher – VRPlumber Consulting Inc.

Because it must

The language is fine We need speed to expand into new areas (e.g. games)

slide-6
SLIDE 6

Mike C. Fletcher – VRPlumber Consulting Inc.

Or PyPy will replace it

Or IronPython Or OLPC-Python Or Parrot Or something else...

slide-7
SLIDE 7

Mike C. Fletcher – VRPlumber Consulting Inc.

(And no...)

We really don't care about a new print syntax

slide-8
SLIDE 8

Mike C. Fletcher – VRPlumber Consulting Inc.

As Python accellerates

We can think about replacing C (Particularly C extensions)

slide-9
SLIDE 9

Mike C. Fletcher – VRPlumber Consulting Inc.

Extensions were about

Speed (zoom, zoom)

slide-10
SLIDE 10

Mike C. Fletcher – VRPlumber Consulting Inc.

But also

Pre-written libraries of code

slide-11
SLIDE 11

Mike C. Fletcher – VRPlumber Consulting Inc.

When sPython ~= sC

(Get working on this peoples)

slide-12
SLIDE 12

Mike C. Fletcher – VRPlumber Consulting Inc.

We still need

Access to pre-written libraries of code (From pure Python)

slide-13
SLIDE 13

Mike C. Fletcher – VRPlumber Consulting Inc.

Lucky we already have it

It's been around for years

slide-14
SLIDE 14

Mike C. Fletcher – VRPlumber Consulting Inc.

Hacker's ctypes (old school)

>>> import ctypes >>> happiness = ctypes.cdll.LoadLibrary( './happiness.so' ) >>> happiness.hello( 'Hello world %i\n', 42 ) Hello world 42 15 # return value

slide-15
SLIDE 15

Mike C. Fletcher – VRPlumber Consulting Inc.

Yawn

(In a mind blowingly “cool” sort of way)

slide-16
SLIDE 16

Mike C. Fletcher – VRPlumber Consulting Inc.

That is so 2004

Just a hacker's toy

slide-17
SLIDE 17

Mike C. Fletcher – VRPlumber Consulting Inc.

But it was COOL

We could poke deep in the machine Twiddle random bits Make things happen

slide-18
SLIDE 18

Mike C. Fletcher – VRPlumber Consulting Inc.

Um, we said “Yawn”

Hacker's backwater for years (No-one really cared)

slide-19
SLIDE 19

Mike C. Fletcher – VRPlumber Consulting Inc.

What's different?

Standard library inclusion

slide-20
SLIDE 20

Mike C. Fletcher – VRPlumber Consulting Inc.

What's different?

Automated code generation

slide-21
SLIDE 21

Mike C. Fletcher – VRPlumber Consulting Inc.

What's different?

PyPy support

slide-22
SLIDE 22

Mike C. Fletcher – VRPlumber Consulting Inc.

What's different?

Numpy Support

slide-23
SLIDE 23

Mike C. Fletcher – VRPlumber Consulting Inc.

What's different?

Bigger projects possible

slide-24
SLIDE 24

Mike C. Fletcher – VRPlumber Consulting Inc.

Bigger, you say?

Comtypes Pygame-ctypes Pyglet PyOpenGL

slide-25
SLIDE 25

Mike C. Fletcher – VRPlumber Consulting Inc.

PyOpenGL Scale

2189 C functions 3475 constant definitions 262 extension modules 6 possible core versions

slide-26
SLIDE 26

Mike C. Fletcher – VRPlumber Consulting Inc.

Where we came from

1.x Manual Wrapping 2.x SWIG Wrapping with custom distutils

slide-27
SLIDE 27

Mike C. Fletcher – VRPlumber Consulting Inc.

1.x died years ago

Manual wrapping way too time consuming (This is where I came into the picture, life support for a dying project)

slide-28
SLIDE 28

Mike C. Fletcher – VRPlumber Consulting Inc.

Really

No one wants to code in C

slide-29
SLIDE 29

Mike C. Fletcher – VRPlumber Consulting Inc.

The promised land

SWIG (Tarn decided to use this to rewrite the project)

slide-30
SLIDE 30

Mike C. Fletcher – VRPlumber Consulting Inc.

Macro headaches

Level upon levels of macro expansion SWIG typemaps, SWIG macros, macros, macro- expanded utility libraries (Easy to write, extremely difficult to maintain)

slide-31
SLIDE 31

Mike C. Fletcher – VRPlumber Consulting Inc.

C compilation problems

Complex build process Edit/compile/run cycle of 20 minutes+ Togl build procedure constantly broken No compiler on one platform

slide-32
SLIDE 32

Mike C. Fletcher – VRPlumber Consulting Inc.

Developer fatigue

No one wanted to do the day-to-day stuff (Because it was such a pain)

slide-33
SLIDE 33

Mike C. Fletcher – VRPlumber Consulting Inc.

Few joined, none stayed

Figuring out how to start was way too hard

slide-34
SLIDE 34

Mike C. Fletcher – VRPlumber Consulting Inc.

So I was going to dump PyOpenGL

Wasn't enough fun to spend my free time on it

slide-35
SLIDE 35

Mike C. Fletcher – VRPlumber Consulting Inc.

Won't someone think of the users?

70+ downloads a day (outside the distributions or applications)

slide-36
SLIDE 36

Mike C. Fletcher – VRPlumber Consulting Inc.

Won't someone think of the users?

Hundreds, maybe thousands of applications (Science, extension systems, graphic libraries)

slide-37
SLIDE 37

Mike C. Fletcher – VRPlumber Consulting Inc.

We should try ctypes

(Rene suggested it IIRC)

slide-38
SLIDE 38

Mike C. Fletcher – VRPlumber Consulting Inc.

First tests

Could create C-like API easily with custom (hacky) auto-generation

slide-39
SLIDE 39

Mike C. Fletcher – VRPlumber Consulting Inc.

But it wasn't compatible

So I ignored it for a while (And things got worse)

slide-40
SLIDE 40

Mike C. Fletcher – VRPlumber Consulting Inc.

Second tests

Not “can we wrap OpenGL” “Can we create PyOpenGL”

slide-41
SLIDE 41

Mike C. Fletcher – VRPlumber Consulting Inc.

The goal

Fully compatible with PyOpenGL 2.x (reasonably compatible) With half a dozen new fixes/features Full extension coverage

slide-42
SLIDE 42

Mike C. Fletcher – VRPlumber Consulting Inc.

A few bugs early on

AMD64 platform issues, mostly

slide-43
SLIDE 43

Mike C. Fletcher – VRPlumber Consulting Inc.

But we could make it work

So we did

slide-44
SLIDE 44

Mike C. Fletcher – VRPlumber Consulting Inc.

Array handling

3 different array systems (now pluggable) Fairly trivial, it turns out (Even easier now)

slide-45
SLIDE 45

Mike C. Fletcher – VRPlumber Consulting Inc.

No high-level automation

No automated type/name matching (Yay, wrote it in Python)

slide-46
SLIDE 46

Mike C. Fletcher – VRPlumber Consulting Inc.

Library loading

Platform specific (no big deal) Needs to be available (hmm) Needs to be dynamic library

slide-47
SLIDE 47

Mike C. Fletcher – VRPlumber Consulting Inc.

No C++

We don't care, others will

slide-48
SLIDE 48

Mike C. Fletcher – VRPlumber Consulting Inc.

Macro problems

We don't have a lot of them Just hacked around them in Python

slide-49
SLIDE 49

Mike C. Fletcher – VRPlumber Consulting Inc.

Speed problems

2-5x slower (Delicious irony of slowing down to speed up)

slide-50
SLIDE 50

Mike C. Fletcher – VRPlumber Consulting Inc.

Documentation

Not as extensive as you'd want

slide-51
SLIDE 51

Mike C. Fletcher – VRPlumber Consulting Inc.

What we got

slide-52
SLIDE 52

Mike C. Fletcher – VRPlumber Consulting Inc.

No C coding

No one wants to code in C

slide-53
SLIDE 53

Mike C. Fletcher – VRPlumber Consulting Inc.

Lower barrier to entry

Hack from day one

slide-54
SLIDE 54

Mike C. Fletcher – VRPlumber Consulting Inc.

Easier to contribute

Developer's “hacks” integrate easily

slide-55
SLIDE 55

Mike C. Fletcher – VRPlumber Consulting Inc.

Easier to install

easy_install PyOpenGL

slide-56
SLIDE 56

Mike C. Fletcher – VRPlumber Consulting Inc.

Easier to build

(do nothing)

slide-57
SLIDE 57

Mike C. Fletcher – VRPlumber Consulting Inc.

Easier to debug

Walk through the whole wrapping process

slide-58
SLIDE 58

Mike C. Fletcher – VRPlumber Consulting Inc.

More coverage

Core library 1.3 through 2.0 (automatic) All registered extensions Silly little regex script creates them (Pyglet guys have a more advanced wrapper)

slide-59
SLIDE 59

Mike C. Fletcher – VRPlumber Consulting Inc.

New features

Pluggable data-format support Optional logged operation Run-time binding (dll substitution)

slide-60
SLIDE 60

Mike C. Fletcher – VRPlumber Consulting Inc.

Compatible

Some users don't even realise it's a new technology, it's just “3.0”

slide-61
SLIDE 61

Mike C. Fletcher – VRPlumber Consulting Inc.

We're future-facing again

Type inferencing ready

slide-62
SLIDE 62

Mike C. Fletcher – VRPlumber Consulting Inc.

We're future-facing again

PyPy ready

slide-63
SLIDE 63

Mike C. Fletcher – VRPlumber Consulting Inc.

We're future-facing again

Numpy compatible (pluggable data-types throughout)

slide-64
SLIDE 64

Mike C. Fletcher – VRPlumber Consulting Inc.

I'm motivated again

Development is fun

slide-65
SLIDE 65

Mike C. Fletcher – VRPlumber Consulting Inc.

Why ctypes?

You have libraries in C you want to use (Loadable libraries without many macros)

slide-66
SLIDE 66

Mike C. Fletcher – VRPlumber Consulting Inc.

Why ctypes?

You want to code in Python (No-one wants to code in C)

slide-67
SLIDE 67

Mike C. Fletcher – VRPlumber Consulting Inc.

Why ctypes?

You want to avoid users having to compile your code (Drop a pure-Python .egg)

slide-68
SLIDE 68

Mike C. Fletcher – VRPlumber Consulting Inc.

You want to code Python

Because Python is fun (and no-one wants to code in C)

slide-69
SLIDE 69

Mike C. Fletcher – VRPlumber Consulting Inc.

Why ctypes?

Because you want to load happiness.dll into your namespace