Interfacing Python and C using Ctypes Giuseppe Piero Brandino and - PowerPoint PPT Presentation

Interfacing Python and C using Ctypes Giuseppe Piero Brandino and Jimmy Aguilar Mena March 9, 2016 Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 1 / 12

Motivation When do we want to interface Python and C? To extend Python functionality Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 2 / 12

Motivation When do we want to interface Python and C? To extend Python functionality To improve performance Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 2 / 12

Motivation When do we want to interface Python and C? To extend Python functionality To improve performance To use Python as a glue language Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 2 / 12

Motivation When do we want to interface Python and C? To extend Python functionality To improve performance To use Python as a glue language To create Python bindings for a library Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 2 / 12

Example 1: C vs Python add numbers.c add numbers.py #i n c l u d e < s t d i o . h > t o t a l = 10000000 i n t main ( i n t argc , char ∗∗ argv ) { f o r i i n xrange ( 1 0 ) : i n t i , j , t o t a l ; avg = 0.0 double avg ; f o r j i n xrange ( t o t a l ) : t o t a l = 10000000; avg += j f o r ( i = 0; i < 10; i ++) { avg = avg/ t o t a l avg = 0; f o r ( j = 0 ; j < t o t a l ; j++) { p r i n t ” Average i s { 0 } ” . format ( avg ) avg += j ; } avg = avg/ t o t a l ; } Execute the Python script p r i n t f ( ” Average i s %f \ n” , avg ) ; r e t u r n 0 ; time ./add numbers.py } Compile and execute gcc -O3 add numbers.c -o add numbers.x time ./add numbers.x Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 3 / 12

Example 1: C vs Python add numbers.c add numbers.py #i n c l u d e < s t d i o . h > t o t a l = 10000000 i n t main ( i n t argc , char ∗∗ argv ) { f o r i i n xrange ( 1 0 ) : i n t i , j , t o t a l ; avg = 0.0 double avg ; f o r j i n xrange ( t o t a l ) : t o t a l = 10000000; avg += j f o r ( i = 0; i < 10; i ++) { avg = avg/ t o t a l avg = 0; f o r ( j = 0 ; j < t o t a l ; j++) { p r i n t ” Average i s { 0 } ” . format ( avg ) avg += j ; } avg = avg/ t o t a l ; } Execute the Python script p r i n t f ( ” Average i s %f \ n” , avg ) ; r e t u r n 0 ; time ./add numbers.py } Program time Compile and execute Python: 20.17s gcc -O3 add numbers.c -o C: 0.09s add numbers.x time Yes, C is so much faster!!! ./add numbers.x Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 3 / 12

Example 1: Sometimes you can use Numpy add numbers np.c from numpy import mean , arange Program time t o t a l = 10000000 Python: 20.17s a = arange ( t o t a l ) C: 0.09s Numpy: 0.17s f o r i i n xrange ( 1 0 ) : avg = mean( a ) p r i n t ” Average i s { 0 } ” . format ( avg ) Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 4 / 12

Example 1: Sometimes you can use Numpy add numbers np.c from numpy import mean , arange Program time t o t a l = 10000000 Python: 20.17s a = arange ( t o t a l ) C: 0.09s Numpy: 0.17s f o r i i n xrange ( 1 0 ) : avg = mean( a ) p r i n t ” Average i s { 0 } ” . format ( avg ) Numpy is almost as fast as C because it is written in FORTRAN Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 4 / 12

Integrating python with other languages Python can be interfaced with almost any popular language see: https://wiki.python.org/moin/IntegratingPythonWithOtherLanguages There are too many options for C Python C API Boost Ctypes Swig Cython pybind11 Ctypes Ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 5 / 12

Ctypes types and C types Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 6 / 12

Example 2: Library example2/add.c f l o a t a d d f l o a t ( f l o a t a , f l o a t b) { r e t u r n a + b ; } i n t a d d i n t ( i n t a , i n t b) { r e t u r n a + b ; } Compile and create the library i n t a d d f l o a t r e f ( f l o a t ∗ a , f l o a t ∗ b , f l o a t ∗ c ) { ∗ c = ∗ a + ∗ b ; r e t u r n 0; $ gcc -fPIC -c add.c } $ gcc -fPIC -c arrays.c example2/arrays.c $ gcc -shared add.o arrays.o -o libmymath.so i n t a d d i n t a r r a y ( i n t ∗ a , i n t ∗ b , i n t ∗ c , i n t n) { i n t i ; f o r ( i = 0; i < n ; i ++) { c [ i ] = a [ i ] + b [ i ] ; } r e t u r n 0; } f l o a t dot product ( f l o a t ∗ a , f l o a t ∗ b , i n t n ) { f l o a t r e s =0; i n t i ; f o r ( i = 0; i < n ; i ++) { r e s = r e s + a [ i ] ∗ b [ i ] ; } r e t u r n r e s ; } Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 7 / 12

Example 2: Library example2/add.c f l o a t a d d f l o a t ( f l o a t a , f l o a t b) { r e t u r n a + b ; } i n t a d d i n t ( i n t a , i n t b) { r e t u r n a + b ; } Compile and create the library i n t a d d f l o a t r e f ( f l o a t ∗ a , f l o a t ∗ b , f l o a t ∗ c ) { ∗ c = ∗ a + ∗ b ; r e t u r n 0; $ gcc -fPIC -c add.c } $ gcc -fPIC -c arrays.c example2/arrays.c $ gcc -shared add.o arrays.o -o libmymath.so i n t a d d i n t a r r a y ( i n t ∗ a , i n t ∗ b , i n t ∗ c , i n t n) { i n t i ; f o r ( i = 0; i < n ; i ++) { c [ i ] = a [ i ] + b [ i ] ; In a python interpreter } r e t u r n 0; } import c t y p e s f l o a t dot product ( f l o a t ∗ a , f l o a t ∗ b , i n t n ) { math= c t y p e s . CDLL( ” libmymath . so ” ) f l o a t r e s =0; math . a d d i n t (4 ,5) i n t i ; f o r ( i = 0; i < n ; i ++) { r e s = r e s + a [ i ] ∗ b [ i ] ; } r e t u r n r e s ; } Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 7 / 12

Example 2: Library example2/add.c f l o a t a d d f l o a t ( f l o a t a , f l o a t b) { r e t u r n a + b ; } i n t a d d i n t ( i n t a , i n t b) { r e t u r n a + b ; } Compile and create the library i n t a d d f l o a t r e f ( f l o a t ∗ a , f l o a t ∗ b , f l o a t ∗ c ) { ∗ c = ∗ a + ∗ b ; r e t u r n 0; $ gcc -fPIC -c add.c } $ gcc -fPIC -c arrays.c example2/arrays.c $ gcc -shared add.o arrays.o -o libmymath.so i n t a d d i n t a r r a y ( i n t ∗ a , i n t ∗ b , i n t ∗ c , i n t n) { i n t i ; f o r ( i = 0; i < n ; i ++) { c [ i ] = a [ i ] + b [ i ] ; In a python interpreter } r e t u r n 0; } import c t y p e s f l o a t dot product ( f l o a t ∗ a , f l o a t ∗ b , i n t n ) { math= c t y p e s . CDLL( ” libmymath . so ” ) f l o a t r e s =0; math . a d d i n t (4 ,5) i n t i ; f o r ( i = 0; i < n ; i ++) { r e s = r e s + a [ i ] ∗ b [ i ] ; } r e t u r n r e s ; } Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 7 / 12

Example 2: Arguments But this: math . a d d f l o a t (4 ,5) math . a d d f l o a t ( 4 . 0 , 5 . 0 ) Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 8 / 12

Example 2: Arguments But this: math . a d d f l o a t (4 ,5) math . a d d f l o a t ( 4 . 0 , 5 . 0 ) Will produce an error: ArgumentError Traceback (most recent call last) < ipython-input-9-a461a3162c94 > in < module > () − − − − − > math.add float ( 4.0 , 5.0 ) ArgumentError : argument 1: < class ’TypeError’ > : Don’t know how to convert parameter 1 Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 8 / 12

Example 2: Arguments But this: math . a d d f l o a t (4 ,5) math . a d d f l o a t ( 4 . 0 , 5 . 0 ) We need to specify the correct type for the arguments and the return type: math . a d d f l o a t . r e s t y p e=ctypes . c f l o a t math . a d d f l o a t ( ctypes . c f l o a t ( 4 . 0 ) , ctypes . c f l o a t ( 5 . 0 ) ) Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 8 / 12

Example 2: Arguments But this: math . a d d f l o a t (4 ,5) math . a d d f l o a t ( 4 . 0 , 5 . 0 ) We need to specify the correct type for the arguments and the return type: math . a d d f l o a t . r e s t y p e=ctypes . c f l o a t math . a d d f l o a t ( ctypes . c f l o a t ( 4 . 0 ) , ctypes . c f l o a t ( 5 . 0 ) ) We can also specify argument types once and for all, using argtypes math . a d d f l o a t . r e s t y p e=ctypes . c f l o a t math . a d d f l o a t . a r gt y p e s= [ ctypes . c f l o a t , ctypes . c f l o a t ] math . a d d f l o a t ( 4 . 0 , 5 . 0 ) Giuseppe Piero Brandino and Jimmy Aguilar Mena Interfacing Python and C using Ctypes March 9, 2016 8 / 12