Projective geometry-aware anisotropic convolutional filters Poster - - PowerPoint PPT Presentation

▶
projective geometry aware anisotropic convolutional
SMART_READER_LITE
LIVE PREVIEW

Projective geometry-aware anisotropic convolutional filters Poster - - PowerPoint PPT Presentation

Nov 16, 2023 182 likes •361 views

Projective geometry-aware anisotropic convolutional filters Poster @ Pacific Ballroom #147 Renata Khasanova, Pascal Frossard, LTS4, EPFL, Switzerland Motivation Photo credits: https://www.ti.uni-bielefeld.de/html/research/equipment.html

slide-1
SLIDE 1

Projective geometry-aware anisotropic convolutional filters

Poster @ Pacific Ballroom #147 Renata Khasanova, Pascal Frossard, LTS4, EPFL, Switzerland

slide-2
SLIDE 2

Motivation

2

Photo credits: https://www.ti.uni-bielefeld.de/html/research/equipment.html https://www.t3.com/features/best-drone https://www.zemax.com/blog/zemax-blog/october-2017/getting-to-the-finish-line-faster-optical-technolo

slide-3
SLIDE 3

3

?

Spherical surface Rectangular representation

Omnidirectional camera representation

slide-4
SLIDE 4

4

Geometric distortion of equirectangular images

slide-5
SLIDE 5

5

Geometric distortion of cube-map images

slide-6
SLIDE 6

6

Graphs for geometry modelling

slide-7
SLIDE 7

7

pixel — node pixel’s intensity — node’s signal

Graphs for geometry modelling

slide-8
SLIDE 8

8

Idea: adapt filters depending on image location Example of a filter applied to equirectangular projection

Geometry aware graph filters

slide-9
SLIDE 9

9

Example of filters Multiple directed graphs for anisotropic filters

Geometry aware graph filters

slide-10
SLIDE 10

10

<latexit sha1_base64="3zIjD/w+WfbO8SbJCx8EBMPIVNo=">AB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpod3+27Vq3lzkFXiF6QKBRp96s3SFgWc4VMUmO6vpdikFONgk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YXQe5UGmGXLHFoiTBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHy4gjrcQOawGAIz/AKb450Xpx352PRWnKmWP4A+fzB9lbjX8=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit>

Directed graph construction with adjacency matrix W1 Example of filters Multiple directed graphs for anisotropic filters

Geometry aware graph filters

slide-11
SLIDE 11

11

<latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit>

Directed graphs construction Example of filters

<latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit>

Multiple directed graphs for anisotropic filters

Geometry aware graph filters

slide-12
SLIDE 12

Modified spherical (MS1, MS2, MS3)

12

Fish-eye stereographic projection (F) Cube-map (CM) Spherical (S) Dataset — projected MNIST

Classification: adaptation to various projective geometries

slide-13
SLIDE 13

13

Method S MS1 MS2 MS3 F CM regular graph (w=1) 69.4 64.3 64.1 62.8 71.8 40.0 regular graph (w=1/d) 69.8 63.4 64.5 62.5 70.2 40.5 GA graph (w=1/d) 70.2 63.9 62.5 62.8 72.1 44.2 ConvNets 94.2 91.3 91.2 90.5 93.4 79.4 SphereNet 94.8 — — — — — SphericalCNN 95.2 84.5 83.3 80.9 94.9 — Ours 96.9 95.1 95.3 94.9 95.7 84.3

Classification results

slide-14
SLIDE 14

14

Cube-map projection of SUN* dataset with 360-indoor images

*https://groups.csail.mit.edu/vision/SUN/

Compression

slide-15
SLIDE 15

15

Decompression challenges

slide-16
SLIDE 16

Original Ours Balle et. al (2017)

16

Visual decompression result

slide-17
SLIDE 17

17

Novel graph construction approach to define a filter, which adapts to the specific

geometry of the wide-angle images

Our filters are anisotropic, which permits richer representation Our filters can be applied to a wider class of tasks compared with standard graph-

based filters

Our approach reaches state-of-the-art performance on classification and

compression tasks Thank you! Poster: @ Pacific Ballroom #147 Code: https://github.com/RenataKh/GAfilters

Summary