projective geometry aware anisotropic convolutional
play

Projective geometry-aware anisotropic convolutional filters Poster - PowerPoint PPT Presentation

Projective geometry-aware anisotropic convolutional filters Poster @ Pacific Ballroom #147 Renata Khasanova, Pascal Frossard, LTS4, EPFL, Switzerland Motivation Photo credits: https://www.ti.uni-bielefeld.de/html/research/equipment.html


  1. Projective geometry-aware anisotropic convolutional filters Poster @ Pacific Ballroom #147 Renata Khasanova, Pascal Frossard, LTS4, EPFL, Switzerland

  2. Motivation Photo credits: https://www.ti.uni-bielefeld.de/html/research/equipment.html https://www.t3.com/features/best-drone � 2 https://www.zemax.com/blog/zemax-blog/october-2017/getting-to-the-finish-line-faster-optical-technolo

  3. Omnidirectional camera representation ? Spherical surface Rectangular representation � 3

  4. Geometric distortion of equirectangular images � 4

  5. Geometric distortion of cube-map images � 5

  6. Graphs for geometry modelling � 6

  7. Graphs for geometry modelling pixel — node pixel’s intensity — node’s signal � 7

  8. Geometry aware graph filters Idea: adapt filters depending on image location Example of a filter applied to equirectangular projection � 8

  9. Geometry aware graph filters Multiple directed graphs for anisotropic filters Example of filters � 9

  10. <latexit sha1_base64="3zIjD/w+WfbO8SbJCx8EBMPIVNo=">AB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpod3+27Vq3lzkFXiF6QKBRp96s3SFgWc4VMUmO6vpdikFONgk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YXQe5UGmGXLHFoiTBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHy4gjrcQOawGAIz/AKb450Xpx352PRWnKmWP4A+fzB9lbjX8=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> Geometry aware graph filters Multiple directed graphs for anisotropic filters Example of filters Directed graph construction with adjacency matrix W 1 � 10

  11. <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> Geometry aware graph filters Multiple directed graphs for anisotropic filters Example of filters Directed graphs construction � 11

  12. Classification: adaptation to various projective geometries Dataset — projected MNIST Spherical (S) Modified spherical (MS1, MS2, MS3) Fish-eye stereographic projection (F) Cube-map (CM) � 12

  13. Classification results Method S MS1 MS2 MS3 F CM 69.4 64.3 64.1 62.8 71.8 40.0 regular graph (w=1) regular graph (w=1/d) 69.8 63.4 64.5 62.5 70.2 40.5 GA graph (w=1/d) 70.2 63.9 62.5 62.8 72.1 44.2 ConvNets 94.2 91.3 91.2 90.5 93.4 79.4 SphereNet 94.8 — — — — — SphericalCNN 95.2 84.5 83.3 80.9 94.9 — Ours 96.9 95.1 95.3 94.9 95.7 84.3 � 13

  14. Compression Cube-map projection of SUN* dataset with 360-indoor images *https://groups.csail.mit.edu/vision/SUN/ � 14

  15. Decompression challenges � 15

  16. Visual decompression result Original Balle et. al (2017) Ours � 16

  17. Summary � Novel graph construction approach to define a filter, which adapts to the specific geometry of the wide-angle images � Our filters are anisotropic, which permits richer representation � Our filters can be applied to a wider class of tasks compared with standard graph- based filters � Our approach reaches state-of-the-art performance on classification and compression tasks Thank you! Poster: @ Pacific Ballroom #147 Code: https://github.com/RenataKh/GAfilters � 17

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend