environment agnostic multitask learning for natural
play

Environment-agnostic Multitask Learning for Natural Language - PowerPoint PPT Presentation

Environment-agnostic Multitask Learning for Natural Language Grounded Navigation Xin (Eric) Wang *, Vihan Jain*, Engene Ie, William Yang Wang, Zornitsa Kozareva, Sujith Ravi Research Conference 2018 Natural Language Grounded Navigation Command


  1. Environment-agnostic Multitask Learning for Natural Language Grounded Navigation Xin (Eric) Wang *, Vihan Jain*, Engene Ie, William Yang Wang, Zornitsa Kozareva, Sujith Ravi Research Conference 2018

  2. Natural Language Grounded Navigation Command embodied agents to navigate in the 3D world with natural language, such as coarse-/fine-grained instructions, questions, dialog Person: Can you grab the plant for me? Sure. Where is it? Get out of the room and go towards the kitchen. The plant is on the window near the kitchen. Gotcha. 2

  3. Vision-and-Language Navigation (VLN) ● Given fine-grained instruction and a starting location ● Agent must reach the target location by following the natural language instruction ● Room-to-Room (R2R) Dataset Anderson et al., CVPR 2018 3

  4. Cooperative Vision-and-Dialog Navigation (CVDN) ● Both Navigator and Oracle are given a hint (e.g., the goal room contains a mat) ● Navigator : go towards the goal room and can stop anytime to ask a question ● Oracle : foresee the next best steps and answer the questions Thomason et al., CoRL 2019 4

  5. Sub-task: Navigation from Dialog History (NDH) Given the dialogue history , predict ● the navigation actions that bring the agent closer to the goal room Thomason et al., CoRL 2019 5

  6. Challenge 6

  7. Poor Generalization Issue ● Navigation models tend to overfit seen environments and perform poorly on unseen environments Training Evaluation Seen Unseen = ! = 7

  8. Data Scarcity is A Big Problem ● Real-world experiments are NOT scalable ● Data collection is prohibitively expensive and time-consuming ● Models break under distribution shift 8

  9. Environment-agnostic Multitask Navigation 9

  10. Towards Generalizable Navigation ● Multitask learning : transfer knowledge across tasks ��� ��� ���������������������������������������� � ���������������������������� ������������������������������������������ �������������������������������������������� ������������������������������������������������ ������������������������������������ ����������� � ������������ ��� ● Environment-agnostic learning : invariant representations that can be better generalized on unseen environments 10

  11. A Strong Baseline for VLN: RCM Leave the living room. Go through the hallway with paintings on the wall and head to the kitchen. Paired Demo Path Stop next to the wooden dining table. VLN Instruction Word Embedding Panoramic Features Language Encoder CM-ATT Trajectory Encoder Wang et al., CVPR 2019 Action Predictor 11

  12. Multitask RCM VLN Instruction Interleaved or Paired Demo Path Multitask NDH Dialog Data Sampling Joint Word Embedding Panoramic Features Language Encoder CM-ATT Trajectory Encoder Action Predictor 12

  13. Multitask Reinforcement Learning ● Navigation Loss: Reinforcement Learning + Supervised Learning ● Reward shaping: ○ VLN: Distance to Goal ○ NDH: Distance to Room 13

  14. Effect of Multitask RL ● NDH benefits from VLN ● VLN benefits from NDH with more fine-grained information about paths Extending visual paths alone is NOT ○ helpful ● Multitask RL improves generalization Seen-unseen gap is narrowed ○ 14

  15. Effect of Multitask RL Multitask learning benefits from • More appearances of unrepresented ��� ��������� ��� ��������� ����������������� ���������������� words • Shared semantic encoding of the whole sentences ���������� ����� �������� �������� ������� ������ � � � � � � ������� ����� ��������� ������� �������� ����� ����� ������� �������� ������� � � � � � � � � � ������� ������ ���� ������ ������ ������ ����� �������� ������� ������ ������ ������� ���� ���� ������� ���� � � � � � � � � � � � � � � � ������ �������� �������� ���� ������� ���� ���������� ������� ����� ������ ����� ������ ������ ��� � � � � � � � � � � � � � � ���� ���� ������� ����� � � � �� � � � � � � � � � ���� ����� � � � � � � � � � � � � � � � � � � � � � � � � � ���� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 15

  16. Environment-agnostic Representation Learning NDH Dialog or VLN Instruction Word Embedding Language Encoder CM-ATT Trajectory Encoder Action Predictor Gradient Reversal Layer Environment ● A classifier to predict the environment identity Classifier House label y 16

  17. Environment-Aware versus Environment-Agnostic NDH (Progress) VLN (Success Rate) Seen Unseen Seen Unseen 57.59 10 60 8.38 52.79 55 52.39 8 6.49 6.07 50 6 44.4 42.93 45 4 3.15 38.83 2.64 40 1.81 2 35 0 30 RCM EnvAware EnvAgnostic RCM EnvAware EnvAgnostic ● Env-aware learning tends to overfit seen environments ● Env-agnostic learning generalizes better on unseen environments ● (Potential) Meta-learning with env-aware & env-agnostic may benefit from both worlds 17

  18. Environment-Aware versus Environment-Agnostic Seen Unseen ��� ��� ��� ��� ���� ���� ���� ���� EnvAware EnvAgnostic EnvAware EnvAgnostic 18

  19. Environment-agnostic Multitask Learning Framework ���������� ���������� ���������������� �� ������������� ���������������� �������������� ���������������� ������ ������������������ ����������������� ���������������� ���������������������� 19

  20. Effect of Environment-agnostic Multitask Learning 20

  21. Ranking 1st on CVDN Leaderboard https://evalai.cloudcv.org/web/challenges/challenge-page/463/leaderboard/1292 21

  22. Future Work 22

  23. Generalized Navigation on Street View StreetLearn ( Mirowski, et al. 2018) TouchDown (Chen, et al. 2019) TalkTheWalk (de Vries, et al. 2018) 23

  24. Thanks! Paper: https://arxiv.org/abs/2003.00443 Code: https://github.com/google-research/valan 24

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend