ENHANCED FLIGHT VISION SYSTEMS: PRESENCE OF RUNWAY MARKINGS AND - - PowerPoint PPT Presentation

ENHANCED FLIGHT VISION SYSTEMS:

PRESENCE OF RUNWAY MARKINGS AND VISIBILITY EFFECTS ON PILOT PERFORMANCE

AN D R E W GR E E N H I L L S E P T E MB E R 1 5 T H, 2 0 1 7

AGENDA

2

• What is an Enhanced Flight Vision System (EFVS)
• Motivation
• Background
• Millimeter wave radar (MMW)
• Forward looking infrared (FLIR)
• Light detection and ranging (LiDAR)
• Situational awareness
• Visual cues
• Objective
• Pilot performance
• Research Plan
• Anticipated Contributions
• Proposed Timeline

WHAT IS EFVS

3

Enhanced Flight Vision Systems is defined as: “an installed aircraft system which uses an electronic means to provide a display of the forward external scene topography through the use of imaging sensors” (FAA AC 90-106A) Therefore, an EFVS must:

• Use real time imaging sensor, like a FLIR
• Use heads-up display
• Be an installed system on the aircraft

EFVS is different from both synthetic vision and enhanced vision systems

Images from EFVS presentation by Terry King

MOTIVATION

4

• New sensors have enabled EFVS to evolve
• Millimeter Wave Radar
• Light Detection and Ranging
• Multispectral IR
• EFVS has the ability to extend the capability of aircraft operations
• Landing below weather minimums
• Reducing delays while waiting for weather to clear
• With advancements in EFVS, the FAA is trying to get a head start on operational

approvalof these systems

• Not just setting sensor requirements
• How they affect pilot performance

MILLIMETER WAVE RADAR

5

Capabilities

• Penetration of various atmospheric

conditions due to frequencies in which it operates

• Some weather phenomenon, such

as fog, light clouds, and some precipitation have shown to improve contrast

• MMW Radar returns different

textures for different objects and terrain types Constraints

• Low resolution and noisy signals

generated; clustering and filtering techniques could be required

• Cannot detect and display runway

markings

• Major trade-offs between sensitivity

and range of the sensors

Image from Yang, 1994

FORWARD LOOKING INFRARED (FLIR)

6

Capabilities

• Thermal imaging sensor allows for

night vision and low-light scenarios

• Specifically help with runway incursions
• Provides distinction between

concrete and grass areas at most times

• Can be beneficial in haze or smoke

Constraints

• Dissipation rates are different for

materials, which causes thermal reversals and ghosting

• Weather greatly affects the thermal

imaging capabilities

• Solar Load
• Precipitation
• Wind speed
• Some types of FLIR can create

contrast issues by using the iHot spot technique

Image from Doehler and Korn, 2006

LIGHT DETECTION AND RANGING (LIDAR)

7

Although many sensors can use a combination, LiDAR typically is utilized with an INS and GPS system Capabilities

• Able to detect pavement markings
• n roads and runways
• LiDAR is typically more accurate

compared to MMW and FLIR sensors, could be due to INS/GPS Constraints

• Laser scanning techniques pose a

safety vs. effectiveness tradeoff

• Limitations with respect to

aerosol/cloud particles as well as weather

• Databases for checking GPS/INS are

not fully implemented and could lead to extreme errors

SITUATIONAL AWARENESS

8

Situational Awareness is defined as a state of knowledge with three levels of perception, interpretation and prediction (Endsley) Level 1 - Perception

• Flaps configuration
• Gear configuration
• Airspeed for flap and gear deployment
• Power setting
• Localizer deviation
• Glideslope deviation
• Visual of the runway centerline
• Visual of the runway sides
• Visual of top and bottom of the runway
• Visual of the touchdown markers
• Visual of the horizon

Level 2 - Interpretation

• Correct configuration of gears and flaps
• Appropriate speed and power settings
• On the desired glide path, typically about 3°
• On the extended centerline of the runway
• Whether the landing will be made in the first

3rd of the runway

• When to flare during landing

Level 3 - Prediction

• Travel path of traffic and conflicts
• Travel path of the aircraft and if it conflicts

with any obstacles or terrain

• Location of touchdown spot on the runway
• What the next objective of the mission is and

how to get there

VISUAL CUES

9

Visual cues for approach and landing mainly include:

• Terrain
• Airport Environment
• Factors of a stabilized approach

The main visual cues of a stabilized approach are part of Level 1 situational awareness

Level 1 - Perception

• Flaps configuration
• Gear configuration
• Airspeed for flap and gear deployment
• Power setting
• Localizer deviation
• Glideslope deviation
• Visual of the runway centerline
• Visual of the runway sides
• Visual of top and bottom of the runway
• Visual of the touchdown markers
• Visual of the horizon

Image from Marks, 2017

SITUATIONAL AWARENESS AND VISUAL CUES

10

In the figure, the Level 2 situational awareness tasks on the left To the right of each Level 2 task, are the Level 1 tasks that directly affect it Certain forms of EFVS have:

• Range issues - Blue
• No way of displaying runway

markings - Red

• Glideslope deviation
• Visual of top and bottom of

runway

• Visual of the horizon

On the desired glide path, typically about 3°

• Localizer deviation
• Visual of the runway centerline
• Visual of runway sides
• Visual of the touchdown

markers

On the extended centerline of the runway

• Visual of the touchdown

markers

• Visual of top and bottom of

runway

Whether the landing will be made in the first 3rd of the runway

• Visual of runway sides
• Visual of top and bottom of

runway

• Visual of the horizon

When to flare during landing

OBJECTIVE

11

The objective is twofold:

• Find the effect of how the visibility range of an EFVS affects the pilot performance on

approach and landing

• Find the effect that the absence of runway markings in EFVS has on the pilot

performance during approach and landing Situational Awareness Display Characteristics Visual Cues Sensor Properties

FITTING IN PILOT PERFORMANCE

12

Pilot performance for approach and landing can be measured primarily by a stabilized approach and some other factors A stabilized approach is when the following criteria are met:

• Correct configuration of gears and flaps
• Appropriate speed and power settings
• On the desired glide path, typically

about 3°

• On the extended centerline of the

runway

• Whether the landing will be made in the

first 3rd of the runway

The stabilized approach criteria align with level 2 of situational awareness Sensor properties Visual cues (Level 1 situational awareness) Level 2 situational awareness (Stabilized approach criteria) Pilot performance

EXPERIMENTAL SETUP

13

Simulator Hardware

• 4 separate monitors
• Three make up the
• utside view of the

environment

• The fourth screen is

the instrument panel.

• A yoke
• Rudder pedals
• Throttle control

system

• Eye tracker

INDEPENDENT AND DEPENDENT VARIABLES

14

Independent Variables

• Presence of runway markings in

EFVS

• Binary true or false
• Visibility range of EFVS
• Fluctuates between 1, 3 or 12 statue miles

Dependent Variables

• Stabilized approach criteria
• Localizer/glideslope error
• Altitude
• Vertical speed
• Airspeed
• Heading
• Latitude and longitude

EXPERIMENT DESIGN

15

Subjects

• 21 pilots would be tested
• Must be instrument rated and have

a minimum of 150 hours Base Conditions

• Start at a 5 nautical mile approach
• Be placed on the glideslope in a

cruise condition

• No flaps added yet
• The nav instruments would be

tuned into the localizer/glideslope Scenarios Each pilot would fly seven different approaches, listed below, these would be fully balanced using a Latin square

• Basic approach, no EFVS
• Approach with EFVS, runway markings and 12

sm visibility

• Approach with EFVS, runway markings and 3

sm visibility

• Approach with EFVS, runway markings and 1

sm visibility

• Approach with EFVS, no runway markings and

12 sm visibility

• Approach with EFVS, no runway markings and 3

sm visibility

• Approach with EFVS, no runway markings and 1

sm visibility

ANTICIPATED CONTRIBUTIONS

16

• Identify possible drawbacks or specific benefits of using EFVS
• Specifically with respect to pilot performance during approach and landing
• Results will be utilized in:
• Issuing operational approvals for EFVS
• Providing limitations for EFVS
• The focus on specific visual cues, will hopefully allow the results to be more
• perationally applied than sensor specific requirements
• Planned papers
• Previous literature review concerning the topic of EFVS
• Paper summarizing the objective and results of the thesis

SUMMARY

17

Motivation: With EFVS able to extend the capability of aircraft operations, the implications these technologies have on pilot performance need to be investigated Background:

• Each of these sensors have different capabilities and constraints
• The situational awareness is in three levels: perception, interpretation and prediction
• The visual cues and situational awareness of the pilot are intertwined since the

perception feeds into the interpretation of the stabilized approach Objective:

• Find the effect of how the visibility range of an EFVS affects the pilot performance on

approach and landing

• Find the effect that the absence of runway markings in EFVS has on the pilot

performance during approach and landing Research Plan:

• Running a set of simulator trials with varying EFVS conditions

NEXT STEPS

18

• 1. IRB Paperwork and approval – In process
• Finalize consent form, invitations and debrief questionnaire
• Hopefully submitted by next week
• 2. Pilot testing – start after IRB submitted
• Verify the experiment design and the data collection/analysis process
• Notice any variables that should be added/removed
• 3. Invite subjects to participate
• Use YJFC and previous CEC pilot email lists to gather subjects
• 4. Start subject testing and data analysis
• 5. During the main portion of these experiments, writing the thesis will be done

PROPOSED TIMELINE

19

Week of 7 / 3 1 / 1 7 8 / 7 / 1 7 8 / 1 4 / 1 7 8 / 2 1 / 1 7 8 / 2 8 / 1 7 9 / 4 / 1 7 9 / 1 1 / 1 7 9 / 1 8 / 1 7 9 / 2 5 / 1 7 1 / 2 / 1 7 1 / 9 / 1 7 1 / 1 6 / 1 7 1 / 2 3 / 1 7 1 / 3 / 1 7 1 1 / 6 / 1 7 1 1 / 1 3 / 1 7 1 1 / 2 / 1 7 Written Proposal Proposal IRB Approval Invitations Subject Testing Data Analysis Conclusions Outline of Thesis Written Thesis Defense Week of 1 1 / 2 7 / 1 7 1 2 / 4 / 1 7 1 2 / 1 1 / 1 7 1 2 / 1 8 / 1 7 1 2 / 2 5 / 1 7 1 / 1 / 1 8 1 / 8 / 1 8 1 / 1 5 / 1 8 1 / 2 2 / 1 8 1 / 2 9 / 1 8 2 / 5 / 1 8 2 / 1 2 / 1 8 2 / 1 9 / 1 8 2 / 2 6 / 1 8 3 / 5 / 1 8 3 / 1 2 / 1 8 3 / 1 9 / 1 8 Written Proposal Proposal IRB Approval Invitations Subject Testing Data Analysis Conclusions Outline of Thesis Written Thesis Defense

QUESTIONS

20

Thank you all for your time

Questions