Natural Resources Analysis and Decision Making Byron K. Williams - - PowerPoint PPT Presentation

Natural Resources Analysis and Decision Making

Byron K. Williams

Cornell University

June 8, 2009

• Management actions are taken through time
• Resource behavior is influenced by management

actions

• Resource behavior is influenced by changing

environmental conditions

• There is uncertainty (or disagreement) about the

expected impacts of management

Natural Resource Situation

Examples

• Hydrologic systems
• Agricultural/grazing lands
• Wildlife or fish populations
• Habitats of species of interest
• Biological communities
• Managed wetlands
• Fire management
• etc

time resource system resource system t t+1 t-1 resource system management action environmental conditions

… …

management action management action environmental conditions environmental conditions

Management of Dynamic Resources

partial control structural uncertainty environmental variation partial

• bservability

time resource system resource system t t+1 t-1 resource system management action

… …

management action management action environmental conditions environmental conditions environmental conditions

Environment

Environmental Conditions

• Examples include seasonal temperatures, precipitation, etc
• Environmental conditions change through time

– Temporal variation may or may not be directional – Long-term directional change may be indicative of climate change

• Environmental factors may or may not be observed
• Environment directly influences resource state and/or the processes

that drive resource dynamics

• Environmental variation induces stochastic system behaviors
• Denoted here by zt , with trajectory {z0 ,…, zt ,…, zT} over [0, T]

time resource system resource system t t+1 t-1 resource system environmental conditions

… …

environmental conditions environmental conditions management action management action management action

Management

Management Actions

• Taken sequentially over some timeframe [0, T]

– typically at regular intervals

• May focus on resource inputs (fish stocking), outputs (water release), or processes

(habitat alterations that affect reproductive success)

• Typically vary with time
• Guided at each point by

– management objectives – current resource status – current resource understanding

• Denoted by at , with trajectory {a0 ,…, at ,…, aT}
• Strategy A0 identifies a particular action for every state xt at every time in the

timeframe [0, T]

time t t+1 t-1 management action environmental conditions

… …

management action management action environmental conditions environmental conditions resource system resource system resource system

Resources

Resource Dynamics

• Resource evolves over [0, T]
• Influenced by environmental conditions and management actions
• Resource systems are infinite-dimensional

– system models are always creative exercises in disregarding almost everything, while retaining only “essential” features

• Typically characterized by state variables and the processes linking state

variables, environmental conditions, and management actions

– State variables include interacting populations or species, population cohorts, habitat structures, social structures, etc – Processes include mortality, reproduction/recruitment, movement, etc

• System state denoted by xt , with trajectory {x0 ,…, xt ,…, xT}

System Transitions

• Generically, transitions can be characterized in terms of discrete or

continuous states, discrete or continuous time, system lags, irregular time steps that may or may not be tied to system state, etc

• Typically incorporates system state, environment, and management action

into Markovian transitions

• Often described as discrete-time systems, in keeping with periodicity of

seasonal events like migration, reproduction, etc

• Environmental variation and other stochastic elements define a discrete-

time Markov Decision Process (MDP) with transition probabilities

1=

( , , )

t t t t

x F x a z

+ 1

( | , )

t t t

P x x a

+

Two Big Issues

• Structural uncertainty – limited understanding of the structural features that control

resource dynamics

• Partial observability – inability to observe the state of the system through time

First Big Issue: Structural Uncertainty

• Uncertainty about the structural features of a resource system

– uncertainty about the form of one or more processes (e.g., mortality, recruitment, movement) that influence system dynamics – uncertainty about the role of process drivers (e.g., environmental conditions, population size or density, habitat structure) – uncertainty about process vital rates (e.g., reproduction or survival rate)

Uncertain Parameterization

• Transitions influenced by uncertain parameters :
• Parameter-specific transition probabilities
• Parameter state is a time-specific distribution of parameter

values

– represented by – updated through time based on observed process behaviors

( )

t

q β

1 t

q

1

( , , )

t t t t

x F x a z

β + = 1

( | , )

t t t

P x x a

β +

β

Uncertain Model Structure

• Transitions characterized by a model with uncertain structure
• Structural uncertainty can be characterized by multiple models:
• Model-specific transition probabilities
• Model state is a time-specific distribution of model indices

– represented by – updated through time based on observed process behaviors

( )

t

q i

2 t

q

1

( , , )

i t t t t

x F x a z

+ =

1

( | , )

i t t t

P x x a

+

Joint Uncertainty in Model Parameterization and Structure

• Index- and parameter-specific transition model
• Index- and parameter-specific transition probabilities
• Model-parameter state is a time-specific joint distribution of model indices

and parameter values

– represented by – updated through time based on observed process behaviors

, 1

( | , )

i t t t

P x x a

β + , 1

( , , )

i t t t t

x F x a z

β + =

( , ) ( ) ( | )

t t t

q i q i q i β β =

( )

1 2

,

t t t

q q q =

Second Big Issue: Partial Observability

• System is not completely observable
• Observations {y1,…, yt,…, yT} are tied to, but not the same as,

system state

• Two new requirements:

– Stochastic observation model

• Random component is a white noise process
• Induces a time-specific distribution

– Probability structure for system state

• Time-specific belief state bt
• Specifies a probability mass bt(xt) for each possible state xt at time t

1

( , , )

t t t t

y H x a ε

−

=

t

ε

1

( | , )

t t t

f y x a −

Belief State

• Evolves through time as observations accumulate,

according to Bayes’ Theorem: with

• Belief state transition probabilities can be expressed as

1 1

( , , )

t t t t

b G b a y

+ +

=

1 1 1 1 1 1

( | , ) ( | , ) ( ) ( ) ( | , )

t

t t t t t t t x t t t t t t

f y x a P x x a b x b x P y b a

+ + + + + +

=

∑

1

1 1 1 1

( | , ) ( | , ) ( | , ) ( )

t t

t t t t t t t t t x x t t

P y b a f y x a P x x a b x

+

+ + + +

= ∑

∑

1 1

1 { } 1 1

Pr( | , ) I ( | , ) ( | , )

t t

t t t b t t t t y t t

b b a y b a P y b a

+ +

+ + +

= ∑

Process Valuation

• Expressed as the aggregation of time-specific returns that

accrue to actions over [0, T]

– with the return at any time that is specific to the actual system state and action taken at that time – and the aggregation averaged over all possible state trajectories

( | )

t t

R a x

Returns with Completely Observable Systems

• Known Structure

– Immediate return for action at given resource state xt

• Structural Uncertainty

– Immediate return for parameterization , with average immediate return – Immediate return for model i, with average immediate return – Immediate return for parameterization and model i, with average immediate return

( | )

t t

R a x

β

β

( | )

t i t

R a x

, (

| )

t i t

R a x

β

1

( | , ) ( ) ( | )

t t t t t t

R a x q q R a x d

β β

β β = ∫

2

( | , ) ( ) ( | )

t t t i t t t i

R a x q q i R a x = ∑

,

( | , ) ( ) ( | ) ( | )

t t t t t i t i t

R a x q q i q i R a x d

β β

β β = ∑

∫

( | )

t t

R a x

β

Returns with Partially Observable Systems

• Known Structure

– Immediate return for action at given resource state xt, with an average across possible states of

• Structural Uncertainty (e.g., uncertain parameters)

– Immediate return for parameterization , with an average across possible states and parameter values of

( | )

t t

R a x ( | )

t t

R a x

β

β

( | ) ( ) ( | )

t

t t t t t t x

R a b b x R a x = ∑

1

( | , ) ( ) ( ) ( | )

t

t t t t t t t t x

R a b q b x q R a x d

β β

β β = ∑

∫

Process Value Function

• Averages the aggregation of returns over all possible state

trajectories

• Specific to a given strategy At
• Conditioned on an initial state xt
• Denoted by V(At|xt)

( | )

T t

R a x

τ τ τ =

∑

Observable system with parameter uncertainty

( )

1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

| , ( ) ( | ) ( ) ( | ) ( | , ) ( | ) = ( | , ) ( ) ( | , , ) ( | ) ( | , ) ( | , , ) | ,

t t

t t t t t t t t t t t t t t x t t t t t t t t t t x t t t t t t t t t t

V A x q q V A x d q R a x P x x a V A x d R a x q q P x x a q V A x d R a x q P x x a q V A x q

β β β β β β β β

β β β β β β

+ +

+ + + + + + + + + + +

= ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦ ⎡ ⎤ + ⎣ ⎦ = +

∫ ∑ ∫ ∑ ∫

( )

1

1

t

x +

∑

1 1

1 1 x 1 1 1 1 x

( | ) ( | ) ( | ) ( | , ) ( | ) ( | ) ( | , ) ( | )

t t

T t t t t T t t t t t t t t t t t t t t

V A x E R a x x R a x P x x a E R a x x R a x P x x a V A x

τ τ τ τ τ τ

+ +

= + + = + + + +

⎡ ⎤ = ⎢ ⎥ ⎣ ⎦ ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦ = +

∑ ∑ ∑ ∑

Observable system with known parameters

Partially observable system with known parameters

1 1

1 1 1 1 1 1

( | ) ( | ) ( | ) Pr( | , ) ( | ) ( | ) Pr( | , ) ( | )

t t

T t t t t T t t t t t t p t t t t t t t t p

V A b E R a b b R a b b b a E R a b b R a b b b a V A b

τ τ τ τ τ τ

+ +

= + + = + + + +

⎡ ⎤ = ⎢ ⎥ ⎣ ⎦ ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦ = +

∑ ∑ ∑ ∑

Partially observable system with parameter uncertainty ( ) ( )

1 1

1 1 1 1 1 1 1 1 1 1 1

| , ( ) ( | ) ( ) ( | ) Pr( | , , ) ( | ) = ( | , ) ( ) Pr( | , , ) ( | ) ( | , ) Pr( | , , ) | ,

t t

t t t t t t t t t t t t t t p t t t t t t t t t t p t t t t t t t t t t p

V A b q q V A b d q R a b b b a V A b d R a b q q b b a q V A b d R a b q b b a q V A b q

β β β β β β β

β β β β β β β

+ +

+ + + + + + + + + + +

= ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦ ⎡ ⎤ + ⎣ ⎦ = +

∫ ∑ ∫ ∑ ∫

1 t+

∑

Optimal Decision making

Two-part dynamic optimization

– Optimizing the future – Optimizing the present given the future

Observable system with known parameters

1 1 1 1 1

1 1 1 , x 1 1 1 x 1 1 1 x

( ) max ( | ) max ( | ) ( | , ) ( | ) max ( | ) ( | , )max ( | ) max ( | ) ( | , ) ( )

t t t t t t t t t

t t t t A t t t t t t t a A t t t t t t t a A t t t t t t t a

V x V A x R a x P x x a V A x R a x P x x a V A x R a x P x x a V x

+ + + + +

+ + + + + + + + +

= ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦ ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦ ⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦

∑ ∑ ∑

Observable system with parameter uncertainty Partially observable system with known parameters Partially observable system with parameter uncertainty

1

1 1 1

( ) max ( | ) Pr( | , ) ( )

t t

t t t t t t t t t a p

V b R a b b b a V b

+

+ + +

⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦

∑

( ) ( )

1

1 1 1 1 1 1 1 1

, max ( | , ) ( | , , ) ,

t t

t t t t t t t t t t t t t a x

V x q R a x q P x x a q V x q

+

+ + + +

⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦

∑

( ) ( )

1

1 1 1 1 1 1 1 1

, max ( | , ) Pr( | , , ) ,

t t

t t t t t t t t t t t t t a p

V b q R a b q b b a q V b q

+

+ + + +

⎡ ⎤ = + ⎢ ⎥ ⎣ ⎦

∑

Extensions

• Computing, computing …
• Allow for evolution of the system model
• Allow for directional environmental change
• Incorporate a salvage value into the objective function
• Incorporate discounting
• Allow for stochastic actions
• Combine partial observability and structural uncertainty
• investigate different kinds and shapes of the objective function
• Multi-objective assessment
• Sensitivity analyses ad infinitum
• Etc

Closing Comments

• Scope and complexity increase with the number of ecological components in

xt and the functional forms connecting components

• Scope and complexity can increase dramatically with uncertainties

– with each form of uncertainty adding to the dimensionality and computing burden

• Both partial observability and structural uncertainty transform a discrete state

problem into a continuous state problem

• Natural resource systems typically have a spatial component

– Populations are distributed over the landscape, and move over the landscape – Habitats exhibit spatial variation in structures as well as functions – Ecological units often are spatially correlated

• Every expression just shown for resource state, dynamics, and values can be

expressed in terms of spatial location, and the geographic linkage of units in space

– I.e., natural resource management is a spatio-temporal problem