Cheshire Puss, she began, rather timidly, as she did not at all know - - PowerPoint PPT Presentation

“Cheshire Puss,” she began, rather timidly, as she did not at all know whether it would like the name: however, it only grinned a little wider. “Come, it's pleased so far,' thought Alice, and she went on. `Would you tell me, please, which way I ought to go from here? “That depends a good deal on where you want to get to,” said the Cat. “I don't much care where—” said Alice. “Then it doesn't matter which way you go,” said the Cat. “--so long as I get somewhere,” Alice added as an explanation. “Oh, you're sure to do that,' said the Cat, `if you only walk long enough”

E RC We binar

L

• g ic Mo de ls fo r

Stra te g ic Pla nning & E va lua tio n

De c e mb e r 18 &19, 2017

L e a rning Ob je c tive s

We b ina r a tte nde e s will b e a wa re o f the :

• E

va lua tive inq uiry c yc le

• Ro le o f mo de ls in stra te g ic pla nning
• T

e rms use d in c re a ting a the o ry o f c ha ng e a nd lo g ic mo de l

• Ste ps in c re a ting mo de ls
• Ste ps to a lig n mo de ls a nd stra te g ic pla n

3

E va lua tio n Ove rvie w

NSF Appro a c h, Princ iple s & E xpe c ta tio ns

4

E va lua tive I nq uiry Pro c e ss

c rite ria fo r wha t c o nstitute s e vide nc e pro c e sse s & me tho ds fo r

• b ta ining e vide nc e

ho w e vide nc e c a n b e a na lyze d, synthe size d & use d

2 1

de fine de sign me asure analyze institutionalize re port/ use

3

Stra teg ic Pla n E va lua tion Pla n Annua l Report

5

ho w e vide nc e ,

• nc e analyze d

and synthe size d, c an be use d

Stra te g ic Pla nning

Cur r ent State Compr ehensive Vision

7

Strate gic Plan

Ho w w ill w e do o ur w o rk? Ho w do w e mo nito r pro g re ss a nd de fine suc c e ss?

Cre a ting Fo c us

E valuation Plan

Wha t e vide nc e do w e ne e d to ma na g e a nd impro ve the pro je c t? Ho w w ill w e g a the r a nd use e vide nc e

Theory of Change and Logic Models help identify key program components

Ma pping & Asse ssing Pro g re ss

Cur r ent State Compr ehensive Vision

Theory of Change

Do Ge Get

9

Do Ge Get

SUPPORT

De ve lo ping Mo de ls

Stra te g ic Pla ns are Blue prints fo r Suc c e ss

10

11

Do Ge Get

E ng ine e ring Wo rkfo rc e De ve lo pme nt Inno vatio n E c o syste m C ulture o f Inc lusio n Infrastruc ture & Manag e m e nt Re se arc h Inc re ase d re se arc h c apabilitie s and ne w kno wle dg e Dive rse , g lo bally c o m pe titive , te am -

• rie nte d wo rkfo rc e

Im pro ve d value c hain, te c hno lo g y transfe r, and e ntre pre ne urial c ulture Partic ipants fro m all bac kg ro unds partic ipate and suc c e e d Im pro ve d m anag e m e nt, infrastruc ture , and im ple me ntatio n

T he o ry o f Cha ng e E xa mple

Mo de ling a T he o ry o f Cha ng e fo r yo ur Stra te g ic Pla n

• 1. Use a she e t o f e a se l po st-it.
• 2. Ide ntify the fo unda tio na l c o mpo ne nts

a nd the ir ma jo r thrusts fo r yo ur pro je c t a nd write e a c h o n a 5x7 po st-it.

• 3. Ide ntify the o ve ra rc hing re sult fo r e a c h

c o mpo ne nt/ thrust a nd write e a c h o n a 5x7 po st-it.

12

T a ke Home E xerc ise 1

13

Do Ge Get

E ng ine e ring Wo rkfo rc e De ve lo pme nt Inno vatio n E c o syste m C ulture o f Inc lusio n Re se arc h

? ? ? ? ?

Infrastruc ture & Manag e m e nt

Simple L

• g ic Mo de l

Strategies Results DO GET Outputs Inputs Activities Impact Short-term Outcomes Long-term Outcomes Intermediate- term Outcomes

14

L

• g ic Mo de l Co mpo ne nt

De finitio ns

T erm Definition

I nputs I nc lude fi nanc i al, hum an, o rg ani zati

• nal, c o m m uni

ty o r syste m s re so urc e s e sse nti al to i m ple m e nt the pro je c t. Ac ti vi ti e s T he spe c i fi c ac ti

• ns that m ake up the pro je c t. T

he y c an i nc lude to o ls, pro c e sse s, pro duc ts, e ve nts, te c hno lo g y and

• the r aspe c ts o f the i

nte rve nti

• n de plo ye d to ac hi

e ve de si re d re sults. Outputs I nc lude de sc ri pti

• ns o f the type s, le ve ls and audi

e nc e o r targ e ts fo r the pro je c t. Co untable attri bute s o f the ac ti vi ti e s i f ac c o m pli she d. (F re que nc y, I nte nsi ty, T arg e ts) Outc o m e s T he c hang e s i n pro je c t parti c i pants o r o rg ani zati

• ns, as a

re sult o f the pro je c t. Can i nc lude c hang e s i n aware ne ss, kno wle dg e , ski ll, and be havi

• r. (Spe c i

fi c , Me asurable , Ac ti

• nable , Re ali

sti c , T i m e d) I m pac t T he ulti m ate c hang e i n an o rg ani zati

• n, c o m m uni

ty o r o the r syste m . Ofte n o c c urs afte r the g rant c yc le has e nde d.

15

Pro g ra m-le ve l L

• g ic Mo de l E

xa mple s

16

Insight Policy Research (2015) EFRI Outcome Monitoring System

Goals Outputs & Outcomes Metric Breakthrough Technologies New products #/5 years/#university partners New methods New processes Papers # of journal publications/5 years/# partners Stakeholder Satisfaction IAB member satisfaction % of membership renewals averaged over a 4-year period Leveraged funding $ other new sources : $ NSF/5 years Researcher satisfaction Likert scale satisfaction Student Outreach Graduate research grants # of grants for theses and dissertations Student participation # of student members/5 years Student Development Student projects # of student publications/presentations/5 years Mentorships Median ratio researcher : graduates/5 years Technology Commercialization # Degrees # (BS + MS + PhD)/5 years Licensing # of new licenses/5 years Students hired by IAB member % of participating graduates hired by IAB member firms averaged/5 years Consulting # of consulting contracts for researchers to IAB member companies/5 years Knowledge Transfer Website Quality of information dissemination on website Prof org memberships # of professional memberships held by IAB members/5 years Papers # of co-publications (researcher and industry member)/5 years Conference presentations # of conference presentations/5 years Workshops # of seminars and workshops held

Go a ls Cha ra c te rize d b y Me a sura b le Outputs & Outc o me s

Gibson, E. and Daim, T. (2016). A measurement system for science and engineering research center performance evaluation. Engineering and Technology Management. 2016 Proceedings of PICMET ‘16: Technology Management for Social Innovation.

Re a lity Re pre se nta tio n Impre ssio n & Ab stra c tio n

22

“Vision” Stra teg ic Pla n L

• g ic Models &

T heories of Cha ng e

Pro po sa l L

• g ic Mo de l Stra te g ic

Pla n

1. F

• r e ac h fo und atio nal c o mpo ne nt and strate g y,

c o mple te the ac tivitie s c o lumn. 2. L ist the e xpe c te d re sults (g o als) and sho w inte rre latio nships if ne e d e d . 3. F ill in the g aps (inputs, o utputs, sho rt- and inte rme diate te rm o utc o me s) to sho w the links be tw e e n yo ur “ d o ” and “ g e t.” 4. Che c k to assure the links fro m le ft to rig ht are in a lo g ic al, fe asible , se que nc e . 5. E nsure that the mo d e l re pre se nts the pro je c t (w / o unne c e ssary d e tail. 6. Re vise d and upd ate the mo d e l pe rio d ic ally to re fle c t c hang e s in the pro je c t.

23

L

• g ic Mo de l & Stra te g ic Pla n

24

Inputs Activities Outputs Long-term Outcomes & Impact Short- & Intermediate Outcomes

Que stio ns to Guide Re vie w o f the L

• g ic Mo de l & Stra te g ic Pla n
• 1. Are the ma jo r inputs, a c tivitie s, a nd
• utputs c o nsiste nt a nd suffic ie nt to

a c hie ve de sire d o utc o me s?

• 2. Are the stra te g ic g o a ls o utc o me
• rie nte d?
• 3. Are the re missing stra te g ic g o a ls?
• 4. Ho w do c o lle a g ue s no t fa milia r with yo ur

pro je c t, inte rpre t yo ur mo de l?

25

Mo de ls fo r E va lua tio n Pla nning

INFLUENCES ACTIVITIES OUTPUTS IMPACT SHORTER- TERM OUTCOMES LONGER -TERM OUTCOMES

Relationships & Capacity Effectiveness

Form at iv t ive Evalu luat i t ion Sum mativ ive Evalu luat i t ion

Context Process Outcomes

INPUTS

Are we doing the “right” work? Are we doing the work “right”?

Quality, Use & Satisfaction

Are products/services accessed and used as intended? What difference have we made? What have we learned about what it takes to do & sustain this work?

1 2 3 4 5

26

Re so urc e s

• https://www.wkkf.org/resource-

directory/resource/2006/02/wk-kellogg-foundation- logic-model-development-guide

• https://fyi.uwex.edu/programdevelopment/logic-

models/

• http://ctb.ku.edu/en/table-of-

contents/overview/models-for-community-health-and- development/logic-model-development/main

• http://www.pointk.org/client_docs/File/logic_model_w
• rkbook.pdf
• http://www.sagepub.com/sites/default/files/upm-

binaries/23938_Chapter_3___Creating_Program_Logic _Models.pdf

27

Ge ne ric Stra te g ic Pla n T e mpla te

28

Project Thrust 1: Description…

Do Do Ge Get

Ano the r Vie w

29

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 1.1.1

Ac tivity 1.1.1

GOAL 1

Ye a r 1 Ye a r 5

OBJECT I VE 1.1

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 1.1.2

Ac tivity 1.1.2

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 1.2.1

Ac tivity 1.2.1

OBJECT I VE 1.2

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 1.2.2

Ac tivity 1.2.2

T HRUST 1

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 2.1.1

Ac tivity 2.1.1

GOAL 2

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 2.1.2

Ac tivity 2.1.2

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 2.2.1

Ac tivity 2.2.1

Outputs, Mile sto ne s, Ac c o mplishme nts fo r Ac tivity 2.2.2

Ac tivity 2.2.2

OBJECT I VE 2.1 OBJECT I VE 2.2

Do Do Ge Get