Using Goals: How to Plan for and Prepare Reports Marie Garland, - - PowerPoint PPT Presentation

The Inclusive, Connective Corridor: Social Networks and the ADVANCEment of Female STEM Faculty

PI: Kal Alston, Special Advisor to the Provost; Professor, Cultural Foundations of Education Co-PI: Shobha Bhatia, Meredith Professor, Civil and Environmental Engineering Co-PI: Pamela Brandes, Associate Professor of Management Co-PI: Jeff Stanton, Interim Dean and Professor, School of Information Studies Co-PI: Karin Ruhlandt, Dean, College of Arts and Sciences; Distinguished Professor of Chemistry Scientific Advisor: Ines Mergel, Associate Professor, Public Administration/International Affairs

Using Goals: How to Plan for and Prepare Reports

Marie Garland, Ph.D. Executive Director, Syracuse University ADVANCE

What are your goals?

• Documented in your logic model

Because of X, Y should produce result(s) Z

• Described in the proposal

Y consists of Activities A, B, and C; done # times

• Developed in the evaluation plan

What is measured about A, B, and C each time? How? What change do you expect to see (Z) and how will you know it when you see it?

Gather your “stuff”

• Logic model
• Activities List
• Applications for support

(formal and informal)

• Attendance data (“touches”)
• Evaluation reports (internal

and external)

• Toolkit (or other program

required) data

• Other key metric data
• Your calendar
• The calendars of others on

the project (including their travel)

• Financial support of team
• Your financial records

(reminders only)

• Plan for the next period

Section B: Accomplishments,

• r, what happened?
• 1. Major Goals: Goals, (primary) objectives, activities
• 2. Achievement of Major Goals

a) Major Activities: What did you do? (A, B, C) b) Specific Objectives: How did it go? (results of measuring A, B, C; short term impact) c) Significant Results: How close are you getting to the change you expected to see for that goal? d) Key Outcomes and other Achievements: Is Y leading to Z? Did something else happen?

B1 B1 B2a B2b B2c Initi tiati tive: Goal Objecti tive Ac Activi vity Det etail Eva valua uati tion:

• n:

Output put Eva valua uati tion:

• n:

Outco come B2d Recruiting: Candidate pools for posted faculty

• penings

demonstrate faculty

• wnership for

early-stage and ongoing strategies to identify and recruit women The % of women in applicant pools increases Promising Researcher Grants [Depts] requested support # of visitors; their demos Feedback from department faculty (I) Recruiting Opportunity Grants [individuals] supported, their plans # of

• pportunities

Feedback from department faculty (I) The number

• f faculty

recommend- ing positions to women increases Search Committee Training [Depts/ Colleges] trained/not Number of attendees at training Feedback from search chair/faculty (with/out) (E) Search Committee Support [Specific searches] supported/ not # of women interviewed Increase in #

• f women

finalists over time (with/out)

Section B, con’t.

• 3. Opportunities for training and professional

development – professional development of project

staff/leadership (including participation in conferences and other training activities); mentoring activities if postdocs

• 4. Dissemination and Outreach – especially awareness

raising activities outside primary population (e.g. students, those on

• ther campuses, etc.)
• 5. Plans for the next reporting period – review (updated)

work plan

• 6. Attachments – invitations, handouts, newsletters,

presentations, brochures, etc.

Section C: Products, or How others learn from your work

1. Publications journals, books/non-periodical, conference papers

and presentations

2. Technologies or Techniques including query codes, etc. 3. Inventions, Patent Applications, and/or licenses 4. Websites list major changes or improvements 5. Other Products, i.e.:

1. Toolkits, guides, workshops “in a box” (not just ppt) 2. Audio/visual materials 3. Databases/data sets (including structure) 4. Survey and other data collection instruments

6. Description of Attachments for Section C

Section D: Participants, or, who did it?

1. Individuals working on the project 2. Organizations involved as partners 3. Other collaborators (internal, mostly; in attachment)

Section E: Impact or, why does it matter?

To: Meanin ning: g: Principal Disciplines Primary constituency/institution; sustainability Other Disciplines Other constituencies/institutions Human Resources impacts on the STEM workforce, access to and persistence in STEM careers Physical Res/Inf physical space of interest to others/their use? Institutional Res/Inf

• ffices, orgs (groups that can continue to

change the [realm] landscape?) Informational Res/Inf data/information standards or protocols Technology Transfer Public-private transfer; economic development Beyond Science and Technology Improving public knowledge, changing social policy or action,

Section F: Changes/Problems, or What is not the same?

• 1. Approach activities changed/modified based on evaluation?
• 2. Actual or Anticipated Problems and plans to address

them: compare your actual work with your anticipated work

• 3. Changes in costs positive and negative
• 4. Changes in human subjects IRB changes
• 5. Changes in use or care of vertebrate animals
• 6. Changes in use of biohazards

Section G: Special Requirements

 External Evaluation  Toolkit

Helpful Strategies for Successful NSF Reporting

ADVANCE/GSE Conference June 2015 Make the most of each section (use space wisely)! Rita Karl, Managing Director, STEM Media and Education Unit National Productions, T win Cities Public T elevision Co-PI, SciGirls CONNECT: A Scale Up Diffusion Project PBS SciGirls

Use space wisely!

 Use Word to create your report –

You can easily divide sections

• ut to different people; revise and reorganize easily before you

submit.

 Give sections to responsible people well ahead of time and then

have one person revise for one voice.

 Make sure you include the length of each section (if any

limitations), so that people know how much to write.

 Let your team know if they feel the need to write more, to do

so and note it – so that you have all the information and you can decide if it should go in a different section or be trimmed.

 Have one person responsible for submitting, and work with

them so if a section is too long, then you can decide what is the most important (to tell your story), or if you can use the section somewhere else.

Use space wisely!

 Take care to review your sections for repetitiveness. Key

Achievements could also be reflected in Activities and/or Impacts, for example. Dissemination can also be reflected in Publications and Products.

 Attachments are space and are reviewed by Program

Officers and can be a great place to put more detailed

• information. Recently we created a short program report

which used graphics to represent major program outcomes and it was very well received.

 Know your deadlines, and make sure your team does too.  Start early, so there is time to complete and revise.  Once online: Save. Often. (Note from my report submission

coordinator!)

There’s Data…and Then There’s Data: Telling Your Institution's Story

Sherry Yennello, ADVANCE PI Texas A&M University

This material is based upon work supported by the National Science Foundation under NSF Cooperative Agreement No. HRD-1008385.

16

Important questions

How do we increase the number of women? Are women recruited at the same rate as men?

Are women tenured at the same rate as men? Are women retained at the same rate as men?

17

2 Broad Requirements:

• 1. Annually report data related to progress

toward the goals of your program

• 2. Use Toolkit Guidelines
• a. NSF Indicators
• b. Salary

c. Space

• d. Start-up Packages

NSF Reporting Requirements

18

1. Standardized cross-sectional data for cross-institutional comparison 2. Relatively straightforward and quick (once you have the data) 3. Snap-shot, a single point in time 4. Best for descriptive studies 5. No causality or trend analysis

Toolkit includes guidelines for what NSF requires

19

NSF Indicators – Table 3

• No problem with promotion

20

NSF Indicators – Table 6

• No problem with attrition

21

What we knew…

• r thought we knew

What We Thought We Knew

Female faculty are not being denied tenure at a higher rate than male faculty. Retention is not a problem at TAMU. On average, 95% of tenured or tenure-track faculty are retained from one year to the next.

22

• Your university’s story is complex, unique, and needs to be fully

understood to bring about institutional transformation

• Survival Analysis
• Say what?
• Time to event (Public Health)
• Duration Analysis (Economics)
• Event History Analysis (Sociology)
• Two common statistical methods
• Kaplan-Meier analysis (non-parametric)
• Cox proportional hazards regression model (semi-parametric)
• Answers the question:
• What proportion of a population will “survive” to an event?

Going Beyond the Toolkit to Tell the Story

23

Baseline Survival Rates for Tenure- Track Faculty in Engineering

Probabilities of Promotion and Separation for STEM Associate Professors

24

.05 .1 .15 .2 2 4 6 8 10 Years as TAMU Associate Professor Female Probability of Promotion Male Probability of Promotion Female Probability of Separation Male Probability of Separation

All STEM

Probabilities of Promotion and Separation

25

What We Know Now

What We Thought We Knew What We Know Now

Female faculty are not being denied tenure at a higher rate than male faculty. As we explored the data on a deeper level, we realized that some faculty are counseled-out along the path to tenure. Retention is not a problem at TAMU. On average, 95% of tenured or tenure-track faculty are retained from one year to the next. Survivability has been significantly lower for female faculty than for male faculty in our College of Engineering. Female faculty are leaving right after tenure Female faculty do not leave after getting tenure disproportionate to their male counterparts

26

Take Home Message

• You must meet NSF reporting

requirements

• Higher level statistical analysis can help

you understand the data (and tell your institution’s story)

advance.tamu.edu

Sherry Yennello, PI yennello@comp.tamu.edu 979.845.1141 Lori Taylor, Evaluation Team Leader lltaylor@tamu.edu 979.458.3015