SETIG Systems (Un)Conference Update AEA 2016 2015 Follow-up - - PDF document

SETIG Systems (Un)Conference Update AEA 2016

2015 Follow-up Survey

7% 23% 23% 48% Yes, frequently No time or opportunity, but would like to (or plan to) Not at all Yes, occasionally

Over half of 2015 participants are using new knowledge or skills learned from the (Un)Conference.

(n=31)

Connecting with like- minded colleagues who are interested in learning together and sharing resources about systems. Great conversation with other evaluators about systems evaluation.

The fostering of a community, which is definitely promoted by the kind of event you

• rganized, is very

important - keep on doing it!

Being new to systems evaluation the most useful things … were [the] basics... and being able to "listen in" on the conversations in various rooms by those with more experience and expertise.

29% 71% Yes No

Fewer participants connected with someone they met through the 2015 (Un)Conference.

(n=31) 11% 0% 11% 22% 56% 5 or more 4 people 3 people 2 people 1 person

For those that did connect, the majority connected with one person.

(n=9)

• connecting with like-minded others
• the informal unconference environment
• great resources shared
• a novel way to learn and connect

• length (three days too much)
• encourage more active participation and

conversation

• develop clearer focus

Western Hemisphere

69 attendees

Eastern Hemisphere

29 attendees

69 51 80 127 191 188 220 249 2016 Day 3 Day 2 Day 1 Attended

2016 Western Hemisphere attendance was lower than the previous year.

And registration continues to be double actual attendance.

2015 Registered

The Eastern Hemisphere session met more participants’ expectations.

76% 24% Americas

92% 8%

Western Hemisphere Eastern Hemisphere

n=46 n=24 n=24 n=46 4% 4% 8% 4% 15% 4% 4% 4% 4% 4% 6% 8% 20% 47% 25% 50% 50% 30% 50% 49% 58% 49% 63% 46% 42% 38% 33% 27% 38% Strongly disagree Disagree Neutral Agree Strongly Agree

Registration process straightforward Received information needed prior to (Un)Conference None or few problems with (Un)Conference software I increased my knowledge of the topic

People still experience problems with the technology.

4% 4% 11% 4% 11% 8% 27% 8% 16% 13% 42% 33% 47% 50% 40% 9% 16% 54% 27% 42% 45% 87% Strongly disagree Disagree Neutral Agree Strongly Agree

Feel more connected to colleagues Good value for time spent Would attend another (Un)Conference

n=24 n=46

The Eastern Hemisphere session had more positive

• utcomes.

2.0 3.3 3.3 4.2 2.0 3.7 1.7 2.4 3.3 3.8 4.5 4.3 4.4 4.5 4.3 3.2 3.0

Basics of Systems in Evaluation: Getting Oriented Bridging the Know-Do Gap: Praxis for Systems-oriented Evaluation Designing a Systems-based Evaluation: Challenges and Strategies Using a Complex Adaptive Systems Approach: Real-life Examples Distinguishing Systems Evaluation: Where are the Boundaries? Designing Useful Systems Visualizations: Tools & Strategies Assisting the Design of Complex Programs: Real-life Examples Identifying System Leverage Points: Helpful Approaches Similarities & Differences between Design vs. Systems Thinking: Applied to Evaluation Importing Tools from the Design World into Systems Evaluation: Think Big! Support for a Systems Approach: Strategies for Promoting Buy-in

Some Western hemisphere sessions were rated higher than others. The first session was usually rated higher than the second.

4.2 5.0 5.0 3.8 4.8 4.0 4.7 4.0 5.0 4.0 4.0 5.0 4.0 Basics of Systems in Evaluation: Getting Oriented Systems thinking and complexity methods & tools we might use in evaluation Thinking systemically when tackling a value for money in evaluation A systemic approach to evaluating Whānau Ora in a Pasifika context

Using a Developmental Evaluation Approach in Systems Evaluation

The cultural fit of systems thinking in indigenous evaluation contexts

Eastern hemisphere sessions, both first and second, were rated more highly.

Suggestions for improvement

Have fewer breakout rooms to ensure rooms are full. Add brief position statements to each room to help set the tone for discussion. Ensure each room has a facilitator to take the lead with topic suggestions and shape discussions, especially in sessions with a lot of beginners. Use a technology which is less likely to be blocked by some employer firewalls, i.e. government. Encourage and support more people to attend and fully participate in the (Un)Conference.

Participants want to learn more about…

Actual techniques and methods for systems evaluations. How design, evaluation, and systems thinking overlap. What defines systems thinking from regular evaluation. How to balance different values that stakeholders bring when tough decisions need to be made about resource allocations. Determining the boundaries of a system and how to facilitate that conversation with clients.

…Process lots of new insights I can add into my work - far more than one.

…Connect more regularly to the Systems group at AEA in some way. Brief occasional meetings aren't a substitute.

Going forward participants will….

2016 Committee Members

Bethany Laursen
 Judy Oakden
 Jan Noga
 Kylie Hutchinson
 Ginger Fitzhugh Jenny Lawlor
 Jen Nunez
 Alana Robilliard

Special thanks to

Heather Britt Brandon Coffee-Borden Bob Williams