Thinking Like Your Audience: How to Report Your Analytics Results so - - PowerPoint PPT Presentation

Thinking Like Your Audience:

How to Report Your Analytics Results so They Actually Get Read

IIBA Houston Chapter | MindEcology, LLC | Jed C. Jones

Presenter

▪ Jed C. Jones, MBA, PhD ▪ Based in Austin, Texas for 12 years. Born and raise in California.

Lived and worked for 5 years in Japan. Have done extensive business travel in Latin America.

▪ Co-founder & Chief Data Scientist of MindEcology, a data-

driven advertising agency

▪ Long-time data scientist, digital specialist, and entrepreneur ▪ Worked for 20+ years at the intersection of data and marketing ▪ Designed and produced hundreds of client-facing and internal

reports

Agenda

▪ Some Quotes on Data and Reports ▪ The 3 Main Types of Information in Analytical Reports ▪ 3 Myths about Analytics Reporting ▪ The Data Fluency Framework ▪ 4 Key Dimensions Along Which Reports Vary ▪ The 5 Hats You’ll Need to Wear ▪ The Mindset: 7 Qualities of an Effective Report Producer ▪ Examples of Poor Design and Good Design

Have Any of These Things Ever Happened to You?

1.

You sent a report via email to several people and nobody has asked you any questions or sent any feedback? Just dead silence.

2.

You presented a report in person and people were checking their smartphones

• r staring with a dull, glazed-over look at your slides?

3.

You presented a report in person, whereby one or more members of the audience constantly needled you on small details or found an error or two that eroded your credibility?

4.

Received complaints from your boss or colleagues that your reports are not clear, are dull, or off-point?

5.

Been asked, after sending or presenting a report, “So what”?

That’s Okay, You’re Not Alone!

If none of the aforementioned things have EVER happened to you, you probably have not created or shared many reports with others. The good news is, by understanding:

a.

What analytics reports are designed to accomplish

b.

What goes into creating an effective report

c.

The purpose of your report and who will be consuming it

d.

What the ideal report creation mindset is you will be well-positioned to get much better responses in the future.

Reports Communicate Timeless Human Concepts

“Categories such as time, space, cause, and number represent the most general relations which exist between things; surpassing all

• ur other ideas in extension, they dominate all the details of our

intellectual life. If humankind did not agree upon these essential ideas at every moment, if they did not have the same conception

• f time, space, cause, and number, all contact between their

minds would be impossible....” Emile Durkheim, 1912

Data Alone is Not Enough

“Data is a cold, lonely medium on its own. Data needs to be humanized and human-sized. It needs to be made relevant to the audience by being clearly linked to relatable problems.” “Data Fluency: Empowering Your Organization with Effective Data Communication,” Gemignani et al, 2014

The Great Divide in Every Organization

“Much of the conversation on data occurs across the great divide between those who have a cultivated knowledge of data and those who have responsibilities that seldom involve digging into data.” “Data Fluency: Empowering Your Organization with Effective Data Communication,” Gemignani et al, 2014

Too Much Data Becomes Noise

“Everyone spoke of an information overload, but what there was in fact was a non-information overload.” Richard Saul Wurman

What an Effective Analytics Report Does

“Effective analytics reports weave number-oriented and/or qualitative observations about entities or phenomena in space and time into a story that motivates decision-makers to take meaningful action.” Jed Jones, 2019

The 3 Main Types of Information in Analytical Reports

When you share a report with others, you will be typically showing

• ne or more of the following:

1.

The numerical relationship between two (or more) entities or phenomena

2.

The change in value of a given entity or phenomenon over time

3.

Facts about a geographical (or spatial) location

The 3 Main Types of Information in Analytical Reports

1.

The relationship between two (or more) variables, such as “revenue vs. cost” or “target sales vs. actual sales.”

The 3 Main Types of Information in Analytical Reports

2.

The change in value of a given thing over time (a time series) such as the 36-month sales history of widgets.

The 3 Main Types of Information in Analytical Reports

3.

Facts about a geographical or spatial location.

3 Myths about Analytics Reporting

▪ Myth #1: The meaning of data is self-evident to your

audience (the report consumer)

–It isn’t. Your job, as an analyst, is to put it into meaningful context.

3 Myths about Analytics Reporting

▪ Myth #1, example from my life:

When I first got a reporting job at Dell in Japan, my job was to pull data from D3 (Dell Data Direct) and send the results to whoever requested it. For the first 1-2 weeks of my employ, I would just send the data that was requested of me. (This is what you asked for, this is what you got). But I soon received a complaint from the department General Manager: “Jed, don’t just send this data along to me. Please add your commentary to this data. I’m busy and I need your input. What’s this data telling us?” It was at that moment that I realized that I wasn’t just a pass-through person. I needed to explain the data I was pulling. Every time I pulled a report moving forward, I thought of the person who was going to read it.

3 Myths about Analytics Reporting

▪ Myth #2: The more data you include, the better

–It isn’t. Less is more. It is as important to know what to leave out as what to include.

3 Myths about Analytics Reporting

▪ Myth #2, example from my life:

When I started my first digital services company in 2007, I would send massive, multi-page reports to my customers to show them the performance of their campaign. But when I checked in with customers, I found out most were not reading the reports. They didn’t really understand them. There was too much in there. So, when I started my next company, MindEcology, in 2009, I designed

• ur own reporting software. I decided to pick and choose only those

data points that told a story for my customers. Every data point had a

• purpose. My reports started getting read.

3 Myths about Analytics Reporting

▪ Myth #3: Reports are about numbers.

–They aren’t. Reports are stories about data that give insights that answer the “what should we do” question.

3 Myths about Analytics Reporting

▪ Myth #3, example from my life:

“Conversions are up.” “Clicks are down.” In the early days of running my own business, I would tell my customers the above kinds of facts each month. But my reporting was machine-like. Automatic. Boring. Lacking in

• insight. And contract renewal rates were not strong.

After that, I made it a point to always include what I call the “so what” factor in my reports. At least one nugget per report. My customers were happier and digital contract renewals went up.

3 Myths about Analytics Reporting

▪ Myth #3, more on the “so what” factor:

The “so what” factor is the insight that acts as a bridge between observation (data point) and action (business decision). Sometimes a valid decision is simply to stay the course. Or, to shift the budget to another campaign. Change your marketing tactics. Your report consumer will ALWAYS appreciate it when you add the “so what” factor.

Data Fluency Framework

From “Data Fluency: Empowering Your Organization with Effective Data Communication,” Gemignani et al, 2014

4 Key Dimensions Along Which Reports Vary

1.

Form and Format: Formal vs. Informal

2.

Optimization: Periodic vs. Ad Hoc

3.

Type of Content: Informational (Presenting What Is) vs. Analytical (Solving a Problem)

4.

Who's Reading: Vertical (For Upper Management) vs. Lateral Internal (For Colleagues) vs. Lateral External (Customers/Analysts/Investors)

The 5 Hats You’ll Need to Wear

• 1. Salesman: Know how to capture your audience's attention and how to

gain buy-in

• 2. Therapist: You need empathy to understand and share the feelings of
• thers
• 3. Connoisseur: Able to weed out merely "interesting" data from your

report and instead focus on that which is important and actionable

• 4. Data Analyst: Be comfortable with core statistical concepts and with

manipulating data

• 5. Ethnographer: How does your organization operate, day-to-day?

The Mindset: The 7 Qualities of an Effective Report Producer

1.

Be Purposeful

2.

Be Thoughtful

3.

Be Accurate

4.

Be Impartial

5.

Be Whole-Brained

6.

Be Clear

7.

Be Helpful

• 1. Be Purposeful

▪ Be clear on the purpose of the report ▪ Think about who will be consuming the report ▪ Decide where your report falls on each of the 4 dimensions (see above) ▪ Decide how it will be delivered: Email? Online dashboard? In person?

Paper?

▪ Confirm you can actually deliver on the required data. Check your access to

data sources before you commit to the report

▪ Decide whether you should present “just the facts” or whether your data

and related comments should be more exploratory in nature

• 2. Be Thoughtful

▪ Understand who is going to consume your report

– Do they trust you and/or the data sources? – How interested are they in the results? Are they reluctant and bored or focused and hungry for what you are offering? Rushed or patient? Tuned-in or tuned-out? – Are they a friendly audience? Collaborative? Contentious? – Are they headline types or body copy types? – How numbers-competent are they?

▪ Put the key findings at the beginning and move the supporting evidence to the

appendix

▪ Offer consumers the opportunity to review your report on the phone or in person

afterward

• 3. Be Accurate

▪ Verify the reliability of your data sources (and their sources) ▪ Run descriptive statistics before and after you transfer data from: raw source file →

PIVOTS/groupings → report body

▪ Sit with it for a day before delivering and scan for what seems out of place ▪ Operationalize and scale the report development process wherever and whenever

you can, for periodic reports

▪ Proofread your numbers and your words and hold yourself responsible for accuracy ▪ Make sure your analytical methodology enhance – and doesn’t hinder – your chances

for accurate results

• 4. Be Impartial

▪ Remain agnostic to the numbers and what they say ▪ Stand by what the data is saying about reality, without reservation ▪ As an analyst, you are helping the data tell its own story

• 5. Be Whole-Brained

▪ Make your first few pages visually-interesting ▪ Start with an outline, agenda or table of contents ▪ Balance data, images and words on the page ▪ Use graphics to tell a visual story, while including the hard numbers ▪ Retain symmetry and placement consistency from page to page ▪ Your report’s layout should feel intuitive: within a given page and from

• ne page to the next

▪ Think art and science, symmetry and facts

• 6. Be Clear

▪ Label every graph, table and chart ▪ Define your terms, where needed ▪ Mind the data-ink ratio (see Tufte’s work) ▪ Eliminate chart junk (see Tufte’s work) ▪ Chart, table and graph titles should include:

– time period(s), if applicable – units (i.e., people, dollars, Likert scale, etc.) – type of data or calculation (i.e., count, percentage, ratio, ranking, rating, etc.) – type of comparison (i.e., change over time, comparison between entities, spatial/ geographical layout, etc.)

• 7. Be Helpful

▪ Include an executive summary and key findings ▪ Include methodology (if applicable) ▪ Consider delivery method: maybe some content goes in body of

email, the rest goes into an attachment

▪ Provide insights and notes, the “so what” factor, next to each table,

chart and graph

On Design of Data Representations

Data representations should provide the user with “the greatest number of ideas, in the shortest time, using the least amount of ink, in the smallest space.” Edward Tufte

Poor Design Example

This bar graph fails to give us enough information to be useful and thus fails in delivering “visual integrity.”

Poor Design Example (2)

This image of business processes with an ERP environment is quite good at conveying which business functions are affected by the ERP processes but what purpose does the color scheme serve?

Good Design Example

The graphic above, relating to US employment statistics in March 2015,

• ffers many ideas in a very small space and is easy to digest. We’d suggest it

meets the criteria of “graphical excellence.”

Pop Quiz: How Can We Make This Graphic Better?

Pop Quiz: How Can We Make This Graphic Better?

 Update the title to indicate it’s a

gender comparison

 Fix typo in title to “Readers”  Add gridlines to more clearly

align data with category

 Add data labels to each page  Add “%” labels to numbers  Include a time period  Add an observation about

women

 Ditch the 3D graph

Questions? ▪ What questions may I answer for you?

Thank you!