Wrap up & Experimentation CS147L Lecture 8 Mike Krieger - PowerPoint PPT Presentation

Jan 02, 2024 •181 likes •1.38k views

Wrap up & Experimentation CS147L Lecture 8 Mike Krieger Wednesday, November 25, 2009 Intro Wednesday, November 25, 2009 Welcome back! Wednesday, November 25, 2009 By the end of today... - Questions from implementations - A few

Even better: Protovis - Stanford Graphics lab project - http://vis.stanford.edu/protovis/ Wednesday, November 25, 2009
Significant change? - Chi-Squared test Wednesday, November 25, 2009
Chi-Squared - Idea: measure whether a particular distribution of measures deviates significantly from expected Wednesday, November 25, 2009
Null hypothesis - Button color has no impact on click- through rate Wednesday, November 25, 2009
Testing Where: O(i) is the observed frequency, E(i) is the expected frequency degrees of freedom = (number of categories) - 1 Wednesday, November 25, 2009
Sample data Blue Red Green Total 267 267 266 800 Significant? Wednesday, November 25, 2009
Sample data Blue Red Green Total 270 250 260 800 Significant? Wednesday, November 25, 2009
Sum the differences Blue: (270-267)^2 / 267 = 0.03 Red: (250-267)^2 / 267 = 1.08 Green: (280-267)^2 / 267 = 0.63 0.03 + 1.08 + 0.63 = 1.74 = χ 2 Wednesday, November 25, 2009
Look it up in table - (or use R/SPSS/something fancier) - http://www2.lv.psu.edu/jxm57/irp/ chisquar.html Wednesday, November 25, 2009
Significant? p ~ 0.4 (we want 0.05) not significant Wednesday, November 25, 2009
Sample data Blue Red Green Total 240 230 330 800 Significant? Wednesday, November 25, 2009
Sum the differences Blue: (240-267)^2 / 267 = 2.73 Red: (230-267)^2 / 267 = 5.12 Green: (330-267)^2 / 267 = 14.86 2.73 + 5.12 + 14.86 = 22 = χ 2 Wednesday, November 25, 2009
Significant? p < 0.001 highly significant Wednesday, November 25, 2009
Wrap-up - Can be a bit of work - But can lead to amazing insights - Plus, data analysis & visualization is really fun Wednesday, November 25, 2009
Google Analytics Wednesday, November 25, 2009
Why? - It's free! - For a small company or project, much better than rolling out some of these analysis tools for yourself Wednesday, November 25, 2009
Signing up - http://analytics.google.com Wednesday, November 25, 2009
Integrating JS Insert before </body> Wednesday, November 25, 2009
Tip: Tracking JS events _trackevent function, takes (category, event, optional_key, optional_value) Wednesday, November 25, 2009
Using Web Optimizer - Attached to AdSense - Provides A/B testing tools with integrated confidence interval and significant different calculations Wednesday, November 25, 2009
Demo Wednesday, November 25, 2009
Final notes Wednesday, November 25, 2009

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Validation and Experimentation Board of Innovation The Growth Revolution Intro Validation

Validation and Experimentation Board of Innovation The Growth Revolution Intro Validation of new Ventures Experimentation Process Assumptions Hypotheses Experiment Types Metrics Wrap-up Introductions HI THERE! 3 ABOUT US

1.08k views • 86 slides

building a culture of experimentation at Spotify @bendressler - experimentation lead user

building a culture of experimentation at Spotify @bendressler - experimentation lead user retention product satisfaction # of track plays predicting things is hard hi expectations why and how we experiment - in a consumer oriented live

1.03k views • 75 slides

Dynamic Delegation of Experimentation Yingni Guo Northwestern University ngni Guo (NU)

Dynamic Delegation of Experimentation Yingni Guo Northwestern University ngni Guo (NU) Delegation of Experimentation 1 / 55 Introduction Delegating Experimentation I study how to manage innovation in a hierarchical organization. A principal

1.68k views • 119 slides

Internet-scale Experimentation The challenges of large-scale networked system experimentation and

Internet-scale Experimentation The challenges of large-scale networked system experimentation and measurements MIT Tech Review on The Loon Project The state of affairs An ever growing Internet ~3 billion people 15 billion devices

1.21k views • 107 slides

Course wrap up J uly 26, 2005 CS 486/ 686 Universit y of Wat erloo Out line Course wrap

Course wrap up J uly 26, 2005 CS 486/ 686 Universit y of Wat erloo Out line Course wrap up Final exam inf o Ot her AI courses AI j obs AI research 2 CS486/686 Lecture Slides (c) 2005 P. Poupart Agent s and

482 views • 12 slides

Course wrap up November 27, 2008 CS 486/686 University of Waterloo Outline Course wrap up

Course wrap up November 27, 2008 CS 486/686 University of Waterloo Outline Course wrap up Final exam info Other AI courses AI jobs AI research 2 CS486/686 Lecture Slides (c) 2008 P. Poupart Topics Covered

448 views • 18 slides

Experimentation in Virtual Environments Will Steptoe 29 th January 2010 Whats in this

Experimentation in Virtual Environments Will Steptoe 29 th January 2010 Whats in this lecture? 1. Introduction to experimentation in VEs 2. Experimental Design 3. Case study What is an Experiment? A set of actions and observations,

692 views • 46 slides

The Cake is Baked...now what? Let cool almost to room temperature Wrap in plastic wrap

The Cake is Baked...now what? Let cool almost to room temperature Wrap in plastic wrap twice Wrap in foil once and FREEZE **Buttercream, frosted cakes, frosted cupcakes, cookies, cookies with royal icing, cake pops all freeze great,

206 views • 4 slides

The Role and Relevance of Experimentation in Informatics Viola Schiaffonati Artificial

The Role and Relevance of Experimentation in Informatics Viola Schiaffonati Artificial Intelligence and Robotics Laboratory Politecnico di Milano Experimentation: role and relevance Starting point: philosophy of science perspective

881 views • 20 slides

YEAR END WRAP- YEAR END WRAP -UP UP A Review of Legislative, Labour and Employment Law

YEAR END WRAP- YEAR END WRAP -UP UP A Review of Legislative, Labour and Employment Law Developments in 2008 Colleen Dunlop Adrian Schofield November 28, 2008 1 www.emondharnden.com Session Overview Case Law Update Keays

693 views • 31 slides

Session 16: Wrap up 1 1. Strategic commitment (from Session 15). 2. Entry as a congestion

P1 SepOct 2012 Timothy Van Zandt Prices & Markets Page 1 Session 16 Wrap up Session 16: Wrap up 1 1. Strategic commitment (from Session 15). 2. Entry as a congestion game. 3. Wrap up. Part of An entry deterrance

425 views • 10 slides

Seminar Series C E E R Cyber-Physical Experimentation Environment for RADICS Advancing the

Seminar Series C E E R Cyber-Physical Experimentation Environment for RADICS Advancing the state of art, CEER is a game changer. A generational leap in capabilities for Cyber- Physical Experimentation in the Electric Power Grid Increased

792 views • 49 slides

Outline Course wrap up Final exam info Course wrap up Other AI courses AI jobs

Outline Course wrap up Final exam info Course wrap up Other AI courses AI jobs July 25, 2006 AI research CS 486/686 University of Waterloo 2 CS486/686 Lecture Slides (c) 2006 P. Poupart Rational Agents Agents and

592 views • 3 slides

Animal Testing Angie Bannister What is Animal Testing? animal testing, animal experimentation or

Animal Testing Angie Bannister What is Animal Testing? animal testing, animal experimentation or animal research, (refers to the experimentation carried out on animals) Cosmetics, pharmaceutical drugs, cleaning chemicals etc. tested

496 views • 13 slides

Session 16: Wrap up 1. Strategic commitment (from Session 15). 2. Entry as a congestion

Session 16: Wrap up 1. Strategic commitment (from Session 15). 2. Entry as a congestion game. 3. Wrap up. Session 16 Wrap up Slide 1 P1 SepOct 2012 Timothy Van Zandt Prices & Markets Part of An entry deterrance game

426 views • 20 slides

FOOD SERVICE UPDATE Seed Change Grant Wrap up Wrap up of the Seed Change grant which sent us

FOOD SERVICE UPDATE Seed Change Grant Wrap up Wrap up of the Seed Change grant which sent us to the Farm to School Conference in Madison, Wisconsin Added a second high tunnel, solar powered temperature control, 6 more hydroponic beds, a

228 views • 9 slides

Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation Chen Jiang

Decision Aid Methodologies In Transportation Lecture 2: Duality and Column generation Chen Jiang Hang Transportation and Mobility Laboratory July 19, 2013 Chen Jiang Hang (Transportation and Mobility Laboratory) Decision Aid Methodologies In

788 views • 43 slides

1 Distinguishing Upper and Lower Bounds Triangular Iteration Space Example Simple Algorithm (

Code Generation Using Fourier Motzkin Code Generation Previously Goals Data dependences and loops express outermost loop bounds in terms of symbolic constants and constants Loop transformations express inner loop bounds in

300 views • 4 slides

Bridging Local Systems Strategies for Mental Health and Social Services Collaboration

Bridging Local Systems Strategies for Mental Health and Social Services Collaboration Trillium Southern Region August 30 ,2017 Sponsors NC Institute of Medicine The Duke Endowment The NC Department of Health and Human

486 views • 17 slides

Retrosp specti ctive: Feedback ack-directed Ran andom Test Ge Generation on Carlos Pacheco,

Retrosp specti ctive: Feedback ack-directed Ran andom Test Ge Generation on Carlos Pacheco, Shuvendu Lahiri, Michael D. Ernst , Thomas Ball ICSE 2007 MIP retrospective May 26, 2017 Wh Who loves to write tests? Problem: Developers do

485 views • 32 slides

Financial Results Third Quarter 2020 Forward-Looking Statements & Non-GAAP Financial Measures

Financial Results Third Quarter 2020 Forward-Looking Statements & Non-GAAP Financial Measures This presentation may contain forward-looking statements with respect to the financial condition, results of operations, trends in lending policies,

595 views • 26 slides

Packet Scheduling: Weighted Fair Queueing (WFQ) ( ) and Virtual Clock (VC) and Virtual Clock

Packet Scheduling: Weighted Fair Queueing (WFQ) ( ) and Virtual Clock (VC) and Virtual Clock (VC) WFQ and VC scheduling (Simon S. Lam) 1 2/2/2017 1 Fair Queueing server Flow 1 packets Flow 2 packets R R Flow n packets Round robin

335 views • 18 slides

Rank Idle Time Prediction Driven Last-Level Cache Writeback Zhe Wang, Samira M. Khan, Daniel

Rank Idle Time Prediction Driven Last-Level Cache Writeback Zhe Wang, Samira M. Khan, Daniel A. Jimnez Computer Science Department University of Texas at San Antonio Memory Latency is Performance Bottleneck Memory wall -

484 views • 21 slides

CENG5030 Part 1-4: Switching Activity Bei Yu (Latest update: March 25, 2019) Spring 2019 1 /

CENG5030 Part 1-4: Switching Activity Bei Yu (Latest update: March 25, 2019) Spring 2019 1 / 15 These slides contain/adapt materials developed by Sukumar Jairam et al. (2008). Clock gating for power optimization in ASIC design cycle

388 views • 19 slides