Getting it Booking right Using Data to make Decisions Iaroslav - - PowerPoint PPT Presentation

getting it booking right using data to make decisions
SMART_READER_LITE
LIVE PREVIEW

Getting it Booking right Using Data to make Decisions Iaroslav - - PowerPoint PPT Presentation

Getting it Booking right Using Data to make Decisions Iaroslav Khramov | GOTO conference 2014 Are you sure? Are you sure that new design is good? Are you sure this change will not break the app? Are you sure customers will like


slide-1
SLIDE 1

Getting it Booking right – Using Data to make Decisions

Iaroslav Khramov | GOTO conference 2014

slide-2
SLIDE 2

Are you sure?

  • Are you sure that new design is good?
  • Are you sure this change will not break the

app?

  • Are you sure customers will like our new

feature?

  • Are you sure …. ?

Real data can help!

slide-3
SLIDE 3

Decision-making using data

  • When should you should run an

experiment (use A/B testing)?

  • Need to decide which option is better
  • What are the alternatives?
  • Expert opinion / intuition
  • Historical data
slide-4
SLIDE 4

Why is experimentation better?

  • 1. Our intuition is not very good

(people are complex)

  • 2. Historical data is second-best

(becomes outdated fast)

slide-5
SLIDE 5

What do we get?

  • Reduced design discussions
  • Less dependent on expert opinions
  • Makes sure all changes are actually

(commercial) improvements

  • Additional safety net for new changes in

the app

slide-6
SLIDE 6

How complex is it?

If(newFeatureEnabled == true) { // new feature code } else { // old reliable solution }

slide-7
SLIDE 7

For more control - turn experiments

  • n and off from server

{ Experiment A: ON Experiment B: OFF Experiment C: OFF Experiment D: ON }

Server

init experiments Client

slide-8
SLIDE 8

It’s easy! Can I start experimenting now?

Yes, but first:

  • Pick a metric
  • Find users to include in an experiment
  • Come up with a new idea
slide-9
SLIDE 9

Metrics Metrics are the main determinant of success!

  • 1. Pick your own metric!
  • 2. Diversify metrics!
slide-10
SLIDE 10

What is my metric?

Type of business Possible key metrics Ecommerce: Booking.com, Amazon items sold, conversion, loyalty Content: Euronews, CNN visitors, engagement, ad clicks Social Networking: Facebook, Twitter visitors, engagement, ad clicks Gaming: Angry birds purchases, subscriptions

slide-11
SLIDE 11

Experiment lifecycle

Come up with new idea to improve your app Run an experiment to see which variant is the most successful Select the winner Beat it with new ideas

slide-12
SLIDE 12

Example of an experiment

A B

slide-13
SLIDE 13

Are there any A/B exp tools for mobile that will allow you to start experimenting today?

Yes, just Google it! For example:

  • Airlock (Facebook)
  • Arise.io
  • Apptimize
  • Leanplum
slide-14
SLIDE 14

Thanks!

Any Burning

questions?

slide-15
SLIDE 15

Quotes

…the ability to experiment easily is a critical factor for Web-based applications. The online world is never static. There is a constant flow

  • f new users, new products and new
  • technologies. Being able to figure out quickly

what works and what doesn’t can mean the difference between survival and extinction.

Hal Varian, chief economist for Google, 2007

slide-16
SLIDE 16
slide-17
SLIDE 17

a network of

Olger Warnier - @owarnier

slide-18
SLIDE 18

Started 2009

slide-19
SLIDE 19

2010

slide-20
SLIDE 20

2014

slide-21
SLIDE 21

Times a car charged

750.000 1.500.000 2.250.000 3.000.000

2012 2013 2014

slide-22
SLIDE 22

Connectivity

slide-23
SLIDE 23

Communication

2010: SOAP / XML (e-laad foundation OCPP)! 2011: Custom Protobuf/HTTP Keep-Alive protocol! 2013: JSON + WebSockets (OCPP 2 beta) OCPP = Open ChargePoint Protocol! http://openchargealliance.org

slide-24
SLIDE 24

SOAP

slide-25
SLIDE 25

Protobuf + Keep-

slide-26
SLIDE 26

JSON + Web Sockets

  • Compressed connection
  • Estimated data usage 2.7 MB
  • Theoretical optimum (perfect network) 0,14 MB
slide-27
SLIDE 27

Network coverage

slide-28
SLIDE 28

! ! ! ! ! !

OSI layers - network

slide-29
SLIDE 29

Lessons learned

Type of protocol saves data! Optimise functional usage saves more data! Lengthy stable connections save data! Cater for offline use! Mobile networks are costly

slide-30
SLIDE 30

Websockets & Akka

Atmosphere (java websockets framework)! Atmosphere to Akka Actor framework! JSON4S

slide-31
SLIDE 31

Atmosphere

<servlet>! <description>AtmosphereServlet</description>! <servlet-name>AtmosphereServlet</servlet-name>! <servlet-class>org.atmosphere.cpr.AtmosphereServlet</servlet-class>! <init-param>! <param-name>org.atmosphere.cpr.AtmosphereInterceptor.disableDefaults</param-name>! <param-value>true</param-value>! </init-param>! </servlet>! <servlet-mapping>! <servlet-name>AtmosphereServlet</servlet-name>! <url-pattern>/ocppws/*</url-pattern>! </servlet-mapping>

slide-32
SLIDE 32

Atmosphere & Akka

@Singleton! @WebSocketHandlerService! class WebSocketEntryPoint extends…

contains a routing actor

slide-33
SLIDE 33

Atmosphere & Akka

private def sendWebSocketEvent(webSocket: WebSocket, event: WebSocketEvent) =! withWebSocketsActor { webSocketsActor =>! webSocketsActor ! Routed(webSocket.resource.uuid, event)! }! ! private def withWebSocketsActor[T](f: ActorRef => T): T = {! webSocketsActor match {! case Some(wsa) => f(wsa)! case None => sys.error("Trying to process websocket traffic before WebSocketEntryPoint is configured")! }! }

slide-34
SLIDE 34

The actor

It’s wise to have an actor that Routes the request to a processing actor! This frees the actor of its’ job and allow it to take (and route) a new message

slide-35
SLIDE 35

JSON4S

  • bject OcppWsMessageHandling extends Logging {!

implicit val formats: Formats = ! DefaultFormats ++ TransportMessageJsonSerializers() class RequestMessageJsonFormat extends CustomSerializer[RequestMessage] (format => (! {! case JArray(JInt(callType) :: JString(callId) :: ! ! ! ! ! ! JString(action) :: payload :: Nil) =>! RequestMessage(callId, action, payload)! },! {! case x: RequestMessage =>! JArray(JInt(BigInt(2)) :: JString(x.callId) :: ! ! ! ! ! ! ! ! JString(x.procedureName) :: x.payload :: Nil)! }))!

slide-36
SLIDE 36
slide-37
SLIDE 37

There’s An App For That Innovating @ ING mobile.

GOTO Amsterdam

slide-38
SLIDE 38

Itroduction

Pim Stolk iOS Developer at ING since January 2011 Nandini Chauhan iOS Developer from Capgemini working at ING

slide-39
SLIDE 39
slide-40
SLIDE 40
slide-41
SLIDE 41

Microsoft Messenger Buddy +

Our first App

=

slide-42
SLIDE 42

We launched in 2010 ...

slide-43
SLIDE 43

One Star!

slide-44
SLIDE 44

What where the reasons for failure?

slide-45
SLIDE 45

Waterfall

slide-46
SLIDE 46
slide-47
SLIDE 47

Plaatje slak

slide-48
SLIDE 48
slide-49
SLIDE 49
slide-50
SLIDE 50

We CAN do IT !

slide-51
SLIDE 51

a completely new App emerged

slide-52
SLIDE 52

Reminder!

slide-53
SLIDE 53

@ 8th of November 2011

slide-54
SLIDE 54

And here we go again ...

slide-55
SLIDE 55

100.000 activations

after just three days

slide-56
SLIDE 56

2 million downloads

1.6 million active apps

1.2 million customers

a million logins a day One year, four months later...

slide-57
SLIDE 57
slide-58
SLIDE 58

22 22

Listen

slide-59
SLIDE 59

Reviews

slide-60
SLIDE 60
slide-61
SLIDE 61

Features

slide-62
SLIDE 62

GeoBlocking:

slide-63
SLIDE 63

GeoBlocking:

slide-64
SLIDE 64

GeoBlocking:

slide-65
SLIDE 65

Balance before login:

slide-66
SLIDE 66

iDEAL Payment:

slide-67
SLIDE 67

Yvonne for testing

slide-68
SLIDE 68

Future

slide-69
SLIDE 69

Thank you