Understanding Hardware Acceleration on Mobile Browsers Ariya - - PowerPoint PPT Presentation

▶
understanding hardware acceleration on mobile browsers
SMART_READER_LITE
LIVE PREVIEW

Understanding Hardware Acceleration on Mobile Browsers Ariya - - PowerPoint PPT Presentation

Dec 22, 2022 5 likes •715 views

Understanding Hardware Acceleration on Mobile Browsers Ariya Hidayat Magical Advice Use translate3d for hardware acceleration Challenges Game vs Web Game Text Textured objects Animation Physics Transformation Large but Still Bounded

slide-1
SLIDE 1

Understanding Hardware Acceleration on Mobile Browsers

Ariya Hidayat

slide-2
SLIDE 2
slide-3
SLIDE 3

Magical Advice

Use translate3d for hardware acceleration

slide-4
SLIDE 4

Challenges

slide-5
SLIDE 5

Game vs Web

slide-6
SLIDE 6

Game

Animation Physics Textured objects Text Transformation

slide-7
SLIDE 7

Large but Still Bounded

slide-8
SLIDE 8

Web Page

Images Text

slide-9
SLIDE 9

Challenges

Predictable contents (assets)

✔

Mostly text

✔

Mostly images

✔

immediate

Initial response

slide-10
SLIDE 10

Browser Rendering

slide-11
SLIDE 11

From Wave to Pixels

Layout engine Network stack Graphics JavaScript engine User interface

slide-12
SLIDE 12

WebKit Components

Render Engine WebCore JavaScript Engine (JSC/V8) Client Interface HTML DOM CSS SVG Canvas

slide-13
SLIDE 13

Platform Abstraction

Network Unicode Clipboard Graphics Theme Events Thread Geolocation Timer

slide-14
SLIDE 14

Graphics Abstraction

iOS Android CoreGraphics Skia Graphics stack GraphicsContext

slide-15
SLIDE 15

SoC = System-on-a-Chip

CPU GPU

slide-16
SLIDE 16

GPU Workflow

Vertices Matrix Rendered Textured

slide-17
SLIDE 17

Optimized for Games

Transformation Textured triangles “Fixed” geometry Parallelism

slide-18
SLIDE 18

Powerful Capabilities

http://epicgames.com/technology/epic-citadel http://www.rage.com/

slide-19
SLIDE 19

http://www.trollquotes.org/619-super-spider-bat-troll-quote/

slide-20
SLIDE 20

Fundamental Physical Limitations

  • Memory
  • Can’t store too much stuff
  • Speed
  • Quad-core CPU can do certain operations better
  • Bandwidth
  • Bottleneck can be in the data transfer
slide-21
SLIDE 21

Traffic Congestion

slide-22
SLIDE 22

Primitive Drawing

slide-23
SLIDE 23

Path is Everything

Path Triangle Rectangle Polygon Ellipse

slide-24
SLIDE 24

Stroke = Outline

Solid Dashed Dotted Textured

slide-25
SLIDE 25

Brush = Fill

Solid None Gradient Textured

slide-26
SLIDE 26

Gradient = Non-uniform Pixel Values

slide-27
SLIDE 27

Drawing Time: Solid vs Gradient

Only border Solid color fill Linear gradient fill Radial gradient fill 14x slower (depending on the dimension)

slide-28
SLIDE 28

Path Approximation

Curves Polygon

slide-29
SLIDE 29

Smooth via Antialiasing

Multiple levels of transparency Perfect for parallelism

slide-30
SLIDE 30

Shadow

Involved pixel “blending” Often obstructed

slide-31
SLIDE 31

Drawing Time: Solid vs Blur Shadow

Solid shadow Blur shadow 20x slower (depending on the blur radius)

slide-32
SLIDE 32

CSS = Box Model

http://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of Stacking Contexts

slide-33
SLIDE 33

Shadow Abuse

12,000 pixels

Blur shadow

slide-34
SLIDE 34

Transformation

Scaling Rotation Perspective

slide-35
SLIDE 35

Text Rasterization

slide-36
SLIDE 36

Font Atlas

Bye

Buffer

slide-37
SLIDE 37

From Simple to Complex

Hello!

Simple shape Solid color Curves Gradient brush Textured stroke Blur shadow Serif text Rotated Make it as an image

slide-38
SLIDE 38

Library Instrumentation

platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff restorePlatformState platformDestroy platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff restorePlatformState platformDestroy platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff fillRect 0,0 800x556 color ff ff ff ff

slide-39
SLIDE 39

Painting Performance

16:24:04.070 D/webcoreglue( 273): drawContent 11 ms 16:24:04.110 D/webcoreglue( 273): drawContent 13 ms 16:24:04.150 D/webcoreglue( 273): drawContent 13 ms 16:24:04.190 D/webcoreglue( 273): drawContent 10 ms 16:24:04.240 D/webcoreglue( 273): drawContent 10 ms 16:24:04.280 D/webcoreglue( 273): drawContent 13 ms 16:24:04.320 D/webcoreglue( 273): drawContent 13 ms 16:24:04.360 D/webcoreglue( 273): drawContent 13 ms 16:24:06.080 D/webcoreglue( 273): drawContent 12 ms 16:24:06.140 D/webcoreglue( 273): drawContent 10 ms 16:24:06.180 D/webcoreglue( 273): drawContent 13 ms 16:24:06.230 D/webcoreglue( 273): drawContent 14 ms 16:24:06.600 D/webcoreglue( 273): drawContent 26 ms 16:24:06.640 D/webcoreglue( 273): drawContent 13 ms 16:24:06.860 D/webcoreglue( 273): drawContent 33 ms 16:24:06.890 D/webcoreglue( 273): drawContent 12 ms 16:24:06.930 D/webcoreglue( 273): drawContent 13 ms 16:24:06.960 D/webcoreglue( 273): drawContent 13 ms 16:24:07.000 D/webcoreglue( 273): drawContent 13 ms

slide-40
SLIDE 40

Backing Store

slide-41
SLIDE 41

Maps

Tiles

slide-42
SLIDE 42

Responsive Interface

Rendering User interaction Processor

slide-43
SLIDE 43

Pinch to Zoom

when you pinch...

slide-44
SLIDE 44

Rendering vs Interaction

Web Page Rendering User interaction

slide-45
SLIDE 45

Checkerboard Pattern

Poor man’s indicator

  • f performance
slide-46
SLIDE 46

Progressive Rendering

Crisp but slow Fast but blurry

slide-47
SLIDE 47

Perceived Responsiveness

User pinches Smooth scaled Blocky (but fast!)

slide-48
SLIDE 48

Tiling System

CPU GPU

.... ....

Streamed

What happens if the page background changes color?

slide-49
SLIDE 49

Tile Transformation

Panning = Translation Zooming = Scaling

slide-50
SLIDE 50

Y U NO position:fixed

slide-51
SLIDE 51

Layer & Compositing

slide-52
SLIDE 52

Typical Animation

  • pacity, movement, scaling, rotation, ...
slide-53
SLIDE 53

Principles of Fluid Animation

1

At the beginning, push as many resources as possible to the GPU

2

During the animation, minimize CPU-GPU interaction

slide-54
SLIDE 54

Immediate vs Retained

draw the shape at (x, y) x = x + 10 blit the buffer at (x, y) x = x + 10

For every animation tick... At the beginning...

draw the shape onto a buffer

Off main thread

slide-55
SLIDE 55

Mechanics of Animation

Initialization Animation step “Hey, this is good stuff. Cache it as texture #42.” “Apply [operation] to texture #42.”

slide-56
SLIDE 56

Elements = Layer

slide-57
SLIDE 57

Logical 3-D

slide-58
SLIDE 58

Compositing By Force

  • webkit-transform: translateZ(0)

Not needed for CSS animation!

.someid {

  • webkit-animation-name: somekeyframeanimation;
  • webkit-animation-duration: 7s;
  • webkit-transform: translateZ(0);

}

Don’t do that

slide-59
SLIDE 59

Debugging in Safari

defaults write com.apple.Safari IncludeInternalDebugMenu 1

slide-60
SLIDE 60

Compositing Indicators

Composited layer Container layer No associated texture Overflow Texture (number = upload)

slide-61
SLIDE 61

Avoid Texture Reupload

No reupload Upload

Opacity Position Size Animation Everything else!

“Hardware accelerated CSS”

slide-62
SLIDE 62

Examples

slide-63
SLIDE 63

Color Transition

@-webkit-keyframes box { 0% { -webkit-transform: background-color: blue; } 100% { -webkit-transform: background-color: green; } } Need a new texture uploaded to the GPU for every in-between color

slide-64
SLIDE 64

Color Transition: The Trick

Blended with

slide-65
SLIDE 65

Prepare and Reuse

Viewport Hide the layer offscreen

slide-66
SLIDE 66

Timer Latency

Depending on user reaction

  • Animation end triggers the JavaScript callback
  • JavaScript code kicks the next animation

Prepare both animation and hide the “wrong” one

slide-67
SLIDE 67

Avoid Overcapacity

.... ....

Texture upload

Think of the GPU memory like a cache.

slide-68
SLIDE 68

Wrap-up

slide-69
SLIDE 69

Hardware Acceleration

Drawing primitives Backing stores Layer & compositing

slide-70
SLIDE 70

Traditional graphics programming has been always full of tricks. It will always be. There is no such

slide-71
SLIDE 71

Thank You

ariya.hidayat@gmail.com @AriyaHidayat ariya.ofilabs.com Wed 1:45 PM Office Hour