Do you wish you could hear the audio and read the transcription of - - PowerPoint PPT Presentation

do you wish you could hear the audio and read the
SMART_READER_LITE
LIVE PREVIEW

Do you wish you could hear the audio and read the transcription of - - PowerPoint PPT Presentation

Do you wish you could hear the audio and read the transcription of this session? Then come to JavaOne SM Online where this session is available in a multimedia tool with full audio and transcription synced with the slide presentation. JavaOne


slide-1
SLIDE 1

Do you wish you could hear the audio and read the transcription of this session?

Then come to JavaOneSM Online where this session is available in a multimedia tool with full audio and transcription synced with the slide presentation. JavaOne Online offers much more than just multimedia sessions. Here are just a few benefits:

  • 2003 and 2002 Multimedia JavaOne conference sessions
  • Monthly webinars with industry luminaries
  • Exclusive web-only multimedia sessions on Java technology
  • Birds-of-a-Feather sessions online
  • Classified Ads: Find a new job, view upcoming events, buy or

sell cool stuff and much more!

  • Feature articles on industry leaders, Q&A with speakers, etc.

For only $99.95, you can become a member of JavaOne Online for one year. Join today! Visit http://java.sun.com/javaone/online for more details!

slide-2
SLIDE 2 | JavaOne 2003 | Session 2472

Compelling Graphics for Mobile Applications using the Mobile 3D Graphics API (JSR-184)

Jyri Huopaniemi Nokia Research Center Mark Patel Motorola Kari Pulli Nokia Mobile Phones

slide-3
SLIDE 3 | JavaOne 2003 | Session 2472 2

Overall Presentation Goal

Learn about the Mobile 3D Graphics API design concepts, technical solutions, and related standards

slide-4
SLIDE 4 | JavaOne 2003 | Session 2472 3

Speaker’s Qualifications

  • Dr. Jyri Huopaniemi is Senior Research

Manager at Nokia Research Center, and specification lead of JSR-135 (Mobile Media API) and JSR-184 (Mobile 3D Graphics API) JCP standards

  • Mark Patel is a Principal Staff Engineer

responsible for J2ME™ graphics architecture at Motorola

  • Dr. Kari Pulli heads Nokia Mobile Phone’s

graphics research and represents Nokia both at JSR-184 and Khronos OpenGL ES

slide-5
SLIDE 5 | JavaOne 2003 | Session 2472 4

Why 3D Graphics? Phone screens may be flat… but the world certainly isn’t!

slide-6
SLIDE 6 | JavaOne 2003 | Session 2472 5

Applications for Mobile 3-D

  • Games!
  • User interfaces
  • Screen savers
  • Character animation
  • Maps
  • Visualization
slide-7
SLIDE 7 | JavaOne 2003 | Session 2472 6

Presentation Agenda

  • Introduction and Background
  • Mobile 3D API overview
  • File Format
  • Integration with MIDP
  • Summary
  • Q&A
slide-8
SLIDE 8 | JavaOne 2003 | Session 2472

Introduction and Overview

slide-9
SLIDE 9 | JavaOne 2003 | Session 2472 8

What Is JSR-184?

  • JSR-184 = Mobile 3D Graphics API for

J2ME™

  • The first Mobile Java™ 3D graphics standard
  • Optional package for J2ME™, to be used with

MIDP and CLDC 1.1

  • Defines a high-level and low-level interface for

bringing 3-D graphics to MIDP

slide-10
SLIDE 10 | JavaOne 2003 | Session 2472 9

JSR-184 Expert Group

  • Nokia (specification lead)
  • Aplix
  • ARM
  • Bandai
  • Cellon France
  • Cingular Wireless
  • Fathammer
  • France Telecom
  • Fuetrek
  • HI Corporation
  • Hybrid Graphics
  • In-Fusio
  • Insignia Solutions
  • Intel
  • Intergrafx
  • MathEngine
  • Motorola
  • Research In Motion
  • Siemens
  • Sony Ericsson
  • Sumea
  • Sun Microsystems
  • Superscape
  • Symbian
  • Texas Instruments
  • 3d4W
  • Vodafone
  • Zucotto Wireless
slide-11
SLIDE 11 | JavaOne 2003 | Session 2472 10

JSR-184 API Requirements

  • Must haves:

– Retained mode access (scene graph) – Immediate mode access (OpenGL subset or similar) – Possible to use both modes in a unified way – No optional parts in this JSR – Importers for various data types (e.g., meshes, scene graphs) – Minimal garbage collection – Interoperable with other related Java APIs

  • Should haves:

– Implementable within 150 KB on a real mobile terminal

slide-12
SLIDE 12 | JavaOne 2003 | Session 2472 11

JSR-184 Schedule

  • Spring 2002—First discussions
  • April 2002—Filing of JSR
  • June 2002—1st Expert Group meeting
  • Feb 2003—Community review
  • May 2003—Public review
  • July 2003—Proposed final draft
slide-13
SLIDE 13 | JavaOne 2003 | Session 2472

The API Structure

slide-14
SLIDE 14 | JavaOne 2003 | Session 2472 13

Immediate vs. Retained Mode

  • Retained mode

– Based on a scene graph

  • Rooted by a World
  • With a Camera, Lights, objects, etc.

– First: update animation tracks – Then: tell World to render itself

  • Immediate mode

– Set current camera, lights, blending modes, … – Then draw geometry – Similar to OpenGL state machine

slide-15
SLIDE 15 | JavaOne 2003 | Session 2472 14

Scene Graphs Are Made of Nodes

  • Camera

– Perspective/orthographic

  • Group

– For organizing things – World, the root of the world, with an active camera

  • Light

– Point, directional, spot, ambient

  • Mesh

– Geometry + appearance

  • Sprite

– Fast-to-draw 2D images

slide-16
SLIDE 16 | JavaOne 2003 | Session 2472 15

Inherited Node Properties

  • Relative transformation (w.r.t parent,

cumulative)

– TxRxSxM

  • First a generic 4x4 projective matrix M
  • Then scale, rotate, and translate (easier to animate)
  • Additional optional transform by aligning to

– Another node; current camera; pick ray – Replaces R: TxAxSxM

  • Inherited alpha factor

– For per-node transparency effects

slide-17
SLIDE 17 | JavaOne 2003 | Session 2472 16

Mesh

  • Mesh is 3D geometry (with associated Appearances)

– Polygonal surfaces, made of triangle strips – VertexBuffer (location, normal, color, …)

  • Composed of submeshes, made of

– IndexBuffer

(triangle strip indices)

– Appearance (material, texture, …)

  • Submeshes are rendered in the order of

ascending Appearance layers

– Opaque submeshes rendered before transparent ones on a layer

slide-18
SLIDE 18 | JavaOne 2003 | Session 2472 17

MorphingMesh

  • Taking a sum of differences from base mesh

allows arbitrary combination of partial animations, including exaggerations

slide-19
SLIDE 19 | JavaOne 2003 | Session 2472 18

MorphingMesh

  • Several VertexBuffers

– Base

  • Base attributes and non-morphed attributes

(attributes: location, color, texcoord, …)

– Several morph targets

  • Outcome

– A weighted sum of base and differences

( )

‡”

  • i

i i

w B T B R + =

slide-20
SLIDE 20 | JavaOne 2003 | Session 2472 19

SkinnedMesh

  • A skeletally animated Mesh
  • Without skinning, you get cracks; you can also

smoothly interpolate the joints

slide-21
SLIDE 21 | JavaOne 2003 | Session 2472 20

SkinnedMesh

  • Details…

– Nodes (bones) attach to groups of vertices – Vertices can have multiple controlling Nodes

  • The weights can vary

– A simple skeleton can be used to animate a complex mesh

  • Requires less storage than mesh morphing

(and much less than vertex animation)

  • The skin

– Comes from connecting the vertices

slide-22
SLIDE 22 | JavaOne 2003 | Session 2472 21

Appearance

  • Appearance has five component classes

– GeometryMode – CompositingMode – Material – Texture2D – Fog

slide-23
SLIDE 23 | JavaOne 2003 | Session 2472 22

Appearance Modes

  • GeometryMode

– Culling and shading (1–2 sided, smooth/flat, …) – Hints for local viewer (light), persp. correction

  • CompositingMode

– Alpha blend/test, depth test/write/offset, color write

  • Material

– Vertex color tracking (ambient and diffuse) – Ambient, diffuse, specular, emissive, shininess

slide-24
SLIDE 24 | JavaOne 2003 | Session 2472 23

Rest of Appearance

  • Texture2D

– Input the actual image – Filtering (mipmap, point/linear) – Wrapping (clamp/repeat) – Modes (add, blend, decal, modulate, replace)

  • Fog
slide-25
SLIDE 25 | JavaOne 2003 | Session 2472 24

Animation: Keyframes

  • Traditional cartoon process a’la Disney

– Master animators draw the main characters and their poses in keyframes – Junior animators draw the frames between the keyframes (in-betweens)

  • KeyframeSequence

– Each keyframe is a value at a time instance

  • Scalar, vector

– Various interpolation methods

  • Step, linear, spline for scalars, vectors
  • Slerp, squad for orientations (quaternions)
slide-26
SLIDE 26 | JavaOne 2003 | Session 2472 25

AnimationController

  • Controls the sequence

– Do-it-once (open) or looping (closed) – Translates world time into sequence time

  • With several animations for the same target

– A weighted (normalized) average is taken

  • Can bundle up several sequences,

control them together

– Hands, feet, body swings -> walk

slide-27
SLIDE 27 | JavaOne 2003 | Session 2472 26

AnimationTrack

  • AnimationTrack associates

– KeyframeSequence and – AnimationController

with

– Animation target (animatable property)

  • Visibility, shininess, color, scale, …
  • Each Object3D can have several

AnimationTracks, even for the same target

slide-28
SLIDE 28 | JavaOne 2003 | Session 2472 27

Searching in a Scene Graph

  • Picking

– Group and Camera have pick methods – Returns a RayIntersection

  • The first Mesh and data about intersection point

– Nodes can have a scopeID to set pick scope

  • BTW, scopeID can be used also for Lighting
  • Finding

– Every Object3D has a property called userID – You can search for reachable objects with a matching userID

slide-29
SLIDE 29 | JavaOne 2003 | Session 2472 28

Graphics3D

  • The way to do immediate mode rendering
  • Setup the rendering target

– A MIDP Graphics object – Image2D

  • Draw to intersection of

– Rendering target – Graphics3D object – Viewport

  • Set state, draw geometry, repeat
slide-30
SLIDE 30 | JavaOne 2003 | Session 2472 29

Floats vs. Fixed Point

  • Speed

– Per-vertex data should be only 8 or 16 bit integer data – Position, texcoord, normal, color

  • Flexibility

– Not all parts of the API are as speed critical – Floating point is easier to use – Introducing new data types for, e.g., 16.16 fixed point not practical

  • Too slow if done as a class
  • No type safety if used int as a container,
  • perators work just wrong
slide-31
SLIDE 31 | JavaOne 2003 | Session 2472

File Format

slide-32
SLIDE 32 | JavaOne 2003 | Session 2472 31

File Format

  • Allows 3D objects to be loaded from a

resource

– Named resource – Byte array

  • Faster and smaller than creating 3D Objects

programmatically

  • Separates 3D data from application code

– Permits split development model

slide-33
SLIDE 33 | JavaOne 2003 | Session 2472 32

Why a New File Format?

  • Tightly coupled with the API

– Direct mapping between data and Java

  • bjects
  • Re-use of libraries already present on MIDP

devices

– ZLIB decompression – PNG Image decoder

  • No Intellectual Property issues

– Tools and readers can be freely implemented

slide-34
SLIDE 34 | JavaOne 2003 | Session 2472 33

Loading

  • Call Loader.load method using the desired

resource

– Reads the data and returns an array of the root Object3D instances

  • Other objects can be accessed from the roots

as needed

– getChild(int index) – find(int userid)

slide-35
SLIDE 35 | JavaOne 2003 | Session 2472

Integration With MIDP

slide-36
SLIDE 36 | JavaOne 2003 | Session 2472 35

J2ME™ Architecture

MIDP CLDC 1.1 Mobile 3D Graphics

  • Also designed to work with other Java

profiles

slide-37
SLIDE 37 | JavaOne 2003 | Session 2472 36

Creating 3D Applications

  • Must be valid a MIDP application

– Jar file format – Application life-cycle

  • Use existing MIDP User Interface classes to

access the screen

– Canvas – GameCanvas – CustomItem

slide-38
SLIDE 38 | JavaOne 2003 | Session 2472 37

Rendering the User Interface

  • Create a Graphics3D object
  • Bind to a 2D Graphics object

– Links the Graphics3D object to a 2D buffer

  • Render the desired World, Node or

sub-mesh

  • Release the Graphics object

– Flushes the rendered 3D view to the 2D buffer

slide-39
SLIDE 39 | JavaOne 2003 | Session 2472 38

Sample Application

public class MyCanvas extends Canvas { Graphics3D g3d; World myWorld; int currentTime = 0; public MyCanvas() { g3d = Graphics3D.create(); Object[] root = Loader.load(“/myWorld.m3d”); myWorld = root[0]; }

slide-40
SLIDE 40 | JavaOne 2003 | Session 2472 39

Sample Application

protected void paint(Graphics g) { g3d.bindTarget(g); myWorld.animate(currentTime); currentTime += 50; g3d.render(myWorld); g3d.releaseTarget(); } }

slide-41
SLIDE 41 | JavaOne 2003 | Session 2472

Summary

slide-42
SLIDE 42 | JavaOne 2003 | Session 2472 41

Summary

  • JSR-184 defines the Mobile 3D Graphics
  • ptional API for MIDP/CLDC
  • The design includes a low-level

immediate mode as well as a high-level retained mode API

slide-43
SLIDE 43 | JavaOne 2003 | Session 2472 42

If You Only Remember One Thing…

3-D Graphics is a must-have for games— and it will be available for MIDP soon!

slide-44
SLIDE 44 | JavaOne 2003 | Session 2472 43

Feedback

  • The Expert Group welcomes comments on

the specification

  • Comments by e-mail:

jsr-184-comments@jcp.org

slide-45
SLIDE 45

Demo

slide-46
SLIDE 46

Q&A

slide-47
SLIDE 47 | JavaOne 2003 | Session 2472 46