Do you wish you could hear the audio and read the transcription of - - PowerPoint PPT Presentation

Do you wish you could hear the audio and read the transcription of this session?

Then come to JavaOneSM Online where this session is available in a multimedia tool with full audio and transcription synced with the slide presentation. JavaOne Online offers much more than just multimedia sessions. Here are just a few benefits:

• 2003 and 2002 Multimedia JavaOne conference sessions
• Monthly webinars with industry luminaries
• Exclusive web-only multimedia sessions on Java technology
• Birds-of-a-Feather sessions online
• Classified Ads: Find a new job, view upcoming events, buy or

sell cool stuff and much more!

• Feature articles on industry leaders, Q&A with speakers, etc.

For only $99.95, you can become a member of JavaOne Online for one year. Join today! Visit http://java.sun.com/javaone/online for more details!

Compelling Graphics for Mobile Applications using the Mobile 3D Graphics API (JSR-184)

Jyri Huopaniemi Nokia Research Center Mark Patel Motorola Kari Pulli Nokia Mobile Phones

Overall Presentation Goal

Learn about the Mobile 3D Graphics API design concepts, technical solutions, and related standards

Speaker’s Qualifications

• Dr. Jyri Huopaniemi is Senior Research

Manager at Nokia Research Center, and specification lead of JSR-135 (Mobile Media API) and JSR-184 (Mobile 3D Graphics API) JCP standards

• Mark Patel is a Principal Staff Engineer

responsible for J2ME™ graphics architecture at Motorola

• Dr. Kari Pulli heads Nokia Mobile Phone’s

graphics research and represents Nokia both at JSR-184 and Khronos OpenGL ES

Why 3D Graphics? Phone screens may be flat… but the world certainly isn’t!

Applications for Mobile 3-D

• Games!
• User interfaces
• Screen savers
• Character animation
• Maps
• Visualization

Presentation Agenda

• Introduction and Background
• Mobile 3D API overview
• File Format
• Integration with MIDP
• Summary
• Q&A

Introduction and Overview

What Is JSR-184?

• JSR-184 = Mobile 3D Graphics API for

J2ME™

• The first Mobile Java™ 3D graphics standard
• Optional package for J2ME™, to be used with

MIDP and CLDC 1.1

• Defines a high-level and low-level interface for

bringing 3-D graphics to MIDP

JSR-184 Expert Group

• Nokia (specification lead)
• Aplix
• ARM
• Bandai
• Cellon France
• Cingular Wireless
• Fathammer
• France Telecom
• Fuetrek
• HI Corporation
• Hybrid Graphics
• In-Fusio
• Insignia Solutions
• Intel
• Intergrafx
• MathEngine
• Motorola
• Research In Motion
• Siemens
• Sony Ericsson
• Sumea
• Sun Microsystems
• Superscape
• Symbian
• Texas Instruments
• 3d4W
• Vodafone
• Zucotto Wireless

JSR-184 API Requirements

• Must haves:

– Retained mode access (scene graph) – Immediate mode access (OpenGL subset or similar) – Possible to use both modes in a unified way – No optional parts in this JSR – Importers for various data types (e.g., meshes, scene graphs) – Minimal garbage collection – Interoperable with other related Java APIs

• Should haves:

– Implementable within 150 KB on a real mobile terminal

JSR-184 Schedule

• Spring 2002—First discussions
• April 2002—Filing of JSR
• June 2002—1st Expert Group meeting
• Feb 2003—Community review
• May 2003—Public review
• July 2003—Proposed final draft

The API Structure

Immediate vs. Retained Mode

• Retained mode

– Based on a scene graph

• Rooted by a World
• With a Camera, Lights, objects, etc.

– First: update animation tracks – Then: tell World to render itself

• Immediate mode

– Set current camera, lights, blending modes, … – Then draw geometry – Similar to OpenGL state machine

Scene Graphs Are Made of Nodes

• Camera

– Perspective/orthographic

• Group

– For organizing things – World, the root of the world, with an active camera

• Light

– Point, directional, spot, ambient

• Mesh

– Geometry + appearance

• Sprite

– Fast-to-draw 2D images

Inherited Node Properties

• Relative transformation (w.r.t parent,

cumulative)

– TxRxSxM

• First a generic 4x4 projective matrix M
• Then scale, rotate, and translate (easier to animate)
• Additional optional transform by aligning to

– Another node; current camera; pick ray – Replaces R: TxAxSxM

• Inherited alpha factor

– For per-node transparency effects

Mesh

• Mesh is 3D geometry (with associated Appearances)

– Polygonal surfaces, made of triangle strips – VertexBuffer (location, normal, color, …)

• Composed of submeshes, made of

– IndexBuffer

(triangle strip indices)

– Appearance (material, texture, …)

• Submeshes are rendered in the order of

ascending Appearance layers

– Opaque submeshes rendered before transparent ones on a layer

MorphingMesh

• Taking a sum of differences from base mesh

allows arbitrary combination of partial animations, including exaggerations

MorphingMesh

• Several VertexBuffers

– Base

• Base attributes and non-morphed attributes

(attributes: location, color, texcoord, …)

– Several morph targets

• Outcome

– A weighted sum of base and differences

( )

‡”

• i

i i

w B T B R + =

SkinnedMesh

• A skeletally animated Mesh
• Without skinning, you get cracks; you can also

smoothly interpolate the joints

SkinnedMesh

• Details…

– Nodes (bones) attach to groups of vertices – Vertices can have multiple controlling Nodes

• The weights can vary

– A simple skeleton can be used to animate a complex mesh

• Requires less storage than mesh morphing

(and much less than vertex animation)

• The skin

– Comes from connecting the vertices

Appearance

• Appearance has five component classes

– GeometryMode – CompositingMode – Material – Texture2D – Fog

Appearance Modes

• GeometryMode

– Culling and shading (1–2 sided, smooth/flat, …) – Hints for local viewer (light), persp. correction

• CompositingMode

– Alpha blend/test, depth test/write/offset, color write

• Material

– Vertex color tracking (ambient and diffuse) – Ambient, diffuse, specular, emissive, shininess

Rest of Appearance

• Texture2D

– Input the actual image – Filtering (mipmap, point/linear) – Wrapping (clamp/repeat) – Modes (add, blend, decal, modulate, replace)

• Fog

Animation: Keyframes

• Traditional cartoon process a’la Disney

– Master animators draw the main characters and their poses in keyframes – Junior animators draw the frames between the keyframes (in-betweens)

• KeyframeSequence

– Each keyframe is a value at a time instance

• Scalar, vector

– Various interpolation methods

• Step, linear, spline for scalars, vectors
• Slerp, squad for orientations (quaternions)

AnimationController

• Controls the sequence

– Do-it-once (open) or looping (closed) – Translates world time into sequence time

• With several animations for the same target

– A weighted (normalized) average is taken

• Can bundle up several sequences,

control them together

– Hands, feet, body swings -> walk

AnimationTrack

• AnimationTrack associates

– KeyframeSequence and – AnimationController

with

– Animation target (animatable property)

• Visibility, shininess, color, scale, …
• Each Object3D can have several

AnimationTracks, even for the same target

Searching in a Scene Graph

• Picking

– Group and Camera have pick methods – Returns a RayIntersection

• The first Mesh and data about intersection point

– Nodes can have a scopeID to set pick scope

• BTW, scopeID can be used also for Lighting
• Finding

– Every Object3D has a property called userID – You can search for reachable objects with a matching userID

Graphics3D

• The way to do immediate mode rendering
• Setup the rendering target

– A MIDP Graphics object – Image2D

• Draw to intersection of

– Rendering target – Graphics3D object – Viewport

• Set state, draw geometry, repeat

Floats vs. Fixed Point

• Speed

– Per-vertex data should be only 8 or 16 bit integer data – Position, texcoord, normal, color

• Flexibility

– Not all parts of the API are as speed critical – Floating point is easier to use – Introducing new data types for, e.g., 16.16 fixed point not practical

• Too slow if done as a class
• No type safety if used int as a container,
• perators work just wrong

File Format

File Format

• Allows 3D objects to be loaded from a

resource

– Named resource – Byte array

• Faster and smaller than creating 3D Objects

programmatically

• Separates 3D data from application code

– Permits split development model

Why a New File Format?

• Tightly coupled with the API

– Direct mapping between data and Java

• bjects
• Re-use of libraries already present on MIDP

devices

– ZLIB decompression – PNG Image decoder

• No Intellectual Property issues

– Tools and readers can be freely implemented

Loading

• Call Loader.load method using the desired

resource

– Reads the data and returns an array of the root Object3D instances

• Other objects can be accessed from the roots

as needed

– getChild(int index) – find(int userid)

Integration With MIDP

J2ME™ Architecture

MIDP CLDC 1.1 Mobile 3D Graphics

• Also designed to work with other Java

profiles

Creating 3D Applications

• Must be valid a MIDP application

– Jar file format – Application life-cycle

• Use existing MIDP User Interface classes to

access the screen

– Canvas – GameCanvas – CustomItem

Rendering the User Interface

• Create a Graphics3D object
• Bind to a 2D Graphics object

– Links the Graphics3D object to a 2D buffer

• Render the desired World, Node or

sub-mesh

• Release the Graphics object

– Flushes the rendered 3D view to the 2D buffer

Sample Application

public class MyCanvas extends Canvas { Graphics3D g3d; World myWorld; int currentTime = 0; public MyCanvas() { g3d = Graphics3D.create(); Object[] root = Loader.load(“/myWorld.m3d”); myWorld = root[0]; }

Sample Application

protected void paint(Graphics g) { g3d.bindTarget(g); myWorld.animate(currentTime); currentTime += 50; g3d.render(myWorld); g3d.releaseTarget(); } }

Summary

Summary

• JSR-184 defines the Mobile 3D Graphics
• ptional API for MIDP/CLDC
• The design includes a low-level

immediate mode as well as a high-level retained mode API

If You Only Remember One Thing…

3-D Graphics is a must-have for games— and it will be available for MIDP soon!

Feedback

• The Expert Group welcomes comments on

the specification

• Comments by e-mail:

jsr-184-comments@jcp.org

Demo

Q&A