Designing Applications that See Designing Applications that See - - PowerPoint PPT Presentation

▶
designing applications that see designing applications
SMART_READER_LITE
LIVE PREVIEW

Designing Applications that See Designing Applications that See - - PowerPoint PPT Presentation

Sep 25, 2022 209 likes •742 views

stanford hci group / cs377s Designing Applications that See Designing Applications that See Lecture 6: Processing Dan Maynes-Aminzade 24 January 2008 24 January 2008 Designing Applications that See http://cs377s.stanford.edu R Reminders i d

slide-1
SLIDE 1

stanford hci group / cs377s

Designing Applications that See Designing Applications that See Lecture 6: Processing

Dan Maynes-Aminzade 24 January 2008 24 January 2008

Designing Applications that See http://cs377s.stanford.edu

slide-2
SLIDE 2

R i d Reminders

A i t # h d d b k t T d Assignment #1 handed back next Tuesday Assignment #2 released next Tuesday g y

24 January 2008 2 Lecture 6: Processing

slide-3
SLIDE 3

Learn More Tools for Your Project Learn More Tools for Your Project

CS247L h l b i b t CS247L has open lab sessions about many useful tools for physical prototyping W d d 8 i W ll b Wednesdays 6-8PM in Wallenberg 332 Upcoming Lab Sessions:

January 30: Flash February 6: Mobile Interaction February 13: Soldering and Electronics (meet at CCRMA instead of Wallenberg 332) February 20: Physical Computing With d.Tools

More info: http://cs247.stanford.edu/lab.html

24 January 2008 3 Lecture 6: Processing

slide-4
SLIDE 4

T d ’ G l Today’s Goals

L th b i f th P i Learn the basics of the Processing environment Understand how to produce and publish Processing applets Processing applets Learn how to capture and process live video in the Processing framework Experiment with color and motion tracking Experiment with color and motion tracking

24 January 2008 4 Lecture 6: Processing

slide-5
SLIDE 5

O tli Outline

P i i t d ti Processing introduction Work through some Processing examples g g p JMyron introduction k b d l Look at basic video processing examples Build some motion and color tracking Build some motion and color tracking examples Add i t ti it t l Add interactivity to our examples

24 January 2008 5 Lecture 6: Processing

slide-6
SLIDE 6

Wh t i P i ? What is Processing?

A t J il An easy-to-use Java compiler A development environment p Focused on interactive graphics, sound, and i ti animation Produces both locally-run programs and y p g web-embeddable applets Can be used together with “real” Java Can be used together with real Java

24 January 2008 6 Lecture 6: Processing

slide-7
SLIDE 7

P i P ti Processing Perspective

A d l t t l f l i A development tool for exploring multimedia programming An educational tool for learning programming fundamentals programming fundamentals An ideation tool or “electronic sketchbook” for trying out ideas Targeted for designers, artists, beginning Targeted for designers, artists, beginning programmers

24 January 2008 7 Lecture 6: Processing

slide-8
SLIDE 8

Ni Thi b t P i Nice Things about Processing

T k f l t f th i t Takes care of a lot of the annoying setup logistics for doing video and graphics in Java Easy to create interesting dynamic visuals programmatically programmatically Allows quick experimentation Strong focus on graphics, sound, and simple interactivity (unlike traditional Java interactivity (unlike traditional Java programming with a text console)

24 January 2008 8 Lecture 6: Processing

slide-9
SLIDE 9

G tti H l P i Getting Help on Processing

L k t th b ilt i l Look at the built-in examples More examples: Function reference:

http://www.processing.org/learning/

u t Discussion forums:

http://www.processing.org/reference/

Discussion forums:

http://www.processing.org/discourse/

User-contributed code samples:

http://www.processinghacks.com/

24 January 2008 9 Lecture 6: Processing

p // p g /

slide-10
SLIDE 10

A il bl Lib i Available Libraries

Built In Built-In

Video Networking Networking Serial Communication Importing XML, SVG Importing XML, SVG Exporting PDF, DXF, etc.

External Contributions te a Co t but o s

Sound: Ess, Sonia Computer Vision: JMyron, ReacTIVision, p y BlobDetection Interface: proCONTROLL, Interfascia M th Many others

24 January 2008 10 Lecture 6: Processing

slide-11
SLIDE 11

A Q i k T A Quick Tour

24 January 2008 11 Lecture 6: Processing

slide-12
SLIDE 12

T lb B tt Toolbar Buttons

Run Stop Stop New Open Save Save Export

24 January 2008 12 Lecture 6: Processing

slide-13
SLIDE 13

Sk t h Sketches

24 January 2008 13 Lecture 6: Processing

slide-14
SLIDE 14

T b Tabs

24 January 2008 14 Lecture 6: Processing

slide-15
SLIDE 15

C di t Coordinates

24 January 2008 15 Lecture 6: Processing

slide-16
SLIDE 16

P i M d Programming Modes

B i Basic

For drawing static images and learning i f d t l programming fundamentals

Continuous

Provides a setup() and draw() structures and allows writing custom functions and classes and using keyboard and mouse events

Java

Most flexible mode, giving access to the full Java programming language p g g g g

24 January 2008 16 Lecture 6: Processing

slide-17
SLIDE 17

B i M d Basic Mode

i (200 200) size(200, 200); background(255); g noStroke(); i fill(255, 204, 0); rect(30, 20, 50, 50); ( , , , );

24 January 2008 17 Lecture 6: Processing

slide-18
SLIDE 18

C ti M d Continuous Mode

void setup() { void setup() { size(200, 200); noStroke(); background(255); background(255); fill(0, 102, 153, 204); smooth(); noLoop(); noLoop(); } void draw() { circles(40 80); circles(40, 80); circles(90, 70); } void circles(int x int y) { void circles(int x, int y) { ellipse(x, y, 50, 50); ellipse(x+20, y+20, 60, 60); }

24 January 2008 18 Lecture 6: Processing

slide-19
SLIDE 19

C ti M d Continuous Mode

void setup() { void setup() { size(200, 200); rectMode(CENTER); noStroke(); fill(0, 102, 153, 204); } void draw() { b k d(255) background(255); rect(width-mouseX, height-mouseY, 50, 50); rect(mouseX, mouseY, 50, 50); ( , , , ); }

24 January 2008 19 Lecture 6: Processing

slide-20
SLIDE 20

J M d Java Mode

public class MyDemo extends PApplet { public class MyDemo extends PApplet { void setup() { size(200, 200); rectMode(CENTER); noStroke(); fill(0, 102, 153, 204); fill(0, 102, 153, 204); } void draw() { b k d(255) background(255); rect(width-mouseX, height-mouseY, 50, 50); rect(mouseX, mouseY, 50, 50); ( , , , ); } }

24 January 2008 20 Lecture 6: Processing

slide-21
SLIDE 21

S B i S t St t t Some Basic Setup Statements

// specifies window size // specifies window size size(200, 200); // specifies background color // p g background(102); // disables filling in shapes noFill(); // disables drawing lines St k () noStroke(); // set fill color fill(255 100 100); fill(255,100,100); // set stroke color stroke(100,255,100); ( , , );

24 January 2008 21 Lecture 6: Processing

slide-22
SLIDE 22

S B i D i F ti Some Basic Drawing Functions

// draw a point in the middle // draw a point in the middle // width and height store the // window size point(width/2, height/2); // draw a 20x20 rectangle rect(10,10,20,20); rect(10,10,20,20); // draw an ellipse ellipse(50,50,30,30); // d i l h // draw an irregular shape beginShape(); vertex(60, 40); vertex(160, 10); ( , ); ( , ); vertex(170, 150); vertex(60, 150); endShape();

24 January 2008 22 Lecture 6: Processing

slide-23
SLIDE 23

S t d D Setup and Draw

void setup() { void setup() { size(200, 200); stroke(255); stroke(255); frameRate(30); } float y = 100; void draw() { () { background(0); y = (y+1) % height; y (y ) g line(0, y, width, y); }

24 January 2008 23 Lecture 6: Processing

slide-24
SLIDE 24

L noLoop

void setup() { void setup() { size(200, 200); stroke(255); ( ); frameRate(30); noLoop(); } float y = 100; id d () { void draw() { background(0); y = (y+1) % height; y = (y+1) % height; line(0, y, width, y); }

24 January 2008 24 Lecture 6: Processing

slide-25
SLIDE 25

Loop

id P d() { void mousePressed() { loop(); p }

24 January 2008 25 Lecture 6: Processing

slide-26
SLIDE 26

R d Redraw

id P d() { void mousePressed() { redraw(); }

24 January 2008 26 Lecture 6: Processing

slide-27
SLIDE 27

E t H dl Event Handlers

D d() mouseDragged() mouseMoved() mousePressed() mouseReleased() ... keyReleased() keyPressed()

24 January 2008 27 Lecture 6: Processing

slide-28
SLIDE 28

M D i Mouse Drawing

void setup() { void setup() { size(200, 200); background(50); background(50); } void draw() { void draw() { stroke(255); if(mousePressed) { ( ) { line(mouseX, mouseY, pmouseX, pmouseY); p , p ) } }

24 January 2008 28 Lecture 6: Processing

slide-29
SLIDE 29

F ti Functions

void draw target(int xloc void draw_target(int xloc, int yloc, int size, int num) { float grayvalues = 255/num; float grayvalues = 255/num; float steps = size/num; f (i i 0 i i ) { for(int i=0; i<num; i++) { fill(i*grayvalues); ellipse(xloc, yloc, size-i*steps, size-i*steps); } }

24 January 2008 29 Lecture 6: Processing

slide-30
SLIDE 30

Oth B i C t Other Basic Concepts

Th b h h ld t These behave how you would expect (exactly as they do in Java)

Data types (int, float, boolean) Arrays Arrays Loops C d l d L l O Conditionals and Logical Operators Strings Variables and Scoping

24 January 2008 30 Lecture 6: Processing

slide-31
SLIDE 31

Images

i (200 200) size(200, 200); PImage img; g g img = loadImage("tennis.jpg"); i i image(img, 0, 0); image(img, 0, 0, img.width/10, g ( g, , , g / , img.height/10);

24 January 2008 31 Lecture 6: Processing

slide-32
SLIDE 32

C l S Color Spaces

noStroke(); noStroke(); colorMode(RGB, 100); for(int i=0; i<100; i++) { for(int j=0; j<100; j++) { for(int j=0; j<100; j++) { stroke(i, j, 0); point(i, j); } } colorMode(HSB, 100); for(int i=0; i<100; i++) { for(int j=0; j<100; j++) { stroke(i, j, 100); j point(i, j); } }

24 January 2008 32 Lecture 6: Processing

slide-33
SLIDE 33

R di Pi l D t Reading Pixel Data

PImage img; PImage img; size(300,300); noStroke(); img = loadImage("monzy.jpg"); noLoop(); for (int x=0; x<img.width; x+=5) { for (int x 0; x<img.width; x+ 5) { for (int y=0; y<img.height; y+=5) { int pixelcolor = img pixels[x+y*img width]; img.pixels[x+y*img.width]; fill(pixelcolor); ellipse(x,y,4,4); } }

24 January 2008 33 Lecture 6: Processing

slide-34
SLIDE 34

L di Vid Loading Video

import processing video *; import processing.video. ; Movie myMovie; void setup() { size(320, 240); myMovie = new Movie(this, "ball.mov"); myMovie.loop(); myMovie.loop(); } void draw(){ // i (255 20) // tint(255, 20); image(myMovie, mouseX, mouseY); } void movieEvent(Movie m) { m.read(); }

24 January 2008 34 Lecture 6: Processing

slide-35
SLIDE 35

C t i Vid Capturing Video

import processing video *; import processing.video. ; Capture myCapture; void setup() { size(160, 120); String s = "Logitech QuickCam Pro 4000-WDM"; myCapture = new Capture(this, s, width, myCapture new Capture(this, s, width, height, 30); } void captureEvent(Capture myCapture) { void captureEvent(Capture myCapture) { myCapture.read(); } void draw() { image(myCapture, 0, 0); }

24 January 2008 35 Lecture 6: Processing

slide-36
SLIDE 36

P Vid (Si l ) Process Video (Simple)

void draw() { void draw() { for (int i=0; i<width; i+=5) { f (i t j 0 j<h i ht j+ 5) { for (int j=0; j<height; j+=5) { int pixel = myCapture.pixels[i+width*j]; fill(pixel); ellipse(i,j,5,5); } } }

24 January 2008 36 Lecture 6: Processing

slide-37
SLIDE 37

P Vid (M C l ) Process Video (More Complex)

  • Declare some new global variables:

Declare some new global variables: int numPixels; int blockSize = 10; color myMovieColors[];

  • Initiali e ariables in setup()
  • Initialize variables in setup():

noStroke(); background(0); numPixels = width / blockSize; M i C l l [ Pi l * Pi l ] myMovieColors = new color[numPixels * numPixels];

  • Add to captureEvent:

for(int j=0; j<numPixels; j++) { for(int i=0; i<numPixels; i++) { myMovieColors[j*numPixels + i] = myCapture.get(i*blockSize, j*blockSize); } }

  • Replace draw() event:

for(int j=0; j<numPixels; j++) { for(int j=0; j<numPixels; j++) { for(int i=0; i<numPixels; i++) { fill(myMovieColors[j*numPixels + i]); rect(i*blockSize, j*blockSize, blockSize-1 blockSize-1); } } blockSize-1, blockSize-1); } }

24 January 2008 37 Lecture 6: Processing

slide-38
SLIDE 38

Oth E l Other Examples

24 January 2008 38 Lecture 6: Processing

slide-39
SLIDE 39

B i C l T ki Basic Color Tracking

for ( int x=0;x<video.width;x++) { ( ; ; ) { for ( int y=0;y<video.height;y++) { int loc = x + y*video.width; color currentColor = video pixels[loc]; video.pixels[loc]; float r1 = red(currentColor); float g1 = green(currentColor); float b1 = blue(currentColor); fl t 2 d(t kC l ) float r2 = red(trackColor); float g2 = green(trackColor); float b2 = blue(trackColor); float d = dist(r1,g1,b1,r2,g2,b2); if (d < closestDiff) { closestDiff = d; closestX = x; closestY = y; y; } } }

24 January 2008 39 Lecture 6: Processing

slide-40
SLIDE 40

B tt T ki ith JM Better Tracking with JMyron

24 January 2008 40 Lecture 6: Processing

slide-41
SLIDE 41

JM S t JMyron Setup

i t JM * import JMyron.*; JMyron m; y void setup(){ i size(320,240); m = new JMyron(); y (); m.start(width,height); }

24 January 2008 41 Lecture 6: Processing

slide-42
SLIDE 42

JM D i JMyron Drawing

void draw(){ void draw(){ m.update();//update the camera view i t[] i i () int[] img = m.image(); loadPixels(); for(int i=0;i<width*height;i++) { pixels[i] = img[i]; } updatePixels(); }

24 January 2008 42 Lecture 6: Processing

slide-43
SLIDE 43

JM Cl JMyron Cleanup

bli id t (){ public void stop(){ m.stop(); p super.stop(); }

24 January 2008 43 Lecture 6: Processing

slide-44
SLIDE 44

JM C l T ki JMyron Color Tracking

S t th l t ki Setup the color tracking m.trackColor(255,255,255,200); m.minDensity(100); Draw boxes around the detected regions g int[][] b = m.globBoxes(); for(int i=0;i<b length;i++) { for(int i=0;i<b.length;i++) { rect(b[i][0],b[i][1], b[i][2], b[i][3]); }

24 January 2008 44 Lecture 6: Processing

slide-45
SLIDE 45

D i “Gl b ” Drawing “Globs”

int list[][][] m globPixels(); int list[][][] = m.globPixels(); for(int i=0; i<list.length;i++) { i t[][] i lli t li t[i] int[][] pixellist = list[i]; if(pixellist!=null) { b i Sh (POINTS) beginShape(POINTS); for(int j=0;j<pixellist.length;j++) { i i j vertex(pixellist[j][0], pixellist[j][1]); } endShape(); } }

24 January 2008 45 Lecture 6: Processing

slide-46
SLIDE 46

Oth U f l JM F ti Other Useful JMyron Functions

G t th i l l i Get the average pixel value across a region:

int c = m.average(mouseX-20, mouseY-20, mouseX+20, mouseY+20);

Get the center points of the globs: t t t p t t g

int[][] gcs = m.globCenters();

Get the bounding quads of the globs Get the bounding quads of the globs:

int[][] bqs = m.globQuads(20,200);

24 January 2008 46 Lecture 6: Processing

slide-47
SLIDE 47

B k d S bt ti Background Subtraction

S t t f d ti it Set rate of adaptivity: m.adaptivity(10); p y Take a snapshot of the background for diff i differencing: m.adapt(); Get the difference image: i t[] i diff I () int[] img = m.differenceImage();

24 January 2008 47 Lecture 6: Processing

slide-48
SLIDE 48

C t lli C Controlling a Cursor

24 January 2008 48 Lecture 6: Processing

slide-49
SLIDE 49

E ti A l t Exporting an Applet

24 January 2008 49 Lecture 6: Processing

slide-50
SLIDE 50

Si i A l t Signing an Applet

  • Generate a keystore:

Generate a keystore:

$ keytool -genkey -alias signFiles

  • keystore mystore
  • keypass thepassword

keypass thepassword

  • dname "CN=projname, OU=name,

O=company, L=location, S=state, C=country" -storepass thepassword y p p

  • Export a certificate file (optional):

$ keytool -export -keystore mystore

  • storepass thepassword

storepass thepassword –alias signFiles

  • file mycertificate.cer
  • Sign your jar file
  • Sign your jar file:

$ jarsigner -keystore mystore

  • storepass thepassword -keypass thepassword

i dj t t j i t j i Fil

  • signedjar output.jar input.jar signFiles

24 January 2008 50 Lecture 6: Processing

(courtesy of Kevin Cox)

slide-51
SLIDE 51

Summary

P i id f i l Processing provides a fun, easy, visual way to program interactive graphics Built-in computer vision capabilities are somewhat limited but you can still do many somewhat limited, but you can still do many interesting things (and you could always try d i i l li ) doing your own pixel wrangling) Check out the examples and take a look at p the various external libraries

24 January 2008 51 Lecture 6: Processing