English as a Macro Language & Programming Environment for Lisp - - PowerPoint PPT Presentation

Henry Lieberman • MIT Media Lab

English as a Macro Language & Programming Environment for Lisp

Henry Lieberman and Hugo Liu

MIT Media Lab Cambridge, Mass. USA http://www.media.mit.edu/~lieber

Henry Lieberman • MIT Media Lab

The crazy dream: Programming in English

Programming languages are too difficult for human beings People can express procedures in English (or

• ther natural language)

Why not program in natural language? That was the original idea behind Cobol (whereas Fortran tried to make programming like math)

Henry Lieberman • MIT Media Lab

Natural Language Programming and Lisp

Lisp community has the best tradition of always looking for new, higher-level programming ideas Lisp well suited for natural language analysis Lisp as a target language for translation from natural language to code

Henry Lieberman • MIT Media Lab

Well, what makes you think you can do it?

That was then, this is now Natural language processing technology has vastly improved in the last decades Nobody’s asked the question lately, can we do it now?

Henry Lieberman • MIT Media Lab

Ambuiguity is your friend

Isn’t English hopelessly ambiguous? Ambiguous, yes; hopeless, no! Conventional programming forces premature commitment to representation Common Sense resolves ambiguity when necessary Interaction resolves ambiguity when necessary

Henry Lieberman • MIT Media Lab

New resources for Common Sense Reasoning

Open Mind Common Sense knowledge base ConceptNet Semantic Net MontyLingua/LangUtils (Eslick, Wed. 2pm)

Henry Lieberman • MIT Media Lab

CMU Natural Programming Study

John Pane and Brad Myers studied 5th graders’ description of Pac-Man Also college students’ descriptions of spreadsheet programs Findings, e.g. Nobody talks about loops Designed HANDS programming language

Henry Lieberman • MIT Media Lab

Childrens’ description of Pac- Man

Henry Lieberman • MIT Media Lab

But what can you do if you can

• nly partially understand English?

Need to make the interaction fail-soft Rely on dialogue to resolve ambiguities and errors Maybe don’t eliminate code entirely Emphasize correspondence between language and code

Henry Lieberman • MIT Media Lab

Metafor

An “outliner” for code as a brainstorming tool Helps you discover and manage the overall structure but doesn’t fill in all the details

Henry Lieberman • MIT Media Lab

Programmatic Semantics

English to “scaffolding” code Roughly, Turns nouns into classes and/or instances; Verbs into functions Adjectives into instance variables Resolves anaphora Untangles conditional and loops Propagates context

Henry Lieberman • MIT Media Lab

Metafor Interface

Henry Lieberman • MIT Media Lab

Target domain: MOOs

(Programmable text adventure games)

Game itself is an interactive narrative MOOs allow programmable objects & characters

Miranda gives you a hug Mouse says, "I'm here to hug you!" Mouse hugs Miranda Mouse says, “I made a mistake”

Stacy i s a f rendl y k i l l e r whale .

She has Brown eyes . Her t a i l has a r ash . 1 scr ip t

• n

S tacy:

• n

f l ap t h is number " t imes" se t f l apped to num ber i f f l apped > 5 t imes emote " b l inks her eyes" end i f end

Henry Lieberman • MIT Media Lab

Narrative stances

I want to make a bar with a customer. (1st person programmer) There is a customer in the bar. (3rd person objective) I am a customer sitting on a stool. (1st person program) The bartender said, “Here is a customer” (3rd person subjective)

Henry Lieberman • MIT Media Lab

Bubble Semantics

car tires rating fast ((a.b).c)=d (a.b).c a.b car paint drying time fast ((a.b).c)=d (a.b).c a.b car top speed fast a.b (a.b )=c car drive

• speed

fast a.b(c=d) a.b(c) a.b() car wash

• speed

fast b.a(c=d) b.a(c) b.a() car drive

• road

fast a.b(c) a.b() speed limit c.d & a.b(c) c.d =e & a.b(c) car drive

• road

fast a.b(c) a.b() road m aterial c.d & a.b(c) p ave- m ent c.d=e & a.b(c) drying tim e e.f & c.d = e a.b(c) e.f=g & c.d = e a.b(c) (a) The car wh

• se top speed is fast.

(b ) Th e car that can be driven at a sp eed that is fast. (c) The car whose tires have a rating that is fast. (d) The car whose paint has a d rying time that is fast. (e) The car that can be wash ed at a speed that is fast. (f) The car that can be driven on a road whose speed limit is fast. (g) The car th at can be driven

• n a road

whose road material is pavement, whose drying tim is fast. e

Henry Lieberman • MIT Media Lab

Refactoring

There is a bar. (atom) The bar contains two customers. (unimorphic list) It also contains a waiter. (unimorphic wrt. persons) It also contains some stools. (polymorphic list) The bar opens and closes. (class / agent) The bar is a kind of store. (inheritance class) Some bars close at 6pm. (subclass or instantiatable)

Henry Lieberman • MIT Media Lab

Dialog for disambiguating

User: We're going to program a game, Pac-Man. System: OK. I know a few things about how to make games. We need to decide what the playing field looks like, design some characters for the game, how the users are going to control the characters, and how the scoring system is going to work. OK? U: Sure. The yellow dots are set through a maze, and Pac-Man, the chomping yellow circle, has to collect them. S: You want the playing field to be a grid of squares, each containing a yellow dot, right? (System displays a grid of squares with yellow dots). To make it a maze, we need to have some

• walls. Can you draw some walls for me?

Henry Lieberman • MIT Media Lab

Dialog for disambiguating

S: OK. What does it mean for Pac-Man to "collect" a dot? U: Whenever Pac-Man is on a square with a dot, he eats it. That means that the dot is gone and the player gets another point. S: OK, now I don't understand what "eats" means. Can you explain it? U: When you draw Pac-Man, you alternate between a picture of Pac-Man with his mouth open and a picture of Pac-Man with his mouth closed

Henry Lieberman • MIT Media Lab

English as a programming language?

The “Holy Grail” of easy to use interfaces for programming Natural language technology has improved to the point that some understanding of programming concepts can be done Good interaction design can make natural language interfaces “fail-soft” Maybe Cobol wasn’t such a bad idea after all!