Approximate Computing and Storage from Programming Language to Hardware and Molecules Luis Ceze Paul G. Allen School of Computer Science & Engineering University of Washington
IEEE JOURN.4L OF SOLID-ST.iTE CIRCUITS, 5> OCTOBER 1974 2456 VOL. SC-9, NO. Kapazitat auf Source und Drain im Ersatzschaltbild eines [31 E. J. Boleky, ‘%ubnanosecond switching delays using CMOS/ MOS-Transistors,” Siemenx und SOS silicon-gate technology,” in 1971 Int. Solid-State Cir- Forxchungs- Ent wicldwrgs- p. 225, berichte 1, no. 3$pp. X4-286, 1972. cuit Conj., Dig. Tech. Papers, [121 J. R. Burns, “Switching response of complementary+sym- [41 E. J. Boleliy and J. E. Meyer, “High-performance low-power metry MOS transistors logic circuits,” vol. 25, CMOS memories using silicon-on-sapphire technology,” RCA Rev., pp. 627481, 1964. IEEE J. Solid-State Circuits (Special Issue on Micropower vol. SC-7, pp. 135-145, Apr. 1972. Electronics), [131 R. w. Ahrons and P. D. Gardner, ‘[Introduction of tech- nology and performance in complementary symmetry cir- R. W. Bower, H. G. Dill, K. G. Aubuchon, and S. A. Thomp- [5’1 son, ‘[MOS field effect transistors by gate masked ion im- cuits,” (Special Issue on IEEE J. Solid-State Circuits Tech- pp. 24–29, plantation,” vol. ED-15, pp. nology jor Integrated-Circuit Design), vol. SC-5, IEEE !t’’rams. Electron Devices, 757-761, Oct. 1968. Feb. 1970. But it is Dennard scaling that [61 J. Tihanyi, “Complementary ESFI MOS devices with gate [141 F. F. Fang and H. Rupprecht, “High performance MOS in- self adjustment by ion implantation,” in Proc. tegrated circuits using ion implantation technique,” pre- 5,th Iwt. Conj. 27–29, 1972. Munchen- sented at the 1973 ESSDERC, Munich, Germany, Microelectronics in Munich, Nov. Moore’s law gives us lots Wien, Germany: R. Oldenbourg Verlag, pp. 437447. E. J. Boleky, “The performance of complementary MOS [71 lets us use them: transistors on insulating substrates,” RCA Rev., vol. 80, pp. 372-395, 1970. K. [81 Goser, ‘[Channel formation in an insulated gate field effect transistor (IGFET) and its emrivalent circuit .“ Sienzen.s Michael Pomper, for a photograph and biography, please see p. of transistors on a chip. no. 1, pp.’ 3-9, Forschungs- und Entwiclclungsbekhte, 1971. 238 of this issue. 2x transistor count, 40% faster, A. E. [91 Ruehli and P, A. Brennan, “Accurate metallization capacitances for integrated circuits and packages,” IEEE J. Solid-State Circwits (Corresp.), vol. SC-8, pp. 289-290, Aug. 1973. 50% more efficient… [101 SINAP (Siemens Netzwerk Analyse Programm Paket), Jeno Tlhanyi, for a photograph and biogra~hy, please see p. Siemens AG, Munich, Germany. 238 of this issue. [111 K, Goser and K. Steinhubl, ‘[Aufteilung der Gate-Kanal- Design of Ion-Implanted MOSFET’S with Very Small Physical Dimensions ROBERT H. DENNARD, LIEMBER, IEEE, FRITZ H. GAENSSLEN, HWA-NIEN YU, MEMBER, IEEE, V. LEO RIDEOUT, MEMBER) IEEE, ERNEST BASSOUS, AND ANDRE R. LEBLANC, MEMBER, IEEE paper considers the design, fabrication, and Absfracf—This LIST OF SYMBOLS characterization of very small MOSI?ET switching devices suitable Inverse semilogarithmic slope of sub- a for digital integrated circuits using dimensions of the order of 1 p. threshold characteristic. Scaling relationships are presented which show how a conventional MOSFET can be reduced in size. An improved small device struc- D Width of idealized step function pro- ture is presented that uses ion implantation to provide shallow fde for chaDnel implant. source and drain regions and a nonuniform substrate doping pro- AW, Work function difference between gate file. One-dimensional models are used to predict the substrate and substrate. doping profile and the corresponding threshold voltage versus Dielectric constants for silicon and source voltage characteristic. A two-dimensional current transport model is used to predict the relative degree of short-channel effects silicon dioxide. for different device parameter combinations. Polysilicon-gate Drain current. MOSFET’S with channel lengths as short as 0.5 ~ were fabricated, Boltzmann’s constant. and the device characteristics measured and compared with pre- Unitless scaling constant. dicted values. The performance improvement expected from using MOSFET channel length. these very small devices in highly miniaturized integrated circuits is projected. Effective surface mobility. Intrinsic carrier concentration. Substrate acceptor concentration. Band bending in silicon at the onset of Manuscript received May 20, 1974; revised July 3, 1974. strong inversion for zero substrate The aubhors are with the IBM T. J. Watson Research Center, Yorktown Heights, N.Y. 10598. voltage. [Dennard, Gaensslen, Yu, Rideout, Bassous, Leblanc, IEEE JSSC , 1974]
With Dennard scaling dead (~2005-8), power per transistor stays constant… ... power Can’t have all transistors on! ... # of transistors “Dark silicon” (boo!) power density generations Explicit parallelism is important, but not a longer term solution…
And… The Economist says… “The number of people predicting the end of Moore’s Law doubles every year.” – Doug Carmean (Intel->MSFT)
So how do we make computer systems better? Unavoidable that we need to exploit application properties and specialize. Source: Bob Broderson, Berkeley Wireless group
One trillion photos were taken in 2015. Compared to 80 billion in 2000, 12.5x over 15 years YouTube : 100 hours of video uploaded per These applications consume most of our minute. 4 billion video cycles/bytes/bandwidth views a day. Input data is inexact by nature (sensors) They have multiple acceptable outputs
What is “approximate computing”? ~ Exploit inherent application-level resilience/ Application redundancies to build more efficient/better Language computers. Compiler ISA/Architecutre Efficiency and performance Output accuracy Circuits Physics In essence, goal is to specialize computation, storage and communication to properties of the data and the algorithm. Squeezes excess precision and enables better use of underlying substrate.
Wait, what about… :) Machine learning Iterative algorithms Applications/ Lossy compression Algorithms Floating point… Reasoning about approximation in PL Language ~4X Approximate compiler optimizations Compiler ~10X Approximate execution models ISA/Architecture ~5X Non-deterministic “unsafe” circuits Circuits (voltage scaling, mem) Physics ~10-100X Analog hardware (closer to physics)
Errors Energy Errors Energy Errors Energy Errors Energy
Ground-Truth Quality from Crowdsourced Opinions Without Trade-O ff :( With Trade-O ff :) 100% 80% Acceptability 60% 40% 20% 0% 0 100 200 300 400 500 Energy (abstract units)
But approximation needs to be done carefully... or...
HW/SW co-design is essential… Approximation just at the hardware level isn’t safe. Approximation just at the algorithm level is suboptimal. Assuming reliable hardware for inherently robust algorithms is a waste.
Three key questions 1 What and how to approximate? Language 2 Compiler How good is my output? Runtime 3 Hardware How to take advantage of it?
“Disciplined” approximate programming ✓ references pixel data neuron weights jump targets audio samples ✗ JPEG header video frames Precise Approximate • Programmer has direct control of approximate/precise and the flow • System is free to approximate as long as rules are obeyed
EnerJ/C++ [PLDI’11, OOPSLA’15] @Approx< 0.9 > int a = ...; ✓ Separate critical and non-critical @Precise int p = ...; program components. Analyzable ✗ • Operator overloading for approximate operations: statically. Precise Approximate p + p; p + a; a + a; • Endorsement of approximate values: @Approx int a = ...; ✓ p = endorse (a); • Dealing with implicit flows in control: @Precise int p = ...; ✓ ✗ ✗ if ( if ( a > 10) ) ) { endorse ( a > 10 p = a; p = 2; p = 2; ✓ } } a = p;
How good is my output? • Quality-of-Result (QoR) • Application dependent – e.g, % of bad pixels, deviation from expected value, % of poorly classified images, car crashes, etc…
Specifying and checking quality res = computeSomething() ; assert diff(res, res ʹ ) < 0.1; precise version of the result passert expr, prob, conf
Verifying quality expressions [PLDI’14] exact int p = 5; @Approx int a = 7; for (int x = 0..) { evaluation a += func(2); @Approx int z; z = p * 2; p += 4; } a /= 9; p += 10; socket.send(z); write(file, z); sampling + Bayesian optimized input and error network IR Bayes net distribution ?
Online quality monitoring [ASPLOS’15] Can react – recompute or reduce approximation But needs to be cheap! Fuzzy Sampled Simple Memoization precise verification functions re-execution Time - < ε ? ✓ - < ε ? - < ε ?
Recommend
More recommend
