Codec Chips Tribute to Prof. Goto Jinjia Zhou 1 , Dajiang Zhou 2 , - - PowerPoint PPT Presentation

▶
codec chips
SMART_READER_LITE
LIVE PREVIEW

Codec Chips Tribute to Prof. Goto Jinjia Zhou 1 , Dajiang Zhou 2 , - - PowerPoint PPT Presentation

Apr 30, 2023 187 likes •465 views

100x Evolution of Vid ideo Codec Chips Tribute to Prof. Goto Jinjia Zhou 1 , Dajiang Zhou 2 , Satoshi Goto 2 1 Hosei University, Tokyo, Japan 2 Waseda University, Kitakyushu, Japan Prof. Goto has been my supervisor from 2008 to 2015. (M.S ->

slide-1
SLIDE 1

100x Evolution of Vid ideo Codec Chips

Jinjia Zhou1, Dajiang Zhou2, Satoshi Goto2

1Hosei University, Tokyo, Japan 2Waseda University, Kitakyushu, Japan

Tribute to Prof. Goto

slide-2
SLIDE 2
  • Prof. Goto has been my supervisor

from 2008 to 2015. (M.S -> Ph.D -> PDF)

2

slide-3
SLIDE 3
  • Prof. S.

. Goto’s Vid ideo Coding Research Group

►One of the first Full-HD H.264 encoders,

first to use SiS DRAM (VLSI’07 and JSSC’09)

►First 4kx2k@60fps H.264 decoder (VLSI’10) ►First 8kx4k H.264 decoder (ISSCC’12) ►First 8Kx4K H.264 (intra-frame) encoder

(VLSI’12)

►First 8Kx4K H.264 ME encoder (VLSI’13 and

JSSC’14)

►First 8Kx4K HEVC decoder (ISSCC’16 and

JSSC’16)

3

slide-4
SLIDE 4

4

Mass media

slide-5
SLIDE 5

Video codec: encoder and decoder

Source video data Encoder Receiver Compressed video stream Decoder Compression Decompression Video camera Display device Restored video data stream Transmit. Channel/ Storage 100% 100% ~1% ~1% ~1%

5

slide-6
SLIDE 6

Applications of f video codec chips

Enc./Dec. Chip TV conference Surveillance Automotive Mobile/Portable Small frame delay Ultra-low power High compression High video quality Free-point view …. ……

Source of the images: http://www.artesanosdecastillalamancha.org/wp-content/uploads/2015/06/28.png http://www.caradvice.com.au/67890/2011-brakes-camera-action-pedestrian-detection-automated-platooning/photos/

Home entertainment

6

slide-7
SLIDE 7

8K UHDTV and fr free-viewpoint TV

25~30fps

≥120fps

7680 pixels

7

slide-8
SLIDE 8

Video coding standards

1990 1995 2000 2005 2010

MPEG-1 MPEG-4 H.261 H.263 (+/++) H.262 MPEG-2 H.264 MPEG-4 AVC

H.265 HEVC ITU-T standards MPEG standards Joint ITU-T & MPEG standards Compression ratio

~50:1 ~100:1

~200:1 2013

8

slide-9
SLIDE 9

Hig igh compression at hig igh complexity

►Recent powerful codecs address the huge video

throughput in the communication channel

►Their high compression ratio, however, is at the expense of

high complexity

9

H.264 (2003) ~480Mbps ~240Mbps RAW HEVC (2013) 48000Mbps

slide-10
SLIDE 10

Complexity of f vid ideo codecs (n (norm.)

1 20 307.2 0.1 1 10 100 1000 1080p/MPEG-2 4K/H.264 8K/HEVC Complexity/pixel Throughput Overall complexity

10

slide-11
SLIDE 11

Real-time 8K UHDTV codec systems

NHK 8K codec (2007) NHK 8K encoder (2013) Our target: Single chip/chipset

11

slide-12
SLIDE 12

Memory ry bandwidth issue

►Performance bottleneck

  • >50GBps BW required for decoding 8K UHDTV
  • >100GBps BW required for encoding 8K UHDTV

►Power consumption

  • Majority of power

consumed by DRAM traffic

►Fabrication cost

  • BW determines chip pin count

12

Codec DRAM Memory

traffic

slide-13
SLIDE 13

Data dependency issue

►Video codecs exploit all kinds of data

dependencies to strengthen compression

  • Inter-frame prediction
  • Intra-frame prediction
  • Context-adaptive entropy coding (CABAC)

►Data dependencies restricts the degree of

efficient parallelism/pipelining

  • Power and area issues
  • Performance issue

13

slide-14
SLIDE 14

Challenges summarized

14

Transform & Quant.

  • Inv. Trans. &
  • Inv. Quant.

Deblocking Filter Reference Frames Frame Output Motion Compensation Motion Estimation Intra Prediction

  • Entropy

Coding

Source Frm. Decoder

Memory bandwidth requirements Computational complexity Data dependencies

slide-15
SLIDE 15

Our efforts to address the challenges

15

System Algorithm Architecture Circuits Device Evaluation

Bus/interface optimization, 3DLSI Processing order optimization Embedded compression 2-D cache Reduce complexity Trade-off b/w time & quality Hardware friendliness, … Alleviate data dependencies Processing order optimization Predictive execution, …

Reduce memory access / Increase memory bandwidth

slide-16
SLIDE 16

System integration

►FIFO vs RAM

  • FIFO: simple interface and

flexibility

  • RAM: random accessibility for

data reordering

►Proposed BIBO (Block-in-block-

  • ut) queues:

combines benefits of the two

<16>

slide-17
SLIDE 17

►Words in a block can be written in a random order ►A block can be pushed after all words are written ►Blocks follows first-in-first-out ►Blocks can be in a variable size

Word write & block push

BIBO queue

Word Block

<17>

slide-18
SLIDE 18

Word read & block pull

►Words in a block can be read in a random order ►A block can be pulled after all words are read ►Blocks can be pulled in different sizes as pushed

BIBO queue

Merged

<18>

slide-19
SLIDE 19

Merge and split

►Block merging/splitting can be

automated by BIBO given both word addressing and block scan follow a Z-scan order

BIBO queue

<19>

slide-20
SLIDE 20

Merge and split

BIBO queue

Push

<20>

slide-21
SLIDE 21

Merge and split

BIBO queue

Pull

<21>

slide-22
SLIDE 22

Merge and split

BIBO queue

<22>

slide-23
SLIDE 23

Im Implemented video codec chips fr from Goto’s La Lab.

Source: http://www.f.waseda.jp/goto/html/chip.html 23

slide-24
SLIDE 24

Vid ideo decoder demo: 4K@FPGA, 8K@chip

24

slide-25
SLIDE 25

Performance of f codec VLSI I chip ips

25

27.6 1990 249 3981 1000 2000 3000 4000 MIT ASSCC'08 Ours ISSCC'12 Ours VLSIC'12 NTU VLSIC'13 MIT ISSCC'13 NTT VLSIC'15 Ours ISSCC'16

Mpixel/s

H.264 decoder HEVC decoder H.264 encoder HEVC encoder

144x

slide-26
SLIDE 26

Thanks to all ll members who have contrib ibuted in in the vid ideo codec chip ip desig ign

26

slide-27
SLIDE 27

Thank you!

ji jinjia.z .zhou.35@hosei.a .ac.jp

27