Traversing Digital Design EECS 150 - Components and Design Techniques for Digital Systems You Are Here EECS150 wks 6-15 Lec 13 Project Overview David Culler Electrical Engineering and Computer Sciences University of California, Berkeley EECS150 wks 1-6 http://www.eecs.berkeley.edu/~culler http://www-inst.eecs.berkeley.edu/~cs150 EE 40 CS61C 10/12/2004 EECS 150, Fa04, Lec 13-Project 2 Caveats Basic Pong Today s lecture provides an overview of the project. Lab will cover it in MUCH more detail. Where there are differences, the lab information is correct! Names of many components are different from what is used in lab, so you won t be confused Court = set of obstacles fixed position Paddle = moving obstacle Position & vertical velocity Function of joystick P = f ( P, j ) Ball 2D position & velocity [ spin, acc ] Bounces off walls and paddles B = f ( B, j,c ) Score Ball hitting sides Effects Display, audio, 10/12/2004 EECS 150, Fa04, Lec 13-Project 3 10/12/2004 EECS 150, Fa04, Lec 13-Project 4 Calinx Board Add-on card Video & Audio Ports AC 97 Codec & Power Amp Video Encoder & Decoder Flash Card & Micro-drive Port Four 100 Mb Ethernet Ports Meg x Quad Ethernet Transceiver Prototype Area Seven Segment LED Displays Xilinx Virtex 2000E 10/12/2004 EECS 150, Fa04, Lec 13-Project 5 10/12/2004 EECS 150, Fa04, Lec 13-Project 6 NOW Handout Page 1 1
Project Design Problem Input-Output Constraints Map this application To this technology Ball moves within a court Players control movement of the paddles with joysticks Observe game as rendered on video display Bounces off walls and paddles till point is scored I/O devices provide design constraints 10/12/2004 EECS 150, Fa04, Lec 13-Project 7 switches LEDS LCD 10/12/2004 EECS 150, Fa04, Lec 13-Project Input/Output Support Digitize and abstract messy analog Rendering pipeline to translate display objects into byte stream Off-chip device to translate digital byte stream into composite video 10/12/2004 EECS 150, Fa04, Lec 13-Project 9 of the Court = set of obstacles fixed position Paddle = moving obstacle Position & vertical velocity Function of joystick P = f ( P, j ) Ball 2D position & velocity [ spin, acc ] Bounces off walls and paddles B = f ( B, j,c ) Score Ball hitting sides Effects Display, audio, 10/12/2004 EECS 150, Fa04, Lec 13-Project 10 Representing State of the game Court obstacles Paddles Ball Score Additional data Display blocks» Paddle & ball image Numerals Frame buffer frame holds frame buffer Rendered to frame buffer Spooled to video encoder has sophisticated Grok sheet, design bus controller block RAM holds also Registers, Counters, Timing sequence, ler FIFOs, Packet buffers 10/12/2004 EECS 150, Fa04, Lec 13-Project 11 N64 (cp 1) Video stream DQ Continually poll N64 and report of buttons and analog joystick Issue -bit command Receive -bit response Each button response is bit value containing button and -bit signed horizontal and vertical velocities Serial protocol Multiple cycles to perform each transaction Bits obtained serially Framing (packet start/stop) velocity pause start reset clock (27 MHz) Bit encoding» start data data stop 10/12/2004 EECS 150, Fa04, Lec 13-Project 12 NOW Handout Page 2 2
Video Encoder (cp 2) Announcements Rendering engine processes display objects into frame buffer Renders rectangles, image blocks, Drive video encoder device so that it outputs the correct NTSC Gain experience reading data sheets Dictates the 27 MHz operation rate Used throughout graphics subsystem Midterm will be returned in section Solutions available on-line Reading: Video In a Nutshell (by Tom Oberheim) on class web page Lab project documents (as assigned) 10/12/2004 EECS 150, Fa04, Lec 13-Project 13 10/12/2004 EECS 150, Fa04, Lec 13-Project 14 Digital Video Basics a little detour Pixel Array: A digital image is represented by a matrix of values where each value is a function of the information surrounding the corresponding point in the image. A single element in an image matrix is a picture element, or pixel. A pixel includes info for all color components. The array size varies for different applications and costs. Some common sizes shown to the right. Frames: The illusion of motion is created by successively flashing still pictures called frames. 10/12/2004 EECS 150, Fa04, Lec 13-Project 15 Refresh Rates & Scaning The human perceptual system A vertical blanking interval can be fooled into seeing corresponds to the time to return continuous motion by flashing from the bottom to the top. frames at a rate of around 20 In addition to the active (visible) lines frames/sec or higher. of video, each frame includes a number of non-visible lines in the Much lower and the movement looks jerky and flickers. TV in the US uses vertical blanking interval. 30 frames/second (originally derived The vertical blanking interval is used from the 60Hz line current frequency). these days to send additional information such as closed captions Images are generated on the and stock reports. screen of the display device by drawing or scanning each line of the image one after another, usually from top to bottom. Early display devices (CRTs) required time to get from the end of a scan line to the beginning of the next. Therefore each line of video consists of an active video portion and a horizontal blanking interval portion. 10/12/2004 EECS 150, Fa04, Lec 13-Project 16 Interlaced Scanning Early inventers of TV discovered that they could reduce the flicker effect by increasing the flash-rate without increasing the frame-rate. Interlaced scanning forms a complete picture, the frame, from two fields, each comprising half the scan lines. The second field is delayed half the frame time from the first. The first field, odd field, displays the odd scan lines, the second, even field, displays the even scan lines. 10/12/2004 EECS 150, Fa04, Lec 13-Project 17 Pixel Components A natural way to represent the The color signals (components) information at each pixel is with are color differences, defined as: the brightness of each of the Y-B and Y-R, where Y is the brightness primary color components: red, signal (component). green and blue (RBG). In the digital domain the three In the digital domain we could components are called: transmit one number for each of red, green, and blue intensity. Y luma, overall brightness C Engineers had to deal with issue B chroma, Y-B C when transitioning from black R chroma, Y-R and white TV to color. The signal Note that it is possible to for black and white TV contains reconstruct the RGB the overall pixel brightness (a representation if needed. combination of all color One reason this representation components). survives today is that the human Rather than adding three new signals visual perceptual system is less for color TV, they decided to encode sensitive to spatial information in the color information in two extra chrominance than it is in signals to be used in conjunction with the B/W signal for color luminance. Therefore chroma receivers and could be ignored for components are usually the older B/W sets. subsampled with respect to luma 10/12/2004 EECS 150, Fa04, Lec 13-Project component. 1 NOW Handout Page 3 3
Subsampling RGB 4:4:4 Y 4:4:4 4:2:2 (ITU-601) 4:2:0 (MPEG-1) 4:2:0 (MPEG-2) R 0 R 2 R 1 R 3 G 0 G 2 G 1 G 3 B 0 B 2 B 1 B 3 0-1 2-3 0-1 0-1 0-3 0-3 0-3 0-3 Variations include subsampling horizontally, both vertically and horizontally. samples are coincident with alternate luma samples or are sited halfway between alternate luna samples. 10/12/2004 EECS 150, Fa04, Lec 13-Project 19 Common Interchange Format (CIF) Example 1: commonly used as output of MPEG-1 decoders. Common Interchange Format (CIF) Developed for low to medium quality applications. Teleconferencing, etc. Variations: QCIF, 4CIF, 16CIF Examples of component streaming: line i: Y Y Y Y Y line i+1: Y Y Y Y Y Alternate (different packet types): line i: Y Y Y Y Y line i+1: Y Y Y Y Y Bits/pixel: 6 components / 4 pixels 4/4 = 12 bits/pixel Frame size Frame rate Scan subsampling alignment Bits per component Effective bits/pixel 352 x 2 30 /sec progressive 4:2:0 2:1 in both X & Y interstitial 10/12/2004 EECS 150, Fa04, Lec 13-Project 20 12 ITU-R BT.601 Format The Calinx video encoder supports this format. Formerly, CCIR-601. Designed for digitizing broadcast NTSC (national television system committee) signals. Variations: 4:2:0 PAL (European) version Component streaming: line i: Y Y Y Y Y line i+1: Y Y Y Y Y Bits/pixel: 4 components / 2 pixels 40/2 = 20 bits/pixel Frame size Frame rate Scan subsampling alignment Bits per component Effective bits/pixel 720 x 47 29.97 /sec interlaced 4:2:2 2:1 in X only coincident 10/12/2004 EECS 150, Fa04, Lec 13-Project 21 10 20 Calinx Video Encoder Analog Devices Supports: Digital side connected to Virtex Multiple formats and outputs pins. Operational modes, slave/master Analog output side wired to on VidFX project will use default mode: connectors or headers. ITU-601 as slave I 2 C for initialization: s-video output Wired to Virtex. 10/12/2004 EECS 150, Fa04, Lec 13-Project 22 ITU-R BT.656 Details Interfacing details for ITU-601. Pixels per line 5 Lines per frame 525 Frames/sec 29.97 Pixels/sec 13.5 M Viewable pixels/line 720 Viewable lines/frame 47 With 4:2:2 chroma sub-sampling need to send 2 words/pixel (1 Y and 1 C). words/sec = 27M, Therefore encoder runs off a 27MHz clock. information (horizontal and vertical synch) is multiplexed on the data lines. Encoder data stream show to right: Luminance data, Y Chrominance data, Chrominance data, Last sample of digital active line 359 Y 71 359 Y 719 360 Y 720 360 359 Y 71 359 Y 719 736 57 71 719 720 721 ( 7) (63) 0 1 2 359 360 36 ( 0 1 366) 36 359 360 ( 366) 0 1 10/12/2004 EECS 150, Fa04, Lec 13-Project 23 Y 721 Replaced by timing reference signal End of active video FIGURE 1 Composition of data stream Sample data for O H instant 36(366) Y 736(7) 36(366) Replaced by digital blanking data Timing reference signals Y 55(61) 42(431) First sample of digital active line Y 56(62) 42(431) Y 57(63) 0 0 Replaced by timing reference signal Start of active video 0 0 Note 1 Sample identification numbers in parentheses are for 625-line systems where these differ from those for 525-line systems. (See also Recommendation ITU-R BT.03.) D01 ITU-R BT.656 Details is provided through End of Video (EAV) and Start of Video (SAV) timing references. Each reference is a block of four words: FF, 00, 00, <code> The <code> word encodes the following bits: F = field select (even or odd) V = indicates vertical blanking H = 1 if EAV else 0 for SAV Horizontal blanking section consists of repeating pattern 0 10 0 10 10/12/2004 EECS 150, Fa04, Lec 13-Project 24 NOW Handout Page 4 4
Calinx Video Decoder (not this term) Analog Devices ADV715 analog side Takes NTSC (or PAL) video signal on analog side and outputs ITU601/ITU656 on digital side. Many modes and features not use by us. VidFX project will use default mode: digital side Generates 27MHz clock synchronized to the output data. Digital side connected to Virtex pins. Analog output side wired to on connectors or headers. Camera connection through composite video. no initialization needed. 10/12/2004 EECS 150, Fa04, Lec 13-Project 25 (cp 3) frame Memory protocols Bus arbitration Address phase phase DRAM is large, but few address lines and slow Row & col address Wait s Synchronous DRAM provides fast synchronous access current block Little like a cache in the DRAM Fast burst of data Arbitration for shared resource 10/12/2004 EECS 150, Fa04, Lec 13-Project 26 READ burst timing (for later) Rendering Engine Video Enc. i/f Fed series of display objects Obstacles, paddles, ball Each defined by bounding box» Top, bottom, left, right Renders object into frame buffer within that box Bitblt color for rectangles Copy pixel image Must arbitrate for and carry out bus protocol 10/12/2004 EECS 150, Fa04, Lec 13-Project 27 10/12/2004 EECS 150, Fa04, Lec 13-Project 2 Video Encode Divide time into two major phases Render Compute new Compute phase is divided into 255 ticks Each tick is small enough that paddles and can only move a small amount Makes fixed point arithmetic each New paddle pos/vel based on old pos/vel and joystick velocity New ball is based on old ball pos/vel and all collisions Stream all obstacles and paddles by the ball next logic to determine bounce More when we look at arithmetic Network Multiplayer network 10/12/2004 EECS 150, Fa04, Lec 13-Project 29 10/12/2004 EECS 150, Fa04, Lec 13-Project 30 NOW Handout Page 5 5
Rendezvous & mode of operation Player with host device publishes channel and ID Write it on the white Set dip switches to select channel Start game as host Wait for guest attach Start game as guest Send out attach request Host compute all the game physics Local joystick and remote network joystick as s» Receive movement packets and xlate to equivalent of local Determines new ball and paddle position» Transmits court update packets Network remote device must have fair service Both devices render display locally 10/12/2004 EECS 150, Fa04, Lec 13-Project 31 Host Device (player 0) Board encoder decoder Network controller 10/12/2004 EECS 150, Fa04, Lec 13-Project SPI CC2420 Guest Device (player 1) Board decoder Encoder Network controller 10/12/2004 EECS 150, Fa04, Lec 13-Project 33 SPI CC2420 Protocol Stacks Usual case is that MAC protocol encapsulates IP (internet protocol) which in turn encapsulates TCP (transport control protocol) with in turn encapsulates the application layer. Each layer adds its own headers. Other protocols exist for other network services (ex: printers). When the reliability features (retransmission) of TCP are not needed, UDP/IP is used. Gaming and other applications where reliability is provided at the application layer. MAC Layer 2 IP Layer 3 TCP Layer 4 Layer 5 application layer ex: http MAC Layer 2 IP Layer 3 UDP Layer 4 Layer 5 Streaming Ex. Mpeg4 10/12/2004 EECS 150, Fa04, Lec 13-Project 34 Standard Hardware-Network- application level MAC (MAC layer processing) Media Independent (MII) Usually divided into three hardware blocks. (Application level processing could be either hardware or software.) MAG. Magnetics chip is a transformer for providing electrical isolation. PHY. Provides serial/parallel and parallel/serial conversion and encodes bit-stream for Ethernet signaling convention. Drives/receives analog signals to/from MAG. Recovers clock signal from data. PHY (Ethernet signal) MAG (transformer) Ethernet connection MAC. Media access layer processing. Processes Ethernet frames: preambles, headers, computes CRC to detect errors on receiving and to complete packet for transmission. Buffers (stores) data for/from application level. Application level Could be a standard bus (ex: PCI) or designed specifically for application level hardware. MII is an industry standard for connection PHY to MAC. Calinx has no MAC chip, must be handled in. You have met ethernet. IEEE 02.15.4 will look similar yet different 10/12/2004 EECS 150, Fa04, Lec 13-Project 35 02.15.4 Frames 10/12/2004 EECS 150, Fa04, Lec 13-Project 36 NOW Handout Page 6 6
Packet protocols Framing definitions IEEE 02.15.4 Packet formats Request game packet Board Packet Schedule of checkpoints CP1: N64 (this week) CP2: Digital video encoder (week ) CP3: controller (two parts, week 9-10) CP4: IEEE 02.15.4 (cc2420) (wk 11-12) unless we bail out to ethernet Overlaps with midterm II Project CP: game engine (wk 13-14) Endgame 11/29 early checkoff 12/6 final checkoff 12/10 project report due 12/15 midterm III 10/12/2004 EECS 150, Fa04, Lec 13-Project 37 10/12/2004 EECS 150, Fa04, Lec 13-Project 3 NOW Handout Page 7 7