4.5 Pipelining. Pipelining is Natural!

Transcription

1 4.5 Pipelining Ovelapped execution of instuctions Instuction level paallelism (concuency) Example pipeline: assembly line ( T Fod) Response time fo any instuction is the same Instuction thoughput inceases Speedup = k x numbe of steps (stages) Theoy: k is a lage constant Reality: Pipelining intoduces ovehead 85 Pipelining is Natual! Laundy Example Ann, Bian, Cathy, Dave each have one load of clothes to wash, dy, and fold Washe takes 30 minutes A B C D Dye takes 30 minutes Folde takes 30 minutes Stoe takes 30 minutes to put clothes into dawes 86

2 Sequential Laundy 6 PM AM T a s k O d e A B C D Time Sequential laundy takes 8 hous fo 4 loads If they leaned pipelining, how long would laundy take? 87 Pipelined Laundy: Stat wok ASAP 12 2 AM 6 PM T a s k O d e A B C D Time Pipelined laundy takes 3.5 hous fo 4 loads! 88

3 Pipelining Lessons T a s k O d e 6 PM Time A B C D Pipelining doesn t help latency of single task, it helps thoughput of entie wokload Multiple tasks opeating simultaneously using diffeent esouces Potential speedup = Numbe pipe stages Pipeline ate limited by slowest pipeline stage Unbalanced lengths of pipe stages educes speedup Time to fill pipeline and time to dain it educes speedup Stall fo Dependences 89 Five-stages of multi-cycle DP 90

4 The Five Stages of the Load Instuction Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Load Ifetch Reg/Dec Exec Mem W Ifetch: Instuction Fetch Fetch the instuction fom the Instuction Memoy Reg/Dec: Registes Fetch and Instuction Decode Exec: Calculate the memoy addess Mem: Read the data fom the Data Memoy W: Wite the data back to the egiste file 91 Pipelined Execution Time IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB Pogam Flow IFetch Dcd Exec Mem WB On a pocesso multiple instuctions ae in vaious stages at the same time. Assume each instuction takes five cycles 92

5 Single Cycle, Multiple Cycle, vs. Pipeline Single Cycle Implementation: Clk Cycle 1 Cycle 2 Load Stoe Waste Multiple Cycle Implementation: Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Clk Load Ifetch Reg Exec Mem W Stoe Ifetch Reg Exec Mem R-type Ifetch Pipeline Implementation: Load Ifetch Reg Exec Mem W Stoe Ifetch Reg Exec Mem W R-type Ifetch Reg Exec Mem W 93 Why Pipeline? Suppose 100 instuctions ae executed (IC) The single cycle machine has a cycle time of 45 ns (CC) The multicycle and pipeline machines have cycle times of 10 ns The multicycle machine has a CPI of 4.2 Single Cycle Machine 100 inst x 1 CPI x 45 ns/cycle = 4500 ns Multicycle Machine 100 inst x 4.2 CPI x 10 ns/cycle = 4200 ns Ideal pipelined machine (100 inst x 1 CPI + 4 cycle ovehead) x 10 ns/cycle = 1040 ns Ideal pipelined vs. single cycle speedup 4500 ns / 1040 ns = 4.33 What has not yet been consideed? 94

6 Gaphically Repesenting Pipelines Time (clock cycles) I n s t. Inst 0 Inst 1 Can help with answeing questions like: How many cycles does it take to execute this code? What is the doing duing cycle 4? Ae two instuctions tying to use the same esouce at the same time? 95 Why Pipeline? Because the esouces ae thee! Time (clock cycles) I n s t. O d e Inst 0 Inst 1 Inst 2 Inst 3 Inst 4 96

7 Can pipelining get us into touble? Yes: Pipeline Hazads stuctual hazads: attempt to use the same esouce two diffeent ways at the same time Two instuctions use the memoy at the same time data hazads: attempt to use item befoe it is eady instuction depends on esult of pio instuction still in the pipeline contol hazads: attempt to make a decision befoe condition is evaluated (banch instuctions) Can always esolve hazads by waiting pipeline contol must detect the hazad take action (o delay action) to esolve hazads 97 Stuctual Hazad 1: Single Memoy Time (clock cycles) I n s t. O d e Load Inst 1 Inst 2 Inst 3 Inst 4 I$ Reg D$ Reg I$ Reg D$ Reg I$ I$ Reg D$ Reg Reg D$ Reg I$ Reg D$ Reg IM = DM => Read same memoy twice in one clock cycle 98

8 Stuctual Hazad 2: Registe File I n s t. O d e Load Inst 1 Inst 2 Inst 3 Inst 4 I$ Reg D$ Reg I$ Reg D$ Reg I$ I$ Reg D$ Reg Reg D$ Reg Time (clock cycles) I$ Reg D$ Reg Ty ead and wite to egistes simultaneously 99 Stuctual Hazads: Solutions Stuctual hazad 1: single memoy Two memoies? infeasible and inefficient => Two Level 1 caches (instuction and data) Stuctual hazad 2: egiste file Registe access takes less that ½ stage time => Use the following convention: Always Wite duing fist half of each cycle Always Read duing second half of each cycle Both, Read and Wite can be pefomed duing the same clock cycle (a small delay between) 100

9 Contol Hazad: Banch Inst. (1/2) Banch decision-making hadwae in stage Two moe instuctions afte the banch will always be fetched, whethe o not the banch is taken Desied functionality of a banch if we do not take the banch, don t waste any time and continue executing nomally if we take the banch, don t execute any instuctions afte the banch, just go to the desied label 101 Contol Hazad: Banch Inst. (2/2) Initial Solution: Stall until decision is made Inset no-op instuctions: those that accomplish nothing, just take time Dawback: banches take 3 clock cycles each (assuming compaato is put in stage) L1: ADD R2, R2, #2 IF ID EX MEM WB SUB R3, R1, #3 IF ID EX MEM WB BEQ R5, R4, L1 IF ID EX MEM WB ADD R2, R2, #2 S S IF ID EX MEM S = stall 102

10 Banch Pediction Longe pipelines can t eadily detemine banch outcome ealy Stall penalty becomes unacceptable Pedict outcome of banch Only stall if pediction is wong In MIPS pipeline Can pedict banches not taken Fetch instuction afte banch, with no delay 103 MIPS with Pedict Not Taken Pedictio n coect Pedictio n incoect 104

11 Moe-Realistic Banch Pediction Static banch pediction Based on typical banch behavio Example: loop and if-statement banches Pedict backwad banches taken Pedict fowad banches not taken Dynamic banch pediction Hadwae measues actual banch behavio e.g., ecod ecent histoy of each banch Assume futue behavio will continue the tend When wong, stall while e-fetching, and update histoy 105 Data Hazad Instuction depends on esult of pio instuction still in the pipeline add 1,2,3 sub 4, 1,3 and 6, 1,7 o 8, 1,9 xo 10, 1,11 Poblem: 1 cannot be ead by othe instuctions befoe it is witten by the add. 106

12 Data Hazad on 1: Dependencies backwads in time ae hazads I n s t. O d e Time (clock cycles) add 1,2,3 sub 4,1,3 and 6,1,7 o 8,1,9 xo 10,1,11 IF ID/RF EX MEM WB Im Reg Dm Reg 107 Data hazad timing add 1,2,3 IF ID EX MEM WB sub 4,1,3 IF S S ID EX MEM WB and 6,1,7 IF ID EX MEM o 8,1,9 IF ID EX S = stall How to delete Stalls? 108

13 Data Hazad Solution: Fowad esult fom one stage to anothe I n s t. O d e Time (clock cycles) IF ID/RF EX MEM WB add 1,2,3 sub 4,1,3 and 6,1,7 o 8,1,9 xo 10,1,11 Im Reg Dm Reg o OK if define ead/wite popely 109 Data hazad timing Without fowading add 1,2,3 IF ID EX MEM WB sub 4,1,3 IF S S ID EX MEM WB and 6,1,7 IF ID EX MEM o 8,1,9 IF ID EX With fowading add 1,2,3 IF ID EX MEM WB sub 4,1,3 IF ID EX MEM WB and 6,1,7 IF ID EX MEM WB o 8,1,9 IF ID EX MEM WB 110

14 Fowading (o Bypassing): What about Loads Dependencies backwads in time ae hazads Time (clock cycles) IF ID/RF EX MEM WB lw 1,0(2) sub 4,1,3 Can t solve with fowading: Must delay/stall instuction dependent on loads 111 Data hazad equiing stalls Without fowading LD R1, 0(R2) IF ID EX MEM WB SUB R4, R1, R5 IF Stall stall ID EX MEM WB AND R6, R1, R7 IF ID EX MEM WB OR R8, R1, R9 IF ID EX MEM WB With fowading LD R1, 0(R2) IF ID EX MEM WB SUB R4, R1, R5 IF ID stall EX MEM WB AND R6, R1, R7 IF stall ID EX MEM WB OR R8, R1, R9 stall IF ID EX MEM WB 112

15 MIPS Pipelined Datapath 4.6 Pipelined Datapath and Contol MEM Right-to-left flow leads to hazads WB 113 Pipeline egistes Need egistes between stages To hold infomation poduced in pevious cycle 114

16 Pipeline Opeation Cycle-by-cycle flow of instuctions though the pipelined datapath Single-clock-cycle pipeline diagam Shows pipeline usage in a single cycle Highlight esouces used c.f. multi-clock-cycle diagam Gaph of opeation ove time We ll look at single-clock-cycle diagams fo load & stoe 115 IF fo Load, Stoe, 116

17 ID fo Load, Stoe, 117 EX fo Load 118

18 MEM fo Load 119 WB fo Load 120

19 EX fo Stoe 121 MEM fo Stoe 122

20 WB fo Stoe 123 Multi-Cycle Pipeline Diagam Fom showing esouce usage 124

21 Multi-Cycle Pipeline Diagam Taditional fom 125 Single-Cycle Pipeline Diagam State of pipeline in a given cycle 126