Joint IEEE 802 and ITU-T Study Group 15 workshop Building Tomorrow s Networks Geneva, Switzerland, 27 January 2018 IEEE 802.3ca Channel Bonding And Skew Remediation Glen Kramer, Broadcom
Multi-channel 100G-EPON 100G OLT serves a mix of 25G, 50G, and 100G ONUs Four DS and four US wavelengths λ0: 25G, 50G, and 100G ONUs, all broadcast traffic λ1: 50G and 100G ONUs λ2, λ3: 100G ONUs Four independent schedulers upstream and downstream 100G OLT λ0 λ1 λ2 λ3 Power Splitter 25G ONU 50G ONU 25G 50G 100G 100G ONU λ0 λ1 λ2 2 λ3
Each is a 100 Gb/s If not all four lanes are active at a given time, the MPRS will pause the (i.e., not accept more bits) to equalize and PHY data rates 100G ONU Layering Diagram Client (LLID 0) OAM (optional) (LLID 0) Client (LLID 1) OAM (optional) MPCP (LLID 1) Client (LLID N) OAM (optional) (LLID N) Multi-Point Reconciliation Sublayer (MPRS) MPRS_CTRL. Request(...) 25 GMII 25 GMII 25 GMII 25 GMII PCS PCS PCS PCS PMA PMA PMA PMA PMD 3
Transmission Envelope A grant may allocate bandwidth to multiple LLIDs Envelope is a continuous transmission by a specific LLID on a specific channel (wavelength) Structure of Envelope: Wraps multiple user frames with a common Envelope Header, that includes LLID, Envelope Position Alignment Marker (EPAM), and envelope payload length. Envelope Length represents the number of units (Envelope Quanta - EQ) granted to a given LLID in a given grant on a given channel. Envelope Length includes the Envelope Header. Beginning of an envelope payload may be a full frame or a tail segment of a frame. End of the envelope payload may be a full frame or a head segment of a frame. LLID A EPAM n Len 250 1 EQ = 8 Bytes 1000 500 492 250 EQ (=2000 bytes) 4
Channel Bonding Channel bonding refers to ability of a single LLID (single ) to transmit on multiple channels at the same time When a transmits on multiple channels, its data stream is demultiplexed into multiple channels A unit of demultiplexing is the Envelope Quantum (EQ) EQ represents 64 bits of data (not a fixed duration of time like TQ) At 25GB/s, an EQ takes 2.56 ns At 10Gb/s, an EQ takes 6.4 ns. Instantaneous data rate (Gb/s) 100 75 50 25 0 UC0 UC1 UC2 UC3 5 1 2 3 5 8 11 14 18 22 26 = Envelope Start Header 7 10 13 16 20 24... 4 6 9 12 15 17 21 25 27 28 19 23 Partially overlapping envelopes for LLID B 29 TX clock
2D Alignment Buffer MPRS Channel Bonding is built around the 2D Envelope Alignment buffer (called ENV_TX at the transmitting end and ENV_RX at the receiving end) Each cell in the buffer stores one EQ The buffer has N columns and M rows. N number of channels (N=4 for 100 Gb/s ONU, N=2 for 50 Gb/s ONU, N=1 for 25 Gb/s ONU) M - should be twice as large as the maximum skew / propagation delay variability (in 802.3ca, M=32) The buffer is filled and read in cyclic pattern rowby-row When four LLIDs transmit in parallel on separate channels, the source for each cell in a row would be a different LLID () When one LLID transmits over 4 channels (channel bonding), the source for each cell in a row is that LLID () Many other combinations are also possible (2+2, 3+1, 2+1+1) 6 [0] [1]... [m] ENV_TX 2x25G (bonded) 0 1 2 3 To 25GMII 0 1 2 3...... 30 31
Independent Schedule on Each Channel Envelopes for different LLIDs can be scheduled independently on different channels MPRS will automatically interleave the EQs to fully utilize granted envelopes. 100 Gb/s 75 Gb/s 50 Gb/s 25 Gb/s 0 Gb/s LLIDs A and C on ch. 0 LLIDs B and A on ch. 1 LLIDs C, A, and B on ch. 2 LLID B and A on ch. 3 7
Skew Remediation No matter at what time a particular EQ is received, it is written in ENV_RX buffer in the same location (row) that it had in the ENV_TX buffer. TX UC0 UC1 UC2 UC3 Envelope Header records the row index of this header in the TX buffer (12) 1 2 3 4 6 8 11 14 17 19 (15) 5 (18) 7 9 12 15 10 13 16 18 20 21...... 12 13 14 15 16 17 18 19 20 21 22 23... ENV_TX row index Thus, the relative alignment of envelopes in the ENV_RX buffer is identical to the alignment these envelopes had in the ENV_TX buffer. RX UC0 UC1 UC2 UC3 (18) skew 10 13 16 18 20 21... (15) skew 5 (12) 1 2 3 4 6 8 11 14 17 19 7 9 12 15 Data is received on multiple channels with different propagation delays (skew) As a result, EQs are passed to the receiving in exactly the same order that they were sourced by the transmitting Realigned RX UC0 UC1 UC2 UC3 Envelopes are written in the RX buffer in a row index as recorded in 8 (12) 1 2 3 4 6 8 11 14 17 19 (15) 5 (18) 7 9 12 15 10 13 16 18 20 21...... 12 13 14 15 16 17 18 19 20 21 22 23... Receive time ENV_RX row index
802.3ca standard will support channel bonding The problem of skew remediation was solved in space domain, rather than in time domain. Dealing with time is hard -- Requires infinite timing resolution since skew may take values on a continuous timing scale. Dealing with space is much easier -- Just need to determine the proper location to store each received data unit in the RX buffer. Skew remediation mechanism equalizes all delays between transmitting and receiving MPRS Service Interfaces on all channels Propagation delays Conclusion Jitter in any sublayers below MPRS (25GMII, PCS, PMA, PMD) 9