CMS Tracker Synchronization K. Gill CERN EP/CME B. Trocme, L. Mirabito Institut de Physique Nucleaire de Lyon
Outline Timing issues in CMS Tracker Synchronization method Relative synchronization Synchronization of readout chain across all channels in system Absolute synchronization Synchronization with LHC collisions and rest of CMS. Monitoring synchronization karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
CMS Silicon Strip Tracker Unprecedented use of microstrip technology. Enormous system. 21m 2 silicon 1 million channels ~25 m 3 Must be very well synchronized to be able to detect, reconstruct and measure particle tracks with expected precision karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
CMS Tracker readout and control parts Detector CLK Tx/Rx Tx/Rx T1 I2C Control module digital optical link TTCrx µp PLL Front End Controller APV Front End Module TTCrx DCU 256:1 APV MUX ADC Optical transmitter analogue optical link FPGA FPGA RAM Front End Driver 25k sensors, 17k modules 3km fibres 75k APVs 17k AOHs Not only large amount of silicon. Rest of system also large and complex. All parts now defined and in production Starting to look in detail at final system aspects calibration and synchronization 32 mfecs 44 FEDs karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
timing issues: components TTCrx Digital optical control link DOH Front-end T1, RST, CAL FEC TTC control ring LHC CK FED Analogue optical readout link FEH TPLL APV TTCrx AOH Back-end karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
timing issues: cable delays (rough estimates) T1, RST, CAL ~5ns TTCrx FEC variable length ~2-3ns DOH ~2ns ~2ns Front-end ~2ns ~2ns ~2ns TTC ~2ns ~2ns LHC CK ~5ns FED variable length ~2-3ns FEH TPLL APV ~2ns ~2ns TTCrx AOH Back-end karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
Tracker timing issues - front-end APVs TTC Slow ctl Goal is to synchronize all APVs in Tracker relative to LHC collisions DOH (a) APV output in peak mode ADC counts 1 8 6 4 2 4 8 12 16 time [nsec] pre-rad 1 Mrads 4 Mrads 1 Mrads 2 Mrads 2+anneal Signal is broad, synchronization to ~25ns OK 2 24 (b) deconvolution mode at high luminosity data FEH TPLL APV AOH adjustable timing skew ADC counts 1 8 6 4 2 4 8 12 16 time [nsec] pre-rad 1 Mrads 4 Mrads 1 Mrads 2 Mrads 2+anneal Peak now narrow, require few ns synchr. ( Also, APV latency must be set correctly to read data from correct pipeline location) 2 24 karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
Timing issues: back-end (FEDs) (a) APV signals at FED after trigger 5 Digital header 4 ADC Counts 3 2 1. 1. 2x128 analog samples 2. 3. 4. Time (μs) 5. APV ticks 6. FEDs must also be well synchronized (b) Zoom on ticks Amplitude (ADC counts) 4 36 32 28 24 2 16 12-25 25 Time (ns) Optimum sampling point 5 75 Delay FPGA before front-end FPGA Coarse clock skew (25ns steps) to analyse same APV sample across whole set of 12 inputs Fine clock skew (1ns steps) to allow enough settling time (readout b/w<1mhz) Need to set to be ~2ns after start of signal pulse karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
CMS Tracker synch requirements summary Synchronization requirement APV trigger latency setting APV frame finding at back-end FED APV sampling of detector signals Optical link sampling at FED Low luminosity running Correct bunch crossing (25ns) Same clock cycle (25ns) Coarse (25ns) (peak mode) Fine (3ns) High luminosity running Correct bunch crossing (25ns) Same clock cycle (25ns) Fine (3ns) (deconvolution) Fine (3ns) karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
Tracker relative synchronization procedure Method proposed after experience in 25ns test-beam in 2 Based on measurement of time of arrival of APV ticks at FED Idea now well thought-out Procedure implemented in basic form using XDAQ Tested in recent beam/system-tests >1 control ring >1 prototype FED Standalone operation for procedure envisaged in final system Local trigger Local DAQ - FED spy channel/vme Idea is that this functionality is available during integration/commissioning karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
APV tick-marks APV output 11 received at APV tick tick 11 clock cycles 7 (4MHz) clock cycles 1.75μs APVs tick every 7 clock cycles starting at a fixed time after re-synch (11) signal received at APV Tick transmitted over analogue optical link to FED Measure arrival time at FED (trigger with APV frame OFF) Knowing analogue optical link lengths (from database) know time when the ticks left the APVs therefore use ticks for precise probe of timing skew between APVs time karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
TK front-end relative synchronization At level of 1 control ring/fed relative delays due to position of APV around the control ring analogue link fibre length Build up detailed picture of ticks by sweeping front-end TPLL delay at front-end in 1ns steps APV tick signals transmitted to FED 11 sent from FEC Different arrival times of ticks at FED - fibre lengths - APV position in control ring Knowing analogue fibre lengths then synchronize APVs using the programmable skew on TPLL XDAQ implementation for beam and system tests L. Mirabito, B. Trocme previously N. Marinelli Time Expand method to cover different control rings in partition by comparing signals on different FEDs Only ~1 channel/fed needed since inside rings, all APVs already synchronized Length of TTC cables to FED must also be known! karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
TK back-end (FED) synchronization - 1 (a) APV signals at FED Optimum sampling point 4 36 Amplitude (ADC counts) 32 28 24 2 16 12-25 25 Time (ns) 5 75 Delay FPGAs Front-end FPGAs Can use APV ticks also to set-up delay FPGAs to synchronize FEDs Fine skew setting optimizes analogue signal settling & S/N Coarse skew setting signals from same APV pipeline locations processed e.g. pedestal subtraction.. Do at same time as synchronizing front-end APVs using ticks karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
TK back-end (FED) synchronization - 2 FED e.g. CMS/TOB tests at CERN J. Valls et al, this workshop FED 1 FED 2 ADC Counts 5 4 3 2 1. 1. 2. 3. 4. Time (μs) 5. 6. Alignment of ticks at FED ensures that APV analogue signals aligned in time for the FED front-end FPGA karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
TK relative synchronization summary TTCrx DOH T1, RST, CAL FEC 5 ADC counts 1 8 6 4 2 pre-rad 1 Mrads 4 Mrads 1 Mrads 2 Mrads 2+anneal LHC CK TTC set timing on delay FPGA at FED ADC Counts 4 3 2 1. FED TTCrx 1. Amplitude (ADC counts) 2. 4 36 32 28 24 2 16 12-25 3. 4. Time (μs) 5. 6. 25 5 Time (ns) 75 AOH PLL APV 4 8 12 16 time [nsec] 2 Compensate delays at front-end PLLs around ring All fibre lengths to FEDs including TTC fibre must be known Fibre lengths to/from FEC and cables around ring good to know but not critical 24 karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
Absolute synchronization of CMS Tracker Relative synchronization to set up the Tracker ADC counts 1 8 6 4 2 4 8 12 16 time [nsec] pre-rad 1 Mrads 4 Mrads 1 Mrads 2 Mrads 2+anneal Peak mode 2 24 Relative synch aligns APVs with respect to one another but not to LHC collisions or rest of CMS Need to align APV sampling to signal generated in silicon strips by passing particles ADC counts 1 8 6 4 pre-rad 1 Mrads 4 Mrads 1 Mrads 2 Mrads 2+anneal Deconvolution Coarse timing adjust latency at APV Fine timing re-adjust PLLs at frontend 2 Absolute synchronization to find the particles 4 8 12 16 time [nsec] 2 24 Same procedure as used in beam-tests. requires: Simple tracking Global TRIDAS karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
TK synchronization summary TTCrx DOH T1, RST, CAL FEC LHC CK TTC Amplitude (ADC counts) 4 36 32 28 24 2 16 12-25 FED ADC Counts 5 4 3 2 1 25 Time (ns) 5 75 PLL APV ADC counts 1 8 6 4 2 4 8 12 16 time [nsec] pre-rad 1 Mrads 4 Mrads 1 Mrads 2 Mrads 2+anneal 2 24. 1. 2. 3. 4. Time (μs) 5. 6. TTCrx AOH Question of how to monitor state of synchronization? karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
Checking synchronization TK+ECAL TK+MU Tracker data should be consistent with rest of CMS FED occupancy should match LHC bunch structure FED compares 8-bit APV pipeline address in header on every channel with that sent from APVE 5 4 Coarse timing defects easy to see ADCCounts 3 2 Fine setting problems more difficult 1. 1. Digital header 2. 3. 4. Time (μs) 5. 6. Details of when/how to react to problems to be defined karl.gill@cern.ch LECC 23 CMS Tracker Synchronization
Conclusions Timing issues in CMS Tracker well understood confident of ability to synchronize final components and whole system Could be ready and debugged in advance of LHC collisions Relative synchronization procedure based on use of APV tick-marks simple, robust, minimal requirements Already integral part of start-up procedure for analogue readout system Implemented in XDAQ, and tested at level of >1 FEDs/FECs system hardware specs/requirements had synch. procedure in mind e.g. Optical link fibre lengths e.g. FED timing skew provision, scope mode and spy channel Absolute synchronization then done by setting correct latency at APV and adjusting PLL fine delays for maximum S/N as in beam tests Can monitor APVs and FEDs for losses of synchronization during running Next steps: further testing, automation and up-scaling of procedures karl.gill@cern.ch LECC 23 CMS Tracker Synchronization