Trigger synchronization and phase coherent in high speed multi-channels data acquisition system

White Paper Trigger synchronization and phase coherent in high speed multi-channels data acquisition system Synopsis Trigger synchronization and phase coherent acquisition over multiple Data Acquisition Systems (DAQ) is critical to applications such as radar, electronic warfare and high energy physics. The latest GSPS ADCs, with sub-nanosecond acquisition period, make the trigger synchronization and phase alignment of multiple acquisition AMCs very challenging to achieve. During the integration many factors can influence the synchronization of multi-channel acquisition system. These factors can be external to the digitizer platform such as the length of cables, or internal such as the backplane and PCB trace routing. It is thus necessary for the integrator to know the phase jitter they can expect from a platform and to be able to re-establish a synchronization after integration with external elements. VadaTech Data Acquisition DAQ Series product line has been enhanced to answer this challenge, and make high density synchronized acquisition at very high speed a reality. In the following document, we demonstrate the synchronization of two very high speed DAQ AMCs, with an accuracy of up to +- 0.5ps. Author: MATHIEU NAU Sr. Signal Processing Engineer VadaTech Asia Pacific Research Center

Contents Synopsis...1 1 Acquisition system architecture...3 2 Synchronization results... 8 3 Clocks architecture...10 4 Synchronized acquisition...11 4.1 PLLs configuration...11 4.2 Deterministic sampling of TRIGGER and frame pulse...12 4.3 Phase calibration...13 5 PLLs Configuration...14 5.1 LMK04828...14 5.2 HMC835...14 6 Deterministic sampling of TRIGGER and frame pulse...15 7 Phase calibration....16 7.1 TRIGGER and SYSREF...16 7.2 ADC SYSREF and DEVCLK...18 8 Annex... 21 8.1 Result DDC at 2500MHz, input 2502MHz...21 8.2 Result DDC at 2500MHz, input 2800MHz...22 8.3 Result DDC at 2500MHz, input 2302MHz... 23 Figure 1: Architecture...3 Figure 2: Deterministic delay between AMC004 trigger and 10MHz clock...4 Figure 3: ADC12J4000 Digital Down Converter...5 Figure 4: DAQ Series dual channel GUI...6 Figure 5: DAQ Series Phase display...7 Figure 6: Result Delta sample vs Frequency...8 Figure 7: FMC225/FMC226 architecture...9 Figure 8: System wide relationship...10 Figure 9: Sysref and trigger sampling...11 Figure 10: System phases relationships...12 Figure 11: LMK04828 Nested 0-delay Dual Loop mode...13 Figure 12: HMC835 Exact Frequency Mode...13 Figure 13: ADC Test Mode, before adjustment...15 Figure 14: Sampled trigger signal before adjustment...15 Figure 15: ADC Test mode, after adjustment...16 Figure 16: Sampled trigger signal after adjustment...16 Figure 17: Measured phase before adjustment...17 Figure 18: Phase error before adjustment...17 Figure 19: Input phase after sysref adjustment...18 Figure 20: Phase error after sysref adjustment...18 Figure 21: Input phase after ADC Sampling clock adjustment..19 Figure 22: Phase error after ADC Sampling clock adjustment..19 Figure 23: Amplitude DDC 2500MHz, input 2502MHz...20 Figure 24: Phase DDC 2500MHz, input 2502MHz...20 Figure 25: Phase error DDC 2500MHz, input 2502MHz...20 Figure 26: Amplitude DDC 2500MHz, input 2800MHz...21 Figure 27: Phase DDC 2500MHz, input 2800MHz...21 Figure 28: Phase error DDC 2500MHz, input 2800MHz...21 Figure 29: Amplitude DDC 2500MHz, input 2302MHz...22 Figure 30: Phase DDC 2500MHz, input 2302MHz...22 Figure 31: Phase error DDC 2500MHz, input 2302MHz...22 2

1 Acquisition system architecture The acquisition system is composed of the following items: VT866 chassis (5U, PCIe gen3 capable, 12 slots chassis) UTC002 MCH AMC004 Reference clock / Trigger generator AMC516 FMC225 (1 channel 4GSPS ADC) AMC517 FMC226 (2 channels 4GSPS ADC) AMC725 running VadaTech DAQ Series Input Output Controller (IOC) Remote computer running VadaTech DAQ Series Graphical User Interface Deterministic synchronized acquisition requires a common reference clock, as well as a fixed phase difference between the trigger signal and the reference clock. These requirements are met by using the AMC004 as a reference clock / trigger generator, with the clock routing capabilities of the VT866 chassis and the UTC002 MCH. The AMC004 s 10MHz reference clock is routed to the backplane TCLKA. The UTC002 clock Cross Bar Switch routes this reference clock to the AMC516 and AMC517 TCLKA. The Acquisition input signal is distributed to the FMC225/FMC226 via a RF splitter, and length-matched cables (to give phase alignment of the two input signals). Figure 1: Architecture 3

The AMC004 s time trigger function generates a trigger event at a given GPS time. This trigger signal has a deterministic phase Figure 2: Deterministic delay between AMC004 trigger and 10MHz clock 4

relationship to the 10MHz reference clock. The trigger generator is controlled over PCIe by the DAQ IOC running on the AMC725. The trigger signal is routed from the AMC004 front panel (CLK OUT) to the FMC225/FMC226 front panel (TRIG IN). The cable lengths between the trigger splitter and the FMC225/FMC226 TRIG IN are the same length. Signal Generator R&S SMB100A ANALOG IN DC Frequency 2: The DDC shift the input signal by the frequency of the Direct Digital Synthesizer Input signal from 2GHz to 3GHz DDS at 2500MHz 1: The input signal is sampled at 4Gsps DDC @ 2500MHz + Downsampling x4 IQ 15bit To ADC12J4000 DC DC Complex ouput signal, from -500MHz to 500MHz Frequency Sampling Frequency 1GHz Frequency Sampling Frequency 4GHz 3: The sampling frequency is divided by 4. The final output is a 1GHz bandwidth complex signal centered at DC. Figure 3: ADC12J4000 Digital Down Converter 5

The ADCs (ADC12J4000) on FMC225/FMC226 are configured with a 4x Digital Down Converter (DDC). The ADC s internal Direct Digital Synthesizer (DDS) center frequency can be configured from the DAQ Series GUI. The ADC output is a 15bit I/Q, 1GHz bandwidth DC-centered signal, transferred to the via the protocol. Two instances of the DAQ Series IOC run in parallel on the AMC725, each providing the control/status/results interface to its respective DAQ AMC over PCIe. The DAQ Graphical User Interface has been customized to be connected to both DAQ Series IOC, over the AMC725 front panel Status AMC516-FMC225 Status AMC517-FMC226 Amplitude graph Control AMC516-FMC225 Control AMC517-FMC226 Timestamp AMC516-FMC225 Timestamp AMC517-FMC226 IQ AMC516-FMC225 IQ AMC517-FMC226 Figure 4: DAQ Series dual channel GUI GbE. The main interface displays the Control and Status of AMC517-FMC226 and AMC516-FMC225. The Amplitude graph displays the I/Q of the current snapshot captured by AMC517-FMC226 and AMC516-FMC225, as well as the snapshot timestamp. A phase computation is available on the DAQ Series IOC, linked to two types of phase plots in the DAQ Series GUI (Phase and Phase error). This allows the user to monitor the phase difference between AMC516- FMC225 and AMC517-FMC226 in real time. 6

The Phase figure displays the angle (in milli-radian) of the current snapshot captured by AMC517-FMC226 and AMC516-FMC225. The Phase error figure displays, for each sample (t) of the current snapshot, a point with the coordinates {X = phase of sample (t) AMC516-FMC225, Y = phase of sample (t) AMC517-FMC226}. This plot provides a representation of the phase difference, as well as phase difference jitter. Phase figure AMC516-FMC225: X = timestamp of sample (t) Y = Phase of sample (t), in milli-radian Phase difference (in milliradian) between AMC516- FMC225 and AMC517- FMC226. AMC517-FMC226: X = timestamp of sample (t) Y = Phase of sample (t), in milli-radian Phase error figure Each point has the coordinates: X = Phase of sample (t) of AMC516-FMC225, Y = Phase of sample (t) of AMC517-FMC226 Phase difference (in milliradian) between AMC516- FMC225 and AMC517- FMC226. Phase difference jitter Figure 5: DAQ Series Phase display 7

2 Synchronization results The phase difference (Delta Phase) between AMC516-FMC225 and AMC517-FMC226 has been measured for different input frequencies, and different DDC configurations. The phase difference is then converted to a sampling time difference (Delta sample). DDC Center Frequency (MHz) Input Frequency (MHz) Delta Phase (mrad) Delta Sample (ps) 1500 1302 184 22.5 1500 1502 183 19.4 1500 1802 101 8.9 2500 2302-65 -4.5 2500 2502 71 4.5 2500 2800 44 2.5 3500 3302-38 -1.8 3500 3502-11 -0.5 3500 3802 13 0.5 Figure 6: Result Delta sample vs Frequency 8

3 Clock architecture FMC225 and FMC226 are built around the same ADC (TI ADC12J4000) and architecture. The LMK04828 Dual Loop PLL generates the for the (glblclk, sysref), the clock for the ADC (sysref), and the reference clock for the HMC835 PLL. The HMC835 PLL generates the 4GHz ADC sampling clock (devclk). The SYNC signal of HMC835 and the ADC12J4000 ADC are driven by the. The front panel trigger signal has a direct connection to the (sampled in the by the glblclk clock). The ADC12J4000 serial outputs follow the Subclass 1 standard. This standard features a deterministic latency between the ADC and the via the sysref signals. Backplane TCLKA TRIGIN Reference clock 10MHz GLBLCLK: 250MHz SYSREF: 10MHz CLK OUT data data CLKIN LMK04828 10MHz CLKIN SYNC HMC835 Acquisition clock DEVCLK 4GHz data Acquisition clock SYNC ADC12J4000 SYSREF 10MHz clock ANALOGIN 0-4GHz Front panel ANALOGIN Front panel Trigger Figure 7: FMC225/FMC226 architecture 9

4 Synchronization acquisition 4.1 PLL configuration The first step for deterministic synchronized acquisition is to generate all the with a deterministic phase relative to the reference clock (10MHz from AMC004). As the reference clock of both acquisition systems (AMC516-FMC225 and AMC517-FMC226) comes from the same source (AMC004), all the of both systems then have a deterministic phase relation. AMC004 10MHz reference clock The phase relationship between the input and the output need to be configured to be deterministic REF LMK CLKIN LMK04828 REF HMC CLKIN HMC835 Acquisition clock DEVCLK data DEVCLK ADC12J4000 ADC SYSREF ADC SYSREF GLBLCLK SYSREF GLBLCLK SYSREF AMC516-FMC225 CLKIN LMK04828 REF HMC CLKIN HMC835 Acquisition clock DEVCLK data DEVCLK ADC12J4000 ADC SYSREF ADC SYSREF GLBLCLK SYSREF GLBLCLK SYSREF AMC517-FMC226 Figure 8: System wide relationship 10

4.2 Deterministic sampling of TRIGGER and frame pulse Some control signals (input trigger, ADC sysref, sysref) are analog signals, sampled by digital. It is critical to optimize the phase relationship between these signals and their respective capture clock, in order to have deterministic sampling across the DAQ AMCs. As all these signals have a deterministic phase relationship to the reference clock, it is possible to optimize the arrival window relative to their sampling clock. data DEVCLK DEVCLK ADC12J4000 ADC SYSREF ADC SYSREF Register SYSREF GLBLCLK GLBLCLK SYSREF SYSREF Register SYSREF Register TRIGIN TRIGIN CLOCK Input Register The input signal is registered on the rising edge of the clock Register clock Min Setup time Optimal Arrival Min Setup time 1 Window 2 Register input A Register output A1 Register output A2 Register input B Register output B The register input A is not in the optimal arrival window. Due to input jitter, the register output is on clock cycle 1 or clock cycle 2 After delaying the input A, the register input B is in the optimal arrival window. Even with input jitter, the register output is always on clock cycle 2 Figure 9: Sysref and trigger sampling 11

4.3 Phase calibration The final step is to adjust the phase relations between both acquisition systems until the required synchronization level is achieved. AMC516-FMC225 data DEVCLK DEVCLK ADC12J4000 ADC SYSREF ADC SYSREF GLBLCLK GLBLCLK Sampled on same DEVCLK clock cycle SYSREF SYSREF TRIGIN Same phase AMC517-FMC226 ADC SYSREF data ADC SYSREF Sampled on same GLBLCLK clock cycle ADC12J4000 DEVCLK DEVCLK GLBLCLK GLBLCLK SYSREF SYSREF TRIGIN Figure 10: System phases relationships 12

5 PLL configuration 5.1 LMK04828 The LMK04828 is configured in Nested 0-delay Dual Loop Mode. In this mode, the feedback to the first PLL (Feedback PLL1) is driven by an output clock (sysref). This causes sysref to have a deterministic phase relationship to the input clock. As a result, all output have a deterministic phase to the input clock. An analog delay with a resolution of 150ps is available on the and ADC sysref path. CLKIN 10MHz Divider 12 250MHz REFCLK Phase Detector PLL1 External Loop Filter External VCXO 100MHz Divider 25 4MHz External Loop Filter Divider 12 250MHz GLBLCLK Divider 750 4MHz Phase Detector PLL2 Partially Integrated Loop Filter Internal VCO 3GHz Divider 300 SYSREF Analog Delay SYSREF Feedback PLL 2: 3GHz Analog Delay ADC SYSREF Feedback PLL 1: 10MHz HMC835 REFCLK Figure 11: LMK04828 Nested 0-delay Dual Loop mode 5.2 HMC835 The HMC835 is configured in Exact Frequency mode. In this mode, the output clock has a deterministic phase relationship with the input clock. The external SYNC signal allows us to finely tune the phase of the output clock relative to the input clock (CLKIN). The SYNC signal forces the HMC835 to initialize the phase of the output clock relative to the input clock at a user defined value. CLKIN SYNC 10MHz Phase Detector External Loop Filter Internal VCO 4GHz Delta Sigma modulator 4GHz ADC Acquisition clock Feedback PLL: 10MHz Divider 400 Figure 12: HMC835 Exact Frequency Mode 13

6 Deterministic sampling of TRIGGER and frame pulse The sysref signal is used in Subclass 1 as a synchronization signal. This signal is sampled by the 4GHz sampling clock on the ADC side (devclk), and the 250MHz glblclk clock on the side. Both the ADC and the implement an analog delay with a dirty bit detection engine, which allows the user to optimize the sampling window by tuning the analog delay. As sysref, glblclk, devclk all have a deterministic phase relationship, this adjustment only need to be done once. The trigger signal is used to synchronize the initial SYNC of the HMC835, the SYNC of the ADC (DDC reset), as well as the start of acquisition. The trigger signal is sampled by the glblclk clock. The also implements an analog delay coupled with a dirty bit detection engine on the trigger path. As the trigger signal has a deterministic phase relationship to the reference clock, it is possible to optimize the sampling window once (by fine tuning the analog delay). The ADC allows adjustment in the optimal arrival window by steps of 20ps controlled over temperature drift. The allows adjustment in the optimal arrival window by steps of 78ps controlled over temperature drift. 14

7 Phase calibration 7.1 TRIGGER and SYSREF The trigger signal has to be sampled by both acquisition systems at the same glblclk clock cycle relative to the sampled sysref signal. This adjustment is done using digital delays in the acquisition system, combined with the ADC configured in test mode (ramp generator). In test mode, the ADC generates a known sequence of data. This sequence is initialized at the first sysref event following the deassertion of the SYNC signal. If the sequences captured on AMC516-FMC225 and AMC517-FMC226 match, this indicates that the ADC SYNC signal is synchronized on both systems, and that the trigger rising event is sampled at the same time (relative to sysref). Before adjustment, the ADC test pattern is not synchronized between AMC516-FMC225 and AMC517-FMC226. By monitoring the sampled trigger signal on an oscilloscope, we measure a one glblclk (4ns) clock period between both trigger. Figure 13: ADC Test Mode, before adjustment Figure 14: Sampled trigger signal before adjustment 15

After adjustment, the ADC test pattern is synchronized, and the delay measured between both trigger signals is less than a glblclk clock period (4ns). Figure 15: ADC Test mode, after adjustment Figure 16: Sampled trigger signal after adjustment 16

7.2 ADC SYSREF and DEVCLK After the tuning of the s sysref and trigger signal, the final adjustment concerns the ADC sysref and devclk. The ADC sysref signal has to be sampled on the same devclk clock cycle on both acquisition systems. The ADC sysref signal can be adjusted using the output analog delay in the LMK04828. After adjustment, the acquisition should be synchronized at +-0.5 Acquisition clock cycle (+-125ps). This fine tuning has to be done once. Finally, the relative phase of both ADCs Acquisition clock (devclk) can be fine-tuned using the HMC835. The following are the results obtained at the different steps of adjustment. Before adjustment, the phase difference of a 2502MHz input signal, between AMC516-FMC225 and AMC517-FMC226 is -3 radian. Figure 17: Measured phase before adjustment Figure 18: Phase error before adjustment 17

After the ADC sysref tuning, the phase error for the same input signal is 1 radian: Figure 19: Input phase after sysref adjustment Figure 20: Phase error after sysref adjustment 18

Finally, the fine tuning of the ADC Sampling clock (devclk) phase decreases the phase error up to around 0 radian phase error: Figure 21: Input phase after ADC Sampling clock adjustment Figure 22: Phase error after ADC Sampling clock adjustment 19

8 Annex 8.1 Result DDC at 2500MHz, input 2502MHz Figure 23: Amplitude DDC 2500MHz, input 2502MHz Figure 24: Phase DDC 2500MHz, input 2502MHz Figure 25: Phase error DDC 2500MHz, input 2502MHz 20

8.2 Result DDC at 2500MHz, input 2800MHz Figure 26: Amplitude DDC 2500MHz, input 2800MHz Figure 27: Phase DDC 2500MHz, input 2800MHz Figure 28: Phase error DDC 2500MHz, input 2800MHz 21

8.3 Result DDC at 2500MHz, input 2302MHz Figure 29: Amplitude DDC 2500MHz, input 2302MHz Figure 30: Phase DDC 2500MHz, input 2302MHz Figure 31: Phase error DDC 2500MHz, input 2302MHz 22

Contact VadaTech Corporate Office 198 N. Gibson Road Henderson, NV 89014 +1 702 896-3337 Asia Pacific Sales Office 7 Floor, No. 2, Wenhu Street Neihu District, Taipei 114,Taiwan +886-2-2627-7655 VadaTech European Sales Office VadaTech House, Bulls Copse Road Totton, Southampton SO40 9LR United Kingdom +44 2380 016403 info@vadatech.com www.vadatech.com Choose VadaTech We are technology leaders First-to-market silicon Continuous innovation Open systems expertise We commit to our customers Partnerships power innovation Collaborative approach Mutual success We deliver complexity End-to-end Processing System management Configurable solutions We manufacture in-house Agile production Accelerated deployment AS9100 accredited AS9100 CERTIFIED