Low Power RF (LLRF) Part III

Low Power RF (LLRF) Part III S. Simrock DESY, Hamburg, Germany LLRF Part III, KEK Seminar, March 14, 20081

Lecture Schedule (March 2008) LLRF Part I (Requirements and Design) March 6, 13:30 LLRF Part 2 (Maschine Studies at FLASH) March 7: 10:00 LLRF Part 3 (LLRF for the XFEL) March 11 at 13:30 Timing and Sync. Part I (Concepts) March 14 at 10:00 Timing and Sync. Part II (Design) March 17 at 10:00 European XFEL (Project Overview) March 26 at 13:30 LLRF Part III, KEK Seminar, March 14, 2008

Outline LLRF Part III LLRF Requirements for the XFEL Comparison of Crate Standards Architecture of LLRF System Hardware ATCA Carrier AMC Mezzanine Cards Communication links IPMI, diagnostics Piezo driver and piezo control Downconverter MO and Distribution LLRF Part III, KEK Seminar, March 14, 2008

Outline LLRF Part III (C tnd) Software Architecture Software Controller (Warsaw) Controller (Lodz) Controller (DESY) Low Level Applications High Level Applications Automation LLRF Servers Summary of LLRF Review LLRF Part III, KEK Seminar, March 14, 2008

LLRF Requirements for the XFEL LLRF Part III, KEK Seminar, March 14, 2008

System Architecture Details XFEL Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 4

XFEL Subsystems Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 5

XFEL Signal diagram for RF Control (1 RF Station) Cavity Signals Interlock Signals Beam Diagnostics HPRF ~130 x ~10 x ~10 x ~3 x LLRF 1 x 32 x 32 x 64 x ~3 x Klystron Drive Cavity Tuner fast and slow RF Power transmission HPRF ~3000x(derived signals) Operator Console Control System Database Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 6

Challenge for Software Development XFEL I/O P19 A34,A54, A109, A23 C08 C19 C03 P21 A12,A78, A102, A54 P14 A49,A91, C13 A35, A28 C09 P23 A77,A67, A31, A13 C18 I/O Pxz = Processor (FPGA,DSP, CPU) Anm = Application P21 A29,A22, A83, A08 I/O C27 C11 P08 A18,A11, A71, A29 C21 P07 A21,A88, A99, A71 Ckl = Communication Link S. Simrock, Summary LLRF Review XFEL Meeting, January 15, 2008 8

Use cases for LLRF System (RF Station) XFEL Standby Calibration Resonance Control Parameter Optimization Establish moderate RF power Enable measurements Database Application Determine Performance Statistics Motor tuner Piezo tuner Change Settings Field error Robustness Exception Detection and Handling Field control Beam Feedback Energy Beam load. Comp. Bunch compression Gradient Pulse length, rep. rate Beam current Prepare new settings Klystron, Modulator Power Transmission Cavity, coupler Frequency Tuners Timing, Synchronization LLRF hardware LLRF software Networks S. Simrock, Summary LLRF Review XFEL Meeting, January 15, 2008 9

RF System Requirements XFEL Maintain Phase and Amplitude of the accelerating field within given tolerances to accelerate a charged particle beam to given parameters - up to 0.02% for amplitude and 0.01 deg. for phase Minimimize Power needed for control RF system must be reproducible, reliable, operable, and well understood. Other performance goals - build-in diagnostics for calibration of gradient and phase, cavity detuning, etc. - provide exception handling capabilities - meet performance goals over wide range of operating parameters Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 7

XFEL LLRF Requirements (C ntd) Availability not more than 1 LLRF station failure / week SEU tolerant Redundancy of LLRF subsystems... Operability One Button operation (Automation) Application assist operators and rf experts Automated calibration of vector-sum... Reproducible Restore beam parameters after shutdown or interlock trip Recover LLRF state after maintenance work... Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 8

XFEL LLRF Requirements (C ntd) Maintainable Remote diagnostics of subsystem failure Hot Swap Capability Accessible Hardware... Well Understood Performance limitations of LLRF fully modelled No unexpected features... Meet (technical) performance goals Maintain accelerating fields - defined as vector-sum of up to 32 cavities - within given tolerances Minimize peak power requirements... Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 9

XFEL Improvement of the FLASH LLRF for the XFEL Field regulation : Short term: Improve by factor 3 (0.03 deg. 0.01 deg.) Phase drifts: Improve by factor 10 (2ps 0.2 ps) Need modular design with high availability (HA): Upgradeability, maintenance Useable by other WP, support collaborative efforts Automation Diagnostics Documentation (initially: good requirements are needed, then concepts, design, acceptance tests) Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 10

New WBS for LLRF for the XFEL XFEL Risk Assessment LLRF Project Management Major System Requirements Hardware Software Infrastructure Inst/Maint. Commissioning/ Operation Special Topics 1.1 MO & distr. 2.1 Controller 3.1 Cabling 4.1 Operation and Evaluation in FLASH/MTS 5.1 Transients detection 1.2 Digital Feedback 2.2 Low Level Applications 3.2 Racks and crate 4.2 Procedures 5.2 WGT Control 1.3 Field det. & Actuators 2.3 High Level Applications 3.3 Documentation and Operation Manuals 4.3 Automation 5.3 Interfaces to other systems 1.4 Piezo Control 2.4 Communication protocols 3.4 QA and QC 4.4 Diagnostics 5.4 Radiation Immunity 1.5 Radiation monitoring 1.6 Communication interfaces 2.5 Control system (DOOCS) 2.6 Global control 3.5 Redundancy, Availability Analysis 3.6 System integration 4.5 Simulation 5.5 Calibration parameters database 3.7 Installation in FLASH/MTS/XFEL Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 11

Main LLRF Requirements for the XFEL XFEL 1. Provide settability of voltage and phase to the desired values in all 4 quadrants up to a klystron peak power output level of 0.9*P_sat. 2. Maintain stability of voltage and phase of the calibrated and high precision vector-sum of individual rf stations within given tolerances for the range of useable operating parameters. 3. Provide highly stable rf references at specified frequencies at selected locations. Includes calibration reference signals. 4. Provide adequate interfaces to other accelerator subsystems. 5. Diagnose faulty or missing hardware and software and localize areas of functional and technical performance degradation including severeness of degradation. For use by operators and experts. Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 12

Main Requirements for the XFEL (Cnt d) XFEL 6. Optimize and/or limit operational and system internal parameters such that the performance function based on rms field stability, accelerator availability, and component lifetime is maximized. 7. Provide a simulation mode, where the klystron-cavity system is replaced by a simulator and which provides performance predictions for planned parameter changes. 8. Provide a high degree of automation of operation to assist the operator and system experts. 9. Provide calibration functions for selected signals. 10.Provide low and high level applications supporting automation. 11.Provide exception detection and handling. 12.Provide operating modes for rf system conditioning (ex. coupler and cavity). 13.Support rf system and accelerator commissioning procedures. Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 13

XFEL Basic and Advanced Use Cases for RF Station Basic Use Cases Establish moderate RF power and cavity gradients Enable and perform measurements of all LLRF relevant signals Stabilize fields for beam operation Advanced Use Cases Optimize parameters for best beam stability Set parameters to maximize availability during beam operation Tune or detune cavity from/to completely detuned state Assess performance and performance limitations of rf station Diagnose problems and identify the source (hardware/software) Detect and handle exceptions Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 14

XFEL Examples for Scenarios 1. Coarse tuning of cavity resonance with motor tuner 2. Compensate Lorenz force detuning 3. By-pass/un-bypass cavities (to/from completely detuned state) 4. Adjust klystron HV for sufficient power margin 5. Set correct timing.. rf gate, rf pulse, klystron HV, flat-top with respect to beam 6. Limit field emission in cavities 7. Apply adaptive feedforward 8. (Re)-start missing or faulty llrf servers 9. (Re)-calibrate rf station 10. Calibrate vector-sum at full beam loading 11. Calibrate downconverter Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 15

XFEL Non-Functional Requirements Field control (up to 0.02% for amplitude, 0.01 deg. for phase) for vector-sum of each rf stations intra-pulse and pulse to pulse (0.03 deg. for several minutes) Long term drifts are corrected by beam based feedback Vector-sum calibration to 1 deg. in phase and 1% in amplitude Adjust incident phase to +- 3 deg. Adjust loaded Q to +- 2% Resonance control Coarse tuning (motor tuner) to 0.2 BW Fast tuning (Pietzotuner) to 0.1 BW (LF detuning) Calibration of downconverter with reference signal Provide frequency and phase reference to llrf and other subsystems. Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 16

Non-Functional Requirements (Cnt d) XFEL Electronics (racks,crates, boards, and cabling) Crate and board standard compatible with control system Must tolerate moderate levels of radiation (n and gamma) Modular design to facilitate maintenance/upgrades Fulfill european standards for electrical safety Crate cabling only from rear possible Installation compatible with racks with no rear access Interfaces to other subsystems Machine protection and personnel safety Control system HPRF, Cryo, vacuum, cavities, couplers Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 17

Crate standard related requirements XFEL Support ~100 ADC measurement and ~100 DAC control channels with significant data processing capability Sampling rate up to 100 MHz at 14 bit (desired 16) resolution Latency from ADC input to DAC output not to exceed 500 ns (desired < 250 ns) Complex data processing <250 ns for real time feedback in FPGAs 1-10 us for intrapulse measurements <50 ms between pulses Support centralized and distributed architecture Requires high bandwidth/low latency communication links Scalable Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 18

XFEL Crate standard related requirements (Cnt d) Modular design Carrier boards with mezzanine cards (5-10 types) Maintainable Build-in diagnostics (IPMI, and for all boards) hot-swap Long lifetime of standard and availability of boards Easy access Upgradable High availability Redundancy supported by hardware and software Rear and front panel IO (few hundred high quality IO channels) Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 19

XFEL Functional Requirements Measurements Signals conditions Components characterization Control actions Diagnostics Warning and fault detection Generate events Exception detection and handling Automation (of operational procedures) Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 20

XFEL Functional Requirements (Cnt d) Measurements (Examples) Cavity gradient and phase (calibrated) Incident and reflected power (calibrated) Detuning and loaded Q Loop phase and loop gain Klystron linearity characterization Beam phase and beam current Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 21

XFEL Functional Requirements (Cnt d) Control actions (Examples) set loop phase, calibrate loop gain set loaded Q and cavity detuning set klystron HV, adjust bouncer timing Calibrate downconverter (every pulse) rf and/or beam inhibit klystron linearization Adaptive feedforward Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 22

XFEL Functional Requirements (Cnt d) Exception detection and handling (Examples) Quench Field emission Operational limit exceeded Klystron saturated Cavity severely detuned Board temperature high Stefan Simrock, DESY LLRF-ATCA Review, Dec. 3, 2007 23

Comparison of Crate Standards LLRF Part III, KEK Seminar, March 14, 2008

LLRF Requirements for crates X- Ray Laser Project XFEL X- Ray Free- Electron Laser Multi channel control require concentration of large amount of analog signals (today 14-16 bit each) in one place, due to perform vector sum calculation. There will be 96 signals per one klystron in X-FEL - it must be multi-board system (crate needed) To reduce number of boards in the crate, large sized PCB boards are preferred Since signals are distributed over several ADC boards, crate must provide fast communication on backplane to handle on-line feedback Crate must be a reliable platform for electronics, to achieve high availability of LLRF systems, and whole machine. Jaroslaw Szewinski, Inst it ute of Electronic Systems, Warsaw University of Technology, Warsaw Poland Soltan Inst it ute for Nuclear Studies, Swierk, Poland LLRF Review, DESY 3-4 December 2007

X- Ray Laser Project XFEL X- Ray Free- Electron Laser Crates evolution- bandwidth serial differential mesh/star backplanes multi-drop backplanes parallel bus Jaroslaw Szewinski, Inst it ute of Electronic Systems, Warsaw University of Technology, Warsaw Poland Soltan Inst it ute for Nuclear Studies, Swierk, Poland LLRF Review, DESY 3-4 December 2007 High bandwidth is needed to perform on-line feedback

X- Ray Laser Project Multi- drop bus vs. Star/ Mesh in LLRF XFEL X- Ray Free- Electron Laser CPU V.M. Board In multi-drop bus, only one ADC board can send data for vector sum DWC ADC Board DWC ADC Board DWC ADC Board calculation, at same time. In mesh or star topology ALL ADC boards can send data simultaneously to the CPU V.M. Board DWC ADC Board DWC ADC Board processing unit. This is needed to make low latency feedback. DWC ADC Board Jaroslaw Szewinski, Inst it ute of Electronic Systems, Warsaw University of Technology, Warsaw Poland Soltan Inst it ute for Nuclear Studies, Swierk, Poland LLRF Review, DESY 3-4 December 2007

X- Ray Laser Project Crate standards, history and overview XFEL X- Ray Free- Electron Laser VERSA (1979) Eurocard PCI (1995) VXI (1987) VME (1981) VME64 (1994) cpci (PICMG 2.0) (1997) PXI (PICMG 2.8) VXS (VITA 41) (2004) GigE on VME64x VME320 (1997) PSB/MESH (PICMG 2.16, PICMG 2.20) cpcie (PICMG EXP.0) (2005) XMC (VITA42) VPX (VITA 46) (2004) ATCA (PICMG 3.x) (2003) mtca/amc (PICMG MTCA.0) Jaroslaw Szewinski, Inst it ute of Electronic Systems, Warsaw University of Technology, Warsaw Poland Soltan Inst it ute for Nuclear Studies, Swierk, Poland LLRF Review, DESY 3-4 December 2007

Architecture of LLRF System LLRF Part III, KEK Seminar, March 14, 2008

X-Ray Laser Project Requirements (technical) One RF Station 96 analog inputs 1.3 GHz, 0dBm Isolation better then 50 db ADC 14 bits resolution, SF 40-100 MHz Clock stability > 5 ps rms RF drive output 1.3 GHz, 0dBm Interface to the interlock system System latency < 500 ns Low latency links to other modules 6 different clocks must available (stability > 5ps rms) Partial redundancy Architecture for the LLRF system based on the ATCA standard - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 2007 XFEL X-Ray Free-Electron Laser

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Concept modular system based on ATCA Problems: analog signals in ATCA are not defined no analog IOs connected from rear Architecture for the LLRF system based on the ATCA standard - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 2007

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Analog signals in ATCA Architecture for the LLRF system based on the ATCA standard - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 2007 Connections to the ATCA carrier board through RTM module Analog signals must be routed from Zone 3 to AMC modules

X-Ray Laser Project Signals in AMC bay AMC-DESY bay compromise between AMC.1 specification and needs of the LLRF System Separation between analog and digital signals in the connector pin assignment Architecture for the LLRF system based on the ATCA standard - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 2007 XFEL X-Ray Free-Electron Laser

System Architecture Architecture for the LLRF system based on the ATCA standard - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 2007 X-Ray Laser Project XFEL X-Ray Free-Electron Laser

ATCA Carrier board LLRF Part III, KEK Seminar, March 14, 2008

X-Ray Laser Project XFEL X-Ray Free-Electron Laser - 6 x AMC bay compatible with AMC.1 and AMC-DESY specification - inputs: - sinusoidal signal IF (1-50 MHz) or RF 1.3 GHz, 0 dbm - sinusoidal signal RF 1.3 GHz, 0 dbm - sinusoidal signal RF 2.6 GHz, 0 dbm - 8 digital interlock signals - outputs - sinusoidal signal RF 1.3 GHz, 0 dbm - inputs/outputs: - 3 x clock B-LVDS (5 ps rms) - 3 x trigger B-LVDS (5 ps rms) - 10 differential lines for Low Latency Protocol (custom) - Signals provided to AMC bays vi Zone 3 (RTM Module) - all signals mentioned above (for AMC bays) - Clock signals distribution on board - Floating Point Processor for Low Level Applications - User FPGA for controller ATCA Carrier Board - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 20077 Requirements

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Carrier board - concept Characteristic signals for the LLRF system AMC AMC AMC clk clk clk M M M M Mainframe FPGA PCIe switch Gbit switch Power reg. Zone 2 Zone 3 DSP 25 x 25 DSP 25 x 25 M M Power reg. Power reg. Power reg. DSP 25 x 25 M M M M M M M M Power reg. User FPGA FF1513 Power reg. M M Power reg. M M Power reg. ATC210 Main power regulator Zone 1 ATCA Carrier Board - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 20077

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Block diagram - communication 30W +3.3V 60W +12V Power AMC AMC Mezzanine Mezzanine Card Card x2 AMC AMC Mezzanine Mezzanine Card Card x2 AMC AMC Mezzanine Mezzanine Card Card x2 6 x 1 PCIe 6 x Ethernet DSP DSP DSP DSP Link PCIExpr. Fast link to AMC Fast link Gbit ethernet Switch 10/100/1000 MHz PPC user FPGA FX4 32A / 64D PCIE switch Processor unit ZONE1 ATCA Carrier Board - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 20077 ZONE2 Firmware FPGA ZONE3 PPC

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Analog signals Analog and digital lines are separated For analog signals and fast digital lines strip-lines are designed ATCA Carrier Board - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 20077

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Timing distribution 3 x clock & 3 x trigger signals generated at any AMC slot must be distributed to each AMC slot and others carrier boards LVDS bus bidirectional buffers OE DIR ATCA Carrier Board - Tomasz Jezynski, DESY LLRF Review, DESY, December 3, 20077

AMC Mezzanine Cards LLRF Part III, KEK Seminar, March 14, 2008

XFEL AMC specification Advanced Mezzanine Cards are printed circuit boards (PCBs) that follow a specification of the PCI Industrial Computers Manufacturers Group (PICMG). ATCA board Features of AMC modules: Edge connector (max. 340 pins), Build-in IPMI controller, Two power supplies +3V3/+12V, Maximum power 60 W per module Module dimessions: 180mm x 73mm x 28mm Communication interfaces: Fabric interface: PCI Express (PCI Express Advanced Switching) Gigabit Ethernet and XAUI Serial RapidIO System Management Interface Synchronisation Clock interface JTAG Test Interface Power (+3V3, +12V) AMC module Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 4

AMC standard and Low Level RF system XFEL Main LLRF controller will be built on ATCA carrier board, while auxiliary submodules will be build on AMC modules: 8 x ADC (100 MHz) + FPGA (front panel and rear connection) industrial module available, our design in progress, Vector modulator + 2 x DACs (800 MHz) + FPGA + memory, Transient detector, 1 x ADC (2 GHz) + fast static memory, Clock Synthesizer and Timing Module, Piezo controller, Radiation monitoring module (detection of neutron and gamma radiation). Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 6

XFEL Block diagram of the typical LLRF AMC module (1) Top module with connector type B Main FPGA with PCIe interface Memory CLK Bottom module with connector type A IPMI controller Connector Voltage and temperature sensors Power supply Main electronics dependent on the module function (analogue signal, low latency links, clocks and triggers) Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 7

XFEL Block diagram of the typical LLRF AMC module (2) Bottom module with main functionality Connector B+ Connector A+ Top module with control logic Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 8

XFEL Octuple Analog-to-Digital module Requirements: Octuple ADC: 14-16 bit, 100 MHz sample rate, conversion time up to 7 clocks Configuration interface: AMC.1 PCI Express x1 FPGA: Virtex 5 with external static memory (2-4 MB, 250 MHz) Two different configurable clocks for ADC 1-4 and ADC 5-8 Clock distribution stability better than 5 ps Full support for IPMI standard Additional signals from rear connector: 8 analog conditioned input signals (±1 V, 10-100 MHz) 6 clock inputs up to 100 MHz (LVDS with jitter less than 5 ps) 2 clocks connected to FPGA and ADC 4 clocks connected to FPGA Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 10

XFEL Octuple Analog-to-Digital module - TAMC900 Features: 8 x LTC2254, 14-bit, 105 Msps ADC converter 4 MB of QDR II memory (data buffer for maximun 2 ms) 3 exernal clock and 3 trigger inputs Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 11

Octuple Analog-to-Digital module Warsaw version AMC connector A+ AMC connector B+ XFEL Analog signals 1-8 Signal conditioning band-pass filters Clocks 1-2 Data 8 bits Clocks 3-6 Interconnector PCI Express JTAG I2C uc Atmega 128 inhib it Power supply ADC 1-4 AD6645 ADC 5-8 AD6645 Clock distribution AD9510 AD9513 Data 8 bits JTAG EEPROM XCF16P Hot -swap switch Data 8x16 bits Buffers Clocks 1-2 Data enable Clocks 3-6 Clock configuration bus FPGA Virtex5 XC5VLX30T Interconnector Features: Data 8x16 bits Board with analog and clock distribution components 8 x AD6645, 14-bit, 105 Msps ADC converter 4 MB (512 kb x36) of QDR II memory 6 external clock/trigger inputs MMC controller Data 36 bits QDR II SRAM 512K x 36 Board with IDT71P74604 digital and IPMI components Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 12

XFEL Vector modulator (1) Requirements: LO input frequency : 1.3 GHz Nominal Input level: ± 1 V pp Nominal Input power range: ± 6 dbm Input impedance: 50 Ohm nom. Input VSWR : max. 1.5:1 (input return loss = 14 db for VSWR = 1.5) Ouput frequency: 1.3 GHz Output level: 0 dbm nom. Output impedance: 50 Ohm nom. Operating temperature range: -10 deg. C to +70 deg.c Humidity: max. 95 % non-condensing Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 13

XFEL Prototype of vector modulator Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 14

XFEL Vector modulator (2) Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 15

Clock Synthesizer and Timing Module (1) XFEL Designed by: Michał Ładno, Krzysztof Czuba Clock synthesizer requirements: Synthesize clock from the MO 1.3 GHz reference signal Clock frequencies: 10 MHz 100 MHz with1 MHz step Clock stability better tham 5 ps, (desirable < 2ps) 3 independent clock outputs (LVDS levels) Timing Receiver requirements: Receive and decode timing signals from the existing FLASH timing Optical fibre input 3 independent trigger outputs (LVDS levels) Internal trigger generation mode (trigger frequency 0.1 33 Hz) Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 17

Clock Synthesizer and Timing Module (2) XFEL 1,3 GHz MO signal 1,3 GHz MO signal : 8 : 8 Timing Timing Signal Signal 162,5MHz 162,5MH z OPTO REC OPTO REC PLL3 PLL3 PLL2 PLL2 PLL Diagnostics Diagnostics (temperature, power levels) Control Control Connector for the Top Module Connector for the Top Module +12V +12V 3.3V, 3.3V, 5V, 5V, -5V -5V DC\DC DC\DC 10-100 MHZ 10-100 LVDS MHZ LVDS 3 x Trigger LVDS 3 x Trigger LVDS A M C A + A M C A + Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007

XFEL Requirements for Radiation Monitoring (RAMC) Requirements: Detection ability: neutron fluence, gamma dose Lowest detectable level of fluence: 10 4 10 5 n*cm -2 Lowest detectable level gamma: 10-3 10-2 Gy(Si) Level of neutron fluence tolerance: in range of 10 12 n*cm -2 Level of gamma rad. tolerance: in range of 10 3 Gy(Si) Dynamic range for neutron fluence: 6 orders of magnitude Dynamic range for gamma: 3 orders of magnitude Gamma and neutron radiation should be monitoring in real-time in each ATCA crate and in various places where other electronics is installed. When allowed dose or fluence is violated alarm should be triggered (IPMI message). All measured data should be stored in main data base for further analysis Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 21

XFEL Radiation monitoring RAMC Top module with connector type B Temperature sensor IPMI controller Gamma radiation sensor Neutron sensor Calibration parameters Temp. stabilisation Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3-4 December 2007 22

Communication Links LLRF Part III, KEK Seminar, March 14, 2008

XFEL Requirements Agenda Requirements control Concept Communication interfaces in ATCA standard 1. Fast, low latency links for fast feedback loop, piezo 2. High throughput between pulses for DAQ systems sending a lot of data recorded during pulse 3. Boards management mechanism 4. Boards control in ATCA Shelf 5. Control of RTMs 6. Interfaces to other systems Base Interface Fabric Interface Links on a Carrier Board Links to RTM Links to Other Systems XFEL-LLRF-ATCA Meeting, 3-4 December 2007 Schedule 3

XFEL Communication interfaces offered by ATCA standard PICMG3.0 General Specification, ATCA Backplane Topology, IPMI E-Keying Subsidiary Specifications: PICMG3.1 Ethernet / Fibre Channel PICMG3.2 Infiniband PICMG3.3 PCI Express / Advanced Switching PICMG3.4 Starfabric PICMG3.5 RapidIO PICMG3.6 PRS fabric XFEL-LLRF-ATCA Meeting, 3-4 December 2007 4

XFEL Other systems AMC1 AMC1 AMC1 RTM1 General concept Low Latency Link PCIe I 2 C Ethernet AMC1 AMC1 RTM2 AMC1 HUB Other systems CPU Shelf Manager ATCA Shelf 5

XFEL Latency in selected communication standards Rocket IO direct peer-to-peer connections latency around 300 ns (Virtex 2 Pro) and 100 ns ( 5 (Virtex PCI Express connections via switch minimum latency 500-700 ns depending on payload size up to 10 us with higher switch load Gigabit Ethernet 2.5 us 10 Gigabit Ethernet 250-600 ns XFEL-LLRF-ATCA Meeting, 3-4 December 2007 6

Agenda XFEL ATCA-LLRF Channel Definition for Shelf Backplane Rx+ Rx- Tx+ Tx- Rx+ Tx+ Rx- Tx- Fabric Channel Port 2 + - + - + - + - Port 3 PICMG 3.0 Port 0 + - + - + - + - Port 1 Rx+ Rx- Tx+ Tx- Rx+ Tx+ Rx- Tx- Port 2 Option 1 ATCA-LLRF Channel Option 2 Custom Link Custom Link Custom Link 1 Custom Link 2 Port 3 Port 2 Custom Link 1 Custom Link 2 Port 3 Port 0 Lane 1 Lane 2 (Optional) Port 1 Port 0 1000BASE-BX 1000BASE-BX (Optional) Port 1 PCIe Link Ethernet XFEL-LLRF-ATCA Meeting, 3-4 December 2007 7

XFEL How many slots we need in one ATCA shelf? 5 x Carrier Boards (2 slots each) 10 slots 4 cards for 4 cryo-modules 1 card as a main controller 1 HUB (or 2 for redundancy) 1 or 2 slots CPU 0,1 or 2 slots 1 CPU without redundancy 2 CPUs with redundancy 0 CPU in case of distributed control system (AMC- CPU cards) TOTAL: 12-14 slots ATCA shelf XFEL-LLRF-ATCA Meeting, 3-4 December 2007 8

Base Interface in 14 Slots Shelf XFEL Clock Update P20 HUB 1 2 3 4 5 6 7 8 9 10 11 12 13 14 To Shelf Manager Fabric Interface Base Interface P21 P22 P23 P24 13 12 11 10 9 8 7 6 5 4 3 2 1

Backplane Links in the 14 Slots 1000BASE-T RocketIO Full Mesh Shelf Clock Update P20 1000BASE-BX XFEL Optional PCIe or 1000BASE-BX HUB CPU C5 C4 C3 C2 C1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Utilization 10 of 91 channels 11% Maximum 16 of 91 channels 17% Fabric Interface P21 P22 To Shelf Manager Utilization 8 of 14 channels 57% 100% with redundant HUB Base Interface P23 P24

IPMI, Diagnostics LLRF Part III, KEK Seminar, March 14, 2008

XFEL Intelligent Platform Management Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 4

XFEL ATCA crate and ShelfManager IPMI interface I 2 C Redundant ShelfManager Ethernet 10/100 Mbit Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 5

XFEL ATCA carrier board with IPMC IPMB-L IPMB-A IPMB-B Peripherals IPMB-L AMC - I2C Bus Peripheral I2C Bus Redundant I2C IPM-Buses that aggregate into IPMB-0 Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 6

XFEL Module Management Controller Fabric Interface Management Power GA [2..0] E-Keying E-Keying FRU information EEPROM A M C B+ Control signals IPMB-L (I2C) PS1 PS0 MMC Sensors Payload reset Hot Swap switch Watchdog Payload Power Blue LED Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 7

XFEL Power Distributtion Management Architecture SM (Power budget) IPMC (Carriet current and AMC bays descriptors) MMC (Current descriptor) Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 8

Electronic-keying XFEL Mechanical KEY Electronic Keying is the mechanism used to dynamically satisfy the needs that had traditionally been satisfied by various mechanical connector keying solutions: Prevent mis-operation Verify fabric compatibility E-Keying between the Modules and switch(es) Point-to-point E-Keying for AMC modules Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 9

Diagnostics of ATCA-based systems XFEL Useful diagnostics offered by ATCA standard: 1. Thermal violation IPMC controls temperature of ATCA and AMC components 2. Monitoring of supply voltages 3. Monitoring of supply currents 4. Monitoring of main power converter (-48V to +3V3/+12V) 5. Payload control (read of device state, reset and other userdependent diagnostics) 6. External watch-dog for IPMC (with external oscillator independent from IPMC oscillator) 7. SM periodically tests state of IPMC on ATCA boards 8. Electronic Keying (E-Keying) Additional diagnostics: 1. Monitoring of gamma and neutron radiation in ATCA crates and other important spots in accelerator tunnel 2. Reference clocks detectors Dariusz Makowski, Technical University of Łódź LLRF review, DESY, 3 December 2007 12

Piezodriver and Fast Tuning Control LLRF Part III, KEK Seminar, March 14, 2008

The main aim of Piezo Control system XFEL Drive the piezoelements assembled in fast tuners frames to minimize the Lorentz force and microphonics effects On-line frequency detuning calculation Dimensions: 10x10x30mm Manufacturer: NOLIAC Microphonics measurement (i.e. diagnostics of cryogenic system) Dimensions: 10x10x36mm Manufacturer: PI Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 3

XFEL General requirements of PiezoControl system Lorentz force detuning (LFD) during flat-top Δω flat top < 10 Hz for field up to 30 MV/m (compensation up to 600 Hz possible resonance compensation up to 1kHz) Commercial available piezoelements (PI and NOLIAC) C 2K = 3 5 µf, V max = 100 V, oper. freq. for LFD/microphonics up to 1 khz (full voltage scale), I load ~ 300mA Maximal repetition rate of RF (LFD compensation) pulse under 10 Hz (20 Hz optional if better klystron available driver need to be verified) Scalable system: 101 modules, 808 cavities, 1616 Piezos Piezo must be protected and monitored 2 piezo for each cavity (higher reliability) (piezo is fragile to over current and over voltage (>150 200), piezo lifetime must by over 10 10 pulses, resonance in the cables, piezo might fall out when stepper motor is wrongly tuned) System must adjust pulse generated to piezo in regards to RF pulse (different accelerating field gradient, flat-top and rising time duration demands different settings feed-forward tables) Possible microphonics compensation between the RF pulses (sensor/actuator mode) (microphonics has smaller impact than LFD, constant offset of Δω during flat top, feedback loop) Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 4

Overview of the piezocontrol system XFEL 8x 8x 8x 8x Forward Reflected Probe 8 8 8 8 8 8 8 8 8 8 8 8 Controller Low Level Application High Level Application Control System Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 6

Overview of the piezocontrol system XFEL Standalone box x4 8x 8x 8x 8x 8 8 ch ch PZD 8 PZD in 1 amplifier 8 ch PZD amplifier PZD amplifier amplifier DAC DAC 32chin1 x1 ADC ADC 32chin1 8 x1 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 Controller Piezo Controller ATCA compatible Low Level Application High Level Application Control System Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 7

XFEL Piezocontroller - FPGA x32 Δt, f, A ref_table LF/piezo ΔA Sine generator K 1 Σ U piezo act trig K 3 reference errors signal Δω reference error signal (LFD) f(δω) K 2 (microphonics) f(u Piezo_sens ) PID controller Piezo diagnostic Interlocks U piezo sens Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 9

Piezocontroller architecture for 32 cavities ATCA XFEL Control System Timing LLA PCIExpress Δt, f, A ref_table trig Low Latency Link 32ch Δω Low Latency Link FPGA PCIExpress Rocket I/O optolink Rocket I/O optolink DAC 32ch in 1 DAC ADC 32ch in 1 ADC PZD 4x 8ch in 1 PZD PZS PZS PCIExpress Control System Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 10

Piezocontroller architecture for 32 cavities ATCA XFEL Control System PCIExpress Δt, f, A ref_table PCIExpress Rocket I/O optolink DAC 32ch in 1 DAC PZD 4x 8ch in 1 PZD Timing trig Low Latency Link FPGA U, I, T monitoring LLA ADC 32ch Δω Rocket I/O 100ch in 1 Low Latency Link optolink ADC PZS PZS PCIExpress Control System Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 11

XFEL Piezo Read-out (ADC card) Main objective: read and adjust the impedance of piezoelements, which work as sensors in fast tuners, convert to digital representation General overview: Standalone box (no ATCA, EuroCrate), close to PZD for exchanging piezostack functionality 32 analog input channels (for 4 cryomodules), 32 channel ADC (sampling higher than 10 khz/channel) Opitcal Link to PiezoController for data exchange and board monitoring Piezo diagnostic (i.e. capacitance measurement) Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 13

XFEL Piezodriver scheme Inputs 8x Piezo Drivers Voltage sense output Current sense output Outputs Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 16

XFEL Results FLASH, ACC6 Acc. Grad = 22MV/m, Rep. Rate=5Hz Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 17

XFEL Results from ACC6 LFD compensation ACC6 cav8 cav7 cav6 cav5 cav4 piezo on piezo off cav3 cav2 cav1 0 50 100 150 200 250 300 350 detuning [Hz] Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 18

XFEL Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 23

XFEL PiezoController architecture The feedback loop s latencies (including calculation)< 0.5 ms The piezo-to-piezo feedback loop based on PID controller (vibration cancellation between the pulses BESSY experiments) The detuning-to-piezo loop based on adaptive feed-forward (short RF pulse in comparison to mechanical action of cavity) BESSY experiments OptoLink to DAC board and ADC cards Piezo diagnostic (i.e. capacitance measurement) Interlocks (i.e. protection (cut-off) when module is warm) Przemek Sekalski, Department of Microelectronics and Computer Science, Technical University of Lodz, Poland Review of LLRF system based on ATCA standard, Dec 3-4, 2007 24

Downconverter LLRF Part III, KEK Seminar, March 14, 2008

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Downconverter for LLRF A,ϕ Receiver CH1 RF-input LNA Δ f BPF Sampling and Field Detection ADC CIC Filter Digital I,Q- Detection Input Calibration Master- Oscillator Muti-channel downconverter LO and CLK Generation LO-input ADC clock f s Sample frequency [50MHz-130MHz] Intermediate frequency [10MHz, 50MHz]: f IF Single cavity field in amplitude and phase Frank Ludwig, DESY XFEL-LLRF-ATCA Meeting, 3-4 December 2007 5

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Main and Booster Section Downconverter n Rear Transition Module (RTM) Number of : 4x8, 4x12 or 4x16 depending Receivers on receiver cell size and PCB routing Cavity Signals (24 channels) forward, reflected, probe Reference (3 channels) Klystron Linearization (3 channels) LO Generation & Distribution - mixer method -VCOs - Laser to RF Receiver REF Receiver CH1 Receiver CH2 Receiver CH3 Receiver CH4 Receiver CH5 Receiver CH6 Receiver CH7 Receiver CH8 Board Diagnostic IPMI PWR 8 Receiver : LT5527 (Gilbert-Mixer) RF: 1300MHz, <10dBm LO: [1310MHz, 1350MHz], 0dBm IF Control : : [10MHz, 50MHz], diff. Outputs - Signal Diagnostic (Level, cable,pwr) - Frontend Attenuator - Slow Timing Drift Calibration X 4 Frontend ADCs Local Oscillator RF-Reference LO Distribution RF-Calibration Signal Frank Ludwig, DESY XFEL-LLRF-ATCA Meeting, 3-4 December 2007 7

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Injector and 3 rd Harmonic Section Downconverter n Rack Layout : Cavity Signals forward, reflected, probe 24 24 24 24 fs Laser Reference RF Reference Laser to RF-Reference Drift Calibration + Reference Receiver Calibration Line IF Signals forward, reflected, probe LO LO Generation & Distribution 24x4 Timing LO ATCA Crate 1 ADCs Upconverter Pizza boxed ATCA System or SIMCON-DSP, ESECON, ACB Piezo Driver Universal IO Frank Ludwig, DESY XFEL-LLRF-ATCA Meeting, 3-4 December 2007 8

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Injector and 3 rd Harmonic Section Downconverter n High Level Downconverter 1.3GHz Injector Receiver : Receiver Passive high level mixer RF: 1300MHz, 20dBm LO: 1313MHz, 20dBm IF : 13MHz (propose), single ended 3.9GHz Injector Receiver : Passive high level mixer RF: 3900MHz, 20dBm LO: 3939MHz, 20dBm IF : 39MHz (propose), single ended or for prototyping... Features : - Needs more than +20dBm LO Power - Required space more than VME sized (Stripline design, directional coupler...) - Amplitude/Phase noise <0.01%, <0.01 deg - Easy servicing and investigation of LO and IF - Individual Power Supplies Frank Ludwig, DESY XFEL-LLRF-ATCA Meeting, 3-4 December 2007 9

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Automated Drift Calibration A drift compensation is needed to compensate drifts from - Cavity pickup cables (4 module) -1-1 -1 5 fs m K, ± 125fs K ( ± 25m), ΔT - Downconverter (mixer) θ A = 2e-3/ C, θ P = 0.2 / C - LO generation (dividers, amplifiers, filters) (Injector) - ADC CLK generation (timing system, less critical) to have a robust machine operation. 1K 1) Injection of the reference signal : 2) Reflection at the cavity : Calibration Line RF Reference In discussion Cavity Flattop Beam pause t Receiver ADC Receiver ADC LO LO ADC + Diagnostic + Compensates connectors Receiver Channel LO LO ADC Frank Ludwig, DESY XFEL-LLRF-ATCA Meeting, 3-4 December 2007 10

X-Ray Laser Project XFEL X-Ray Free-Electron Laser Problems to be solved n Distribution within crates : - Interference from high-speed digital circuits - RTM ATCA AMC diff. / single ended interfacing? ADC, LPF ADC ADC Lowpass filtering of distortions CM ADC CM Frank Ludwig, DESY XFEL-LLRF-ATCA Meeting, 3-4 December 2007 11

MO and Distribution LLRF Part III, KEK Seminar, March 14, 2008

XFEL Need for Precise Synchronization Master Oscillator ~ Timing Distribution System Phase Reference Signals Timing Signals Injector Klystron LLRF Cavities RF Station Klystron LLRF Cavities RF Station Klystron LLRF Cavities RF Station Electronic devices should be synchronized with high accuracy Required jitter for phase reference signals: 0.1 ps short term (10 fs at some locations in XFEL) 1 ps long term Krzysztof Czuba, ISE LLRF reviev, DESY, 3.12.2007,

XFEL Global and Local Distribution Master Oscillator ~ Distribution System Phase Reference and Timing Signals LLRF Rack(s) LLRF Rack(s) LLRF Rack(s) LLRF Rack(s) RF Station RF Station RF Station RF Station LLRF Rack Distribution within racks ATCA ATCA ATCA R T M ATCA Crate AMC AMC AMC Distribution within crates, between PCBs Local signal generation Krzysztof Czuba, ISE LLRF reviev, DESY, 3.12.2007,

XFEL Problems Global: choice of distribution frequency Distribution within racks: choice of cables and connectors Distribution within crates: Stable distribution in multilayer PCBs and dense connectors Distribution between PCBs, problems with terminations when PCB removed Interference from high-speed digital circuits Local signal generation Building a circuit generating required signals from the distribution frequency with satisfying accuracy Krzysztof Czuba, ISE LLRF reviev, DESY, 3.12.2007,

XFEL AMC Clock Synthesizer and Timing Module Objectives: Synthesize ADC clock signals from the MO signal Receive signals from the FLASH timing system and provide trigger signals for FPGAs Module will consist of two PCBs: Analogue part (bottom): receiver and synhesizer. Designed by M. Ladno and K. Czuba Digital part: control and interface for the analog part. Universal module (top) with FPGA designed by D. Makowski Krzysztof Czuba, ISE LLRF reviev, DESY, 3.12.2007,

Requirements for the AMC Timing Module XFEL Clock synthesizer requirements: Input signal: 1.3 GHz from MO Output clock frequencies: 10 MHz 100 MHz, 0.1MHz step Clock stability <5 ps, (desirable < 2ps) 3 independent clock outputs (LVDS bus) Timing Receiver requirements: Receive and decode timing signals from the existing FLASH timing Optical input (at front panel) 3 independent trigger outputs (LVDS) Mode with self trigger generation (trig freq. 0.1 33 Hz) Krzysztof Czuba, ISE LLRF reviev, DESY, 3.12.2007,

XFEL Block Diagram : 8 : 8 Timing Timing Signal Signal 1,3 GHz MO signal 1,3 GHz MO signal 162,5MHz 162,5MH z OPTO OPTO REC REC PLL3 PLL3 PLL2 PLL2 PLL Diagnostics (temperature, power levels) Control Control Connector for the Top Module Connector for the Top Module +12V +12V 3.3V, 3.3V, 5V, 5V, -5V -5V DC\DC DC\DC 10-100 10 -MHZ 100 LVDS MHZ LVDS A M C 3 3 x x Trigger A LVDS LVDS + A M C A + Krzysztof Czuba, ISE LLRF reviev, DESY, 3.12.2007,

Software Architecture LLRF Part III, KEK Seminar, March 14, 2008

XFEL Overview of the distributed system klystron cryomodule cryomodule cryomodule cryomodule 24 channels 3x 8 channel board 3x 8 channel board 3x 8 channel board 3x 8 channel board DAC board Computation board multiple FPGAs multiple DSPs embedded systems surrounding devices Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 5

XFEL Requirements The software architecture must fit to the distributed and redundant hardware platform. The scheme must integrate and interact with all possible applications to allow them to fulfill their requirements. Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 6

XFEL Possible algorithm locations 50 ms 200 ns 5 ns Calculation Clusters Embedded system DSP FPGA Remote CPUs CPUs on site High Level Applications Low Level Applications Controller Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 7

Definitions of different software groups Controller XFEL The controller is the element of the system which has direct influence on the RF station. The main task of this application is to provide RF power to the cavities in a controlled way. During normal operation it controls vector sum of the cavities with a given precision. To achieve that, its reaction time must be as short as possible. Moreover, it provides interfaces for other software components giving them possibility to change the RF field parameters. Low Level Applications This is a set of applications designed mostly to execute algorithms and measurements between pulses. Parameters calculated during that time can be uploaded to the controller and used in the next pulse. These applications may also function as support to the controller during on-line operation. In this case, only DSP processor can be considered other elements of the system cannot meet timing constraints. High Level Applications This is a set of applications designed mostly to run in the background of working system in long execution intervals. Parameters calculated by these algorithms are uploaded to the controller every few pulses. The main execution platforms for these applications are control servers and computation cluster. Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 8

XFEL Controller Cavity signals Computation Pipe of the controller Field detection module Feedback module Output module - detection of the components of the field - I-Q - Amplitude, Phase - parametrized control function - P-I-D - MIMO - output corrections - klystron chain linearization - beam load compensation - etc. To Vector Modulator Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 9

XFEL Piezo Control The controller is used for interaction with RF part of the RF station. The piezo sensors and actuators are handled by piezo controller. It executes algorithms and measurements related to the detuning and microphonics compensation. The piezo controller implemented on the separate hardware platform with high analogue channel count interacts with the controller, low and high level applications, using dedicated interfaces. Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 10

XFEL Low Level Applications This includes: - Adaptive Feed-Forward - System Identification - Loop gain and loop phase calculation - detuning and loaded Q calculation - Vector sum calibration - Beam Diagnostic - Exception Detection and Handling Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 11

XFEL High Level Applications This includes: - Adaptive Feed-Forward - Vector sum calibration - Beam Diagnostic - Automated frequency tuning - Exception Detection and Handling - RF-Gun control Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 12

XFEL Applications summary The functionality of the low and high level applications overlaps. So how can we know where to put given algorithm? It depends on our timing constraints: if we want to execute given algorithm as fast as possible (during a pulse or just between) it should be implemented as low level if we want to execute given algorithm with extended precision and functionality we implement it as high level Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 13

XFEL Software layout piezo RF station RF Piezo control Controller High Level Applications Low Level Applications Diagnostics Automation Control system Wojciech Jalmuzna, Technical University of Lodz, Department of Microelectronics and Computer Science XFEL-LLRF-ATCA Meeting, 3-4 December 2007 19

Controller I (Warsaw) LLRF Part III, KEK Seminar, March 14, 2008

XFEL Controller definition Simple & low latency automat stabilizing the amplitude & phase of electrical field during the single pulse managing DOOCS C o n t r o l a l g o r i t h m s a r r a n g e m e n t e x e c u t i o n LOW & HIGH CONTROLLER LEVEL APLICATIONTS field control RF SYSTEM TASKS: Execution of control algorithm basing on Control Data provided by Low Level Applications and on results of measurement of the Input signals. Providing of processed measurement data to the Low Level Applications layer Monitoring and Exception Handling for safety requirements Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007

Requirements for the controller Basic technical requirements Multiple input channels Low latency (as low latency as possible, however it depends on IF and technological limits) Modular, parametrized and reconfigurable structure Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007 XFEL Modularity - It should be possible to distribute the design between a few AMC boards, and even between a few ATCA carrier boards (however it will impose some additional latency) Parametrized the number of memory blocks, of DSPs used, of input channels serviced may be changed without significant redesign Reconfigurable the general structure of the controller will be stable, even if some blocks are moved from FPGA to DSP. Design methodology related requirements Controller is supposed to be a complex system, to assure high maintainability e.g. to avoid human errors the automatic implementation methods (DSP on FPGA, DOOCS integration) must be developed Full testability is needed (in simulation, with real hardware, with hardware-software cosimulation)

XFEL Reliability requirements of the XFEL controller Required: Continuous operation, one maintenance day per month ATCA provided functions for increased availability: Redundant power supply Full mesh topology no global bus for boards' interconnection, which could be blocked by a damaged board Controller design features contributing to increased availability Algorithm able to operate even when some analog inputs are failed Use of redundancy in the analog input signals Use of feed-forward alone as the last resort solution Possible redundant implementation of the controller Cavity gradient monitoring with exception handling Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007

Functional requirements for the controller XFEL Channel calibration Rotation matrix Vector sum Error calculation Feed forward Generation of output signals with klystron linearization Provision of measured data for: Low Level Applications layer for updating of Control Data for the Controller For monitoring (DOOCS) and diagnostics (Exception Handling) Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007

Functional Concept of Controller XFEL LOW LEVEL APPLICATIONS Simulation Coefficients Set-Point Gain Feed-Forward ADC I/Q Vector DETECTOR CALIBRATOR Error CORRECTOR Feedback + Sum FPGA + C O N T R O L L E R out DAC D a t a A c q u i s i t i o n M e m o r y Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007

XFEL Multiboard implementation of the controller Single board implementation impossible: Main problem multiple A/D converters (96 channels needed) reasonable limit: 8 or 12 channels/board It is necessary to transmit the converted signals between the boards If possible, it will be preferrable to split the algorithm between the boards Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007

XFEL Proposal of distribution of the LLRF system (LLRF Team, presented by T.Jezynski and W.Jalmuzna) Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007

Structure of the ATCA LLRF controler XFEL Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic Systems ATCA LLRF meeting, 3.12.2007