J-PARC timing system N.Kamikubota, N.Kikuzawa J-PARC / KEK and JAEA 2015.10.17 at timing workshop
Facts in short 1. J-PARC is an accelerator complex located in Ibaraki, Japan 1. Rapid cycle: LI(400MeV Linac) and RCS(3GeV) - 25Hz 2. Slow cycle: MR(30GeV Main Ring) - 2.48s or 6.00s (5.52s after Oct.2015) 2. Hardware 1. Home-design VME modules for control, NIM modules for signal generation (Sorry, not MRF-based yet!) 3. Software 1. Developed by ourselfves 2. EPICS and its tools are used in general 3. Java and python are preferred for table-data handling (epics waveform) 4. Scale of the system 1. One Send-module (=EVG) 2. LI/RCS/MR 118/43/45 VME receiver-modules(evr), ~540/220/300 endpoints
VME modules developed 1. In collaboration with a company, Hitachi-zosen ( 日立造船 ) 1. Design in 2002-2004 2. Production in 2005-2007 => J-PARC 1 st beam was in Dec.2006 to the Linac 2. Send module [Send. <-> EVG] 1. Type-code : 3 x 8bit (+ optional 7bit) [type<->event] Each 8bit is called M2, M3, M4 2. Has a 64 sequences (max 256) of type-codes Send one of 64 sequences at a time 3. Receiver module [. <-> EVR] 1. Produces 8 independent delay set-points 1. dt (resolution) = 10.41ns (1/96MHz) 2. One 8-bit of 3x8bit is used M2=Linac(25/50Hz), M3=RCS ring(25hz), M4=MR(2.48s/ 6.00s)
Send module ; type-sequence and type-code A send module: - has a type-memory, which consists of 64 type-sequences - uses one selected type-sequence at a moment - sends the type-code in the selected type-sequence one-bye-one at the 25Hz trigger rate One type-sequence: - can contain 256 type-codes at maximum - corresponds to one machine-cycle; 62(150) type-codes for 2.48s(6.00s) One type-code: - has - M1(MSB+ spare 7bit), M2(8bit for LI), M3(8bit for RCS), and M4(8bit for MR) - MSB is used to notify the end of the type-sequence TYPE- SEQ. S63 Example of type-code) The 20 th type-code in the type-seq. S63 is 00-66-66-46 In which 66 for LI, 66 for RCS, 46 for MR A Java application GUI for a send-module control During MR injection (red area), LI/RCS behave differently according to received type-codes
Beam slots in single machine-cycle 0ms 800ms 960ms 840 880 920 2480ms = to MLF = to MR = (Empty) machine-cycle 2.48s USER-RUN: Beam to MLF+MR(NU) 0 1 19 20 21 22 23 24 25 26 27 60 61 4 slots are used for MR injection 25Hz beams to MLF interrupted 6000ms machine-cycle 6.00s USER-RUN: Beam to MLF+MR(HD) 0 1 19 20 21 22 23 24 25 26 27 148 149 2 empty slots to avoid miss-steering by the residual magnetic field of a switching magnet machine-cycle 2.48s Stand-by No Beam 0 1 19 20 21 22 23 24 25 26 27 60 61 http://j-parc.jp/researcher/acc/bi/totalstatus.html
Base-Signal Distribution and VME/NIM Modules 1 Send Fiber-optic cable network: single-mode cables (3 cores used) 3-4 us distance between buildings ~15 MR-D3 (MR 3rd build.) CCB MR-D2 (MR 2nd build.) (central control building) 43 RCS (RCS build.) LI ~15 MR-D1 (MR 1st build.) End-point example at MR-D3: UP 4 NIM modules 1 gate module (4 signals) 3 trigger modules (8 signals each) DOWN 4 VME. Modules each module controls one NIM module ~15 118 (Linac build.) Send module located in the CCB: is a VME module, only one exists 2 Base-signals (CLK, Trig) come in generates series of Type-codes Typecode E/O 25Hz Send Module Trig. Gate/Trig modules (NIM) for signal generation. Modules (VME) for control 3 base-signals: CLK, Trig, Type CLK 12MHz Base-signal distribution example: E/O, O/E, and fanout NIM modules O/E, E/O Fiber cables Fanout patch panel NIM modules 3 base-signals come into VMEs
Schematic view: Base-signals, VME, and NIM CLK 12MHz Trig. 25Hz VME Send Module Typecode NIM E/O Fiber-optic cables O/E NIM F.Out VME VME CLK (nim) Trig (nim) Type-code (lvds) base-signals are fed into a. 3 base-signals: CLK, Trig, Type CPU. CPU. 3 base-signals are broadcasted using fiber-optic cables NIM T R I G 1 2 7 8 NIM G A T E 1 2 3 4 a receiver module is for control, and a NIM module to generate signals three base-signals are necessary for a receiver module four different NIM modules: 8ch trigger (TTL,NIM), 4ch gate (TTL), 8ch trigger (opt), 4ch gate (opt)
Ideas for the future Send Send FPGA SFP E/O FPGA SFP O/E Fiber-optic cables Possible Future #1 CLK Trig Type-code CLK Trig Type-code F.Out the opt-device (Finisar v23826) has been used, but discontinued now Now, SFP and FPGA are standard technical choice VME CPU. NIM T R I G VME CPU 1 2 7 8 CLK(nim). Trig(nim) Type-code(lvds) NIM G A T E 1 2 3 4 Proto. Converter J-PARC to MRF MRF colony SFP Fan-out CPU MRF EVR MRF TCA, cpci, etc EVR 1 2 Possible Future #2 develop a JPARC=>MRF protocol converter MRF colony(s) will appear in J- PARC?
High- Inten. Proton Summary 1. J-PARC Timing hardware was developed roughly 10 years before, in collaboration with domestic companies 2. We start discussion on possible migration and extension for the next decade 1. Introduce SFP as a new opt-deice standard 2. Introduce MRF colonies? ->WEPG122 comments are welcome (or help us)
backup
J-PARC timing: History Overview 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 LI Beam start (2006.12) RCS Beam start (2007.10) MR Beam start (2008.05) Earthquake (2011.03.11) HD accident (2013.05.23) VME modules : design, develop (ftamura, yoshii, chiba + nichizou) NIM modules : design, develop (ftamura + repic) For LI and RCS: 25Hz (50Hz in design) For MR: 2.48s (for NU) 6.00s (for HD)* * 5.52s after RUN64 (Oct.2015) Studies, followed by Implementations for LI (sakaki, kiyomichi, terashima) Implementations for RCS (takahashi, kawase) Operation and Maintenance for LI/RCS (kikuzawa, kawase, maruta, futatsu) Studies for slow machine cycle (initially 3.64s) (matumot, kami) Implementations for MR (matumot, kami) Operation and Maintenance for MR (kami) Study for the next generation system (kikuzawa, kami)
Receiver module ; LUT and delay-word A Python application GUI for one receiver module in MR A receiver module: - must select one of four subsections; ex) M2 for LI, M4 for MR - has a Look-up table (LUT), which contains 256x8ch delay-words A delay-word: - contains 24bit delay value and control bits - has a MSB(31 st ), which is used to disable output - has a special control-bit(30 th ), to continue delay counting without reset The 24bit delay count: - runs using 96MHz clock after receiving the 25Hz trigger, where 96MHz clock is generated from 12MHz master (CLK) by PLL - is reset when next 25Hz trigger arrives, however.. - can generate >40ms delay using the special control-bit above Spec of a receiver module: - minimum time resolution is 10.41ns (96MHz) - maximum delay is ~170ms (24bit) Example of LUT in MR) - The receiver module is set to use M4 - Most of the elements in LUT here indicates disabled ; i.e. no output - The delay word 96000 here means: - generate a delayed pulse at ch5, when the received type(m4) is 6 delay after the 25Hz trigger will be 96000 x 10.41ns - the special control-bit(30 th ) is set for MR, but not shown Part of LUT
Transitions of machine-cycles Study, tuning User-RUN (beam delibery) Start by an operator Start by an operator MLF+MR(NU), 2.48s MLF only, 2.48s MR(NU) only, 2.48s Stand-by with different parameters (LI macro-pulse etc.) exist behind Stand-by Stand-by (no (no beam), beam), 2.48s 2.48s Stand-by (no beam), 2.48s MLF 1 shot, 2.48s MR 1 shot, 2.48s Stop automatic Fault (MPS) Stop by an operator Stop automatic Change of cycle by an operator MLF+MR(HD) 6.00s MLF only 6.00s MR(HD) only 6.00s Stand-by, 6.00s Switching a type-sequence to another in the send module results in a machine-cycle transition Variety of transitions is possible using 64 type-sequences: - ex1) single shot to MR during user-run for MLF - ex2) all combinations of start and stop to MLF and MR Automatic transition to the stand-by - when a MPS event (machine fault) occurs - when 1-shot is pre-specified
J-PARC timing : marching band Our timing system is like a marching band: - three rows; corresponds to LI, RCS, MR (sorry, only two are shown!) - a music master behind (a send module) conducts at 25Hz beat - each parade member has a number flag, to bring the type-code to a receiver
misc. information Synchronization timing Extraction kicker of RCS must be synchronized with the circulating beams. This trigger is generated by the RCS RF. MR injection devices are triggered by the same signal with appropriate delays. This delay is generated by a dedicated VME board, with resolution of 2ns. Master oscillator The master oscillator is a commercial product, a high-stability function generator. It generates the master clock: 12.000,000,000 MHz. We always keep a stand-by together, since the master oscillator is indispensable. Daily modulation The length of the fiber-optic cables between buildings is about 1 km. One-way path-through time is 3-4 us. Daily modulation due to environmental changes was observed: roughly ~1 ns. This modulation is considered small enough and permissible for our timing. A stand-by. MOTTAINAI?
Evaluation 1. Since 2006, VME modules (~200 modules in total) have worked very well without faults, except a few pieces 2. Problems and considerations 1. Base signals, especially type-codes as a LVDS form, suffer external noise influences from pulsed power-supplies. 2. Life of optic devices : optic devices used in E/O and O/E modules, made in 2006-2010, are already discontinued 3. ADS, a new facility in J-PARC, will be constructed. Additional 25Hz beams will be needed around 2018-2019. 4. No good proposal for small component: when only one delay is necessary for new component, set of a VME system and NIM modules is indispensable, results in too much space.
Ideas for the future 1. Introduce SFP and FPGA SFP is a safe optic device for future long availability merge/divide 3 base-signals into/from a fiber cable using FPGA technology E/O Fiber-optic cables O/E CLK Trig Type-code FPGA SFP Fiber-optic cables FPGA SFP CLK Trig Type-code 2. Introduce MRF products O/E J-PARC Timing System F.Out VME CPU NIM. T R I G VME CPU 1 2 7 8 CLK(nim). NIM Trig(nim) Type-code(lvds) G A T E 1 2 3 4 Proto. Converter J-PARC to MRF SFP Fan-out CPU Extension to MRF system TCA, cpci, etc (small-factor platform) MRF EVR MRF EVR 1 2 MRF provides various timing modules for accelerators they have been used especially in EPICS community Small-factor platform would solve space problem optic link to a front-end module (EVR) would make good effects against external noise Develop a protocol converter from J-PARC to MRF Extend the J-PARC timing system to use MRF products
Ideas for the future 3. 50Hz time slot for ADS machine-cycle 2.48s Beam to ADS+MLF+MR(NU) LI=50Hz, RCS=25Hz, ADS=25Hz, MR=2.48s 0ms 800ms 960ms 840 880 920 2480ms 50Hz beam operation was in the original design of the Linac hence, beam slots to ADS can be assigned as above Timing modules were also possible to use at the 50Hz rate = to MLF(RCS) = to MR = to ADS = (Empty) E/O Fiber-optic cables O/E CLK Trig Type-code FPGA SFP Fiber-optic cables FPGA SFP CLK Trig Type-code