The CHIME Pathfinder and Correlator Matt Dobbs for the CHIME Collaboration
Intense Competitive Sports Atmosphere in BC Bridge tournament taking place this week at the Days Inn, Penticton. Matt.Dobbs@McGill.ca, DRAO 2011 2
Cosmic Sound Start: z=0.01 End: z=0.62 Simulation Video: Nick Gnedin & Dave McGinnis (Fermilab), Matt.Dobbs@McGill.ca, DRAO 2011 3
Pathfinder ADC ADC ADC Cartoon: 1 cylinder with 12 feeds summed into 3 digitizers. Pathfinder will have 64 feeds x 2 pol x 2 cylinders. Matt.Dobbs@McGill.ca, DRAO 2011 6
Single Dish vs. Interferometer vs. CRT ( 50K) 0.3 o ( 0.3 o ) 2 Size = (100m) 2 = 0.5m T Feed = 50K 100m 0.6 o ( 4 50K) ( 2 0.3 o ) 2 Matt.Dobbs@McGill.ca, DRAO 2011 7
Matt.Dobbs@McGill.ca, DRAO 2011 9
Aliasing & Fringe Stopping (for non-equatorial CRT) each lobe fringe-stops at a different rate. Matt.Dobbs@McGill.ca, DRAO 2011 10
tile Matt.Dobbs@McGill.ca, DRAO 2011 11
Matt.Dobbs@McGill.ca, DRAO 2011 12
Matt.Dobbs@McGill.ca, DRAO 2011 13
Analog Beam Forming Cannot adjust (G, ) after-the-fact. Initial calibration/stability essential. just one beam per digitizer Adjusting phase steers the beam 1 Digitizer per Feed FFT correlated: cannot adust (G, ) after-the-fact N 2 correlated: can calibrate offline Hybrid Trades FLOPS for ease-of-calibration (money for robustness) Large scales: cannot adust (G, ) after-the-fact Small scales, FFT correlated: dito. Small scales N 2 correlated: can calibrate offline. For fixed N digitizer (=cost) allows to trade survey area for angular resolution Time hydrid (not shown, applies to middle & bottom) Can N 2 correlate for a small fraction of time or frequency-bandwidth, and use this to correct (G, ) for FFT correlation. Matt.Dobbs@McGill.ca, DRAO 2011 14
->-------x Hybrid Can also N 2 correlate specific pairs, like each feed with a pseudorandom source. Provides time-stability calibration for gain, phase creating a static beams. Matt.Dobbs@McGill.ca, DRAO 2011 15
Beam Forming One Feed Beam pattern of one feed (Analog) Sum feeds to shape beam, increase sensitivity Equivalent to throwing away all but one digitally formed beam Cost saving anti-aliasing measure. Can choose deep (north) vs wide (equatorial). 8 Feeds Summed (Digital) beam forming, divides beams into several One Channel from 16 Digitizer Beam Former (This example, 128 feeds, 16 digitizers, w=16m cylinder) Plots From Dave McGinnis simulation code, Fermilab Matt.Dobbs@McGill.ca, DRAO 2011 16
McGinnis Sim Code, Fermilab (this sim: two 8x25m cylinders with 64 feeds integrating for 3 years, showing just one frequency slice) Matt.Dobbs@McGill.ca, DRAO 2011 18
Pathfinder Forecast Plot: Kiyoshi Masui Matt.Dobbs@McGill.ca, DRAO 2011 19
Baselines Uniform feed spacing: Unique feed spacing for each cylinder: 16 feeds across 40m Matt.Dobbs@McGill.ca, DRAO 2011 20
Tool Kit Angular resolution requirement determines collecting area: 100m x 100m for BAO at z=3 Cost drives # of digitizer channels. # feeds to sum into a single digitizer Trades off survey speed (wide) for cost Steering primary beam north (south) trades deep for wide. Can choose between FFT beamforming Full N 2 correlation Trades cost (compute power) for calibration robustness. Can do full N 2 for fraction of time or bandwidth. Matt.Dobbs@McGill.ca, DRAO 2011 21
CHIME Dataflow Matt.Dobbs@McGill.ca, DRAO 2011 26
Channelizer/Beamformer/Correlator Digitize Analog Signals Channelize (FFT in time) Assemble data from each frequency bin in one place FFT Beam-form along cylinder Correlate between cylinders Or do the full N 2, if cost permits. Matt.Dobbs@McGill.ca, DRAO 2011 30
Networking (similar to ASKAP implementation) Example using DRAO Kermode FPGA board, with ATCA backplane Data assembled in 3 hops Uses low-cost integrated FPGA transceivers Can be implemented with low-cost FPGA demo boards with PCIe backplane for CHIME Pathfinder Matt.Dobbs@McGill.ca, DRAO 2011 31
Digitizer/Channelizer/Beamformer Digitizer hardware being constructed now at McGill. Firmware under test for Channelizer, networking. Matt.Dobbs@McGill.ca, DRAO 2011 32
Cost of the full N 2 Correlation Full CHIME ~ 5 Cyl, 256 dual pol feeds per cyl 2560 Digitized Signals to Correlate. 2560 2 400 ch at 1 MHz 4 COMPLEX 400 MHz = 5243 TFLOP 2 John referred to this scenario simply as ouch yesterday. GPU Radeon 6990 (Available Yesterday, sitting in Siever s desktop today) ~5 TFLOP / $1K / 400W 2 FLOP = 1 multiply + 1 addition 2000 GPU boards, $2M cost, $0.7M/year electricity. Pathfinder ~ 2 Cyl, 64 dual pol feeds per cyl 256 2 2 4 COMPLEX 400 ch at 1 MHz 400 MHz = 52.4 TFLOP 20 GPU boards, $20K cost, $7K/year electricity. Matt.Dobbs@McGill.ca, DRAO 2011 33
CHIME Digitizer Boards (with J-F Cliche, Adam Gilbert) 8 Channels at 1.2 GSPS 24 GBPS IO Standard FMC interface, Mates to DRAO Kermode board Xilinx V6 Demo Boards Use Casper and inhouse firmware Matt.Dobbs@McGill.ca, DRAO 2011 34
Summary Cylinders are a flexible medium for instrument design especially if the Fourier modes of your signal are confined, as is the case for the BAO. CHIME Pathfinder: 2 cylinder instrument to demonstrate CRT concepts, with a good short at providing first Hydrogen Intensity mapping probe of BAO. Prototyping systems now. Matt.Dobbs@McGill.ca, DRAO 2011 35