Supercam Spectrometer Synchronization at the SMT 7 February 2007 Craig Kulesa Summary of basic needs: 1. External (hardware) synchronization: We will need to monitor or set numerous TTLlevel digital signals for real time synchronization of the spectrometer where timing is critical. These signals are routed throughout the SMT on a 32-bit digital bus (called the SAMBUS) and connected to the Omnisys spectrometer with standard coaxial cable and BNC connectors. 2. Internal (software) synchronization: For receiver tuning and non-realtime spectrometer needs, we will send commands via 100 Mbit ethernet with a TCP/IP socket interface (e.g. telnet to a network port). This mirrors the current interface for the prototype spectrometer unit, where integration cycles are under software control. Background: All observations made from the ground at submillimeter wavelengths are differential measurements made between signal and reference positions. There are multiple ways of achieving this naturally, each technique has advantages and disadvantages. Thus, Supercam will ultimately have to support all of them. The four principal observing modes of the spectrometer are: 1. Beam switching: the subreflector of the telescope nods back and forth at a full-cycle frequency of 1 to 4 Hz, moving the field of view between signal and reference positions. Thus, the spectrometer must be read out in sync with the subreflector at 2 to 8 Hz. 2. Position Switching: the entire telescope is moved between signal and reference positions; the subreflector remains stationary. Integration times are 10-30 seconds. 3. On-The-Fly (OTF) Mapping: A variation on position switching, except that the signal position is not a single position the telescope drifts over a large region of sky while the spectrometer is read out at a high rate (~10 Hz). After drifting for 30-60 seconds, a single reference position is observed for ~20 seconds. 4. Receiver Characterization and Calibration: Single integrations of 1-10 seconds are occasionally made of known (local) objects to establish an intensity (brightness) scale for the spectra and measure the sensitivity of the receiver. The read (R) and write (W) TTL-level signals (LOW=0V, HIGH=5V) that are used for real-time control of the spectrometer are as follows: 1. BLANKING (R): goes LOW when the telescope's subreflector is moving and data should not be taken. Otherwise HIGH. This signal is inverted in the control room for the AOS's. (Beam switching only) 2. SYNC (R): Defines whether we are in the SIGNAL or REFERENCE beams. Very inconsistently defined at the SMT. (Beam-switching only)
3. OFF-SOURCE (R): LOW when we are properly tracking the source, HIGH when we are off-source and data should not be taken. (All modes) 4. OTF (W): On-The-Fly bit must be set HIGH by the spectrometer, or some computer synchronized with it, to tell the telescope control system to dump positional data to the data-taking process. (OTF mode only) 5. RECEIVER LOCK (R): This should always be LOW. If it goes HIGH, we must stop taking data and complain! (all modes) 6. FOCUS (R): This should always be LOW. If it goes HIGH, we must stop taking data and complain! (All modes) 7. TRACKER ACKNOWLEDGE (R): After a command to tracker (the process controlling the motion of the telescope), it will acknowledge the command and send this bit HIGH to indicate that the command is being processed, the telescope is about to move, etc. It will be reset once the telescope is on source and the OFF-SOURCE flag drops to LOW. We can use this signal to hint that the telescope is just about ready to observe. Ideally, we would start the spectrometers integrating but not saving data so that the very instant this ACK bit goes LOW, we are already in full swing. Hardware Interface: The figure below describes one possible interface between the Omnisys spectrometer, the SMT and our (to be purchased) data server PC. The aim is for the data server PC to not only accumulate and process the raw spectrometer data files into CLASS and FITS files, but also to insulate the two ARM-based microcontroller boards in the spectrometer chassis from the details of the SMT observing sequence. Thus, the microcontrollers should only get basic start and stop requests. There are seven digital I/O bits to carry between the SAMBUS and the Omnisys spectrometer. We can reduce this to five (4 READ, 1 WRITE) by employing a logical OR operator on three sanity check bits the OFF-SOURCE bit, the FOCUS bit, and the RECEIVER LOCK bit. It may also be possible to include (an inverted) BLANKING in this OR operation. The resultant digital bit amounts to a generalized INVALID bit HIGH if something's bad, LOW if we should be busy integrating.
Figure 1: Hardware Installation Configuration Data Bandwidth and Storage: The 32 power combined inputs of the Omnisys spectrometer will serve 1024 channels of 32- bit depth over 550 MHz of total bandwidth. Thus, each of the 64 Supercam beams will dump 2KB of data per scan, or 128 KB for the entire spectrometer. At the 10 Hz rate of OTF scanning, this represents a maximum data bandwidth of 1.3 MB/s or <20 Mbit/s with TCP/IP packet overhead. Given that we are specifying two dedicated 100 Mbit/s ethernet links to the data server PC, this allows for <10% bandwidth utilization. One week's worth of OTF maps at 60% duty cycle is 500 GB of raw spectrometer data. This is readily managed with a modest redundant disk array. We are currently specifying an array of 4 x 500 GB SATA drives in a RAID-5 configuration, for a total of 2 TB of storage. Data Taking Sequence: Common initialization At the SMT, the backend spectrometers are essentially in charge of the entire data taking process, the telescope timing, everything. We can integrate our spectrometer control into the SMT system, or declare independence as a separate backend controller. For the latter, we
will have considerably more work to do. However, this will eventually be necessary since Supercam will travel to more than one telescope! Based on the SMT control system source code, for all data modes, the following sequence will take place. #2 and #3 can be ignored if we use some of the existing services in SMTCON. But if we do, we will have to funnel the obtained spectra also through those same network services! 1. The observer will set up their observation sequence and click start. 2. The SMT Executive will send an observation header telling us what has been requested. But that's all it will do. It is up to us to parse this header and determine what needs to be done make sure the observations are scheduled correctly and that all subsystems are started. If beam-switching is requested, now is when we start the subreflector moving... etc. 3. We will now tell TRACKER to move the telescope to the reference position to start the sequence. 4. TRACKER will send the acknowledge bit HIGH. The spectrometer is commanded to begin an observation subject to the digital I/O bit states. The bit states will tell us that we're NOT READY YET. If beam-switching or OTF mapping, the spectrometer should (ideally) start the integration cycles NOW, but not save any spectra to disk until we are actually on-source. 5. Once the telescope is on source the OFF-SOURCE (INVALID) bit goes LOW and we start immediately saving data. A brief guide with pseudo-timing-diagrams for each mode follows: Beam-switching: First, a software command will prompt the spectrometer to begin integrating subject to the state of the BLANKING and INVALID (RECEIVER LOCK FOCUS OFF SOURCE) signals. To start integration, we want BLANKING to be HIGH and INVALID to be LOW. Any deviation from this will stop the integration and cause readout. For BLANKING, LOW lasts for 30-40 milliseconds before it goes HIGH and the next integration can begin. The SYNC signal is read and passed along to the data archiving process to identify whether it is a signal or reference spectrum. Sadly, the polarity of SYNC is ill-defined at the SMT, and we will likely have to change it in software upon installation (in some parts of the building, HIGH=signal... in others, HIGH=reference). See the figure below for details.
Figure 2: Beam Switched Data Sequence Position-switching: Here, the subreflector is fixed and the telescope itself slews between SIGNAL and REFERENCE scans. We will look at the OFF-SOURCE (INVALID) bit to synchronize; it is LOW when the telescope is properly tracking the source. We stop when the INVALID bit goes HIGH or when an integration timer expires. Here, the integration times are long (at least 10 seconds), so timing is not critical we only need to synchronize when to start the integration.
Figure 3: Position Switched Data Sequence On-the-Fly Mapping: In OTF mode, we alternate between long (~20 sec) integrations at a REFERENCE position and rapidly dumping spectrometer data at ~10 Hz during the SIGNAL period... as the telescope is drifting across the sky. We will use the OFF-SOURCE (INVALID) bit for long integrations at the reference position just like Position Switching. Once the telescope is commanded to move to the mapping source, we SET the OTF bit HIGH to tell the telescope to generate positional data at a high frequency (~20 Hz) so that each spectrum can be stamped with an accurate time and a telescope position for subsequent data processing. At the same time, a software signal will be sent to the spectrometer to signal to start reading out at 10 Hz but until INVALID goes LOW, data should not be stored. Alternately, we simply start the 10 Hz dumps as soon as INVALID goes LOW. The spectrometer microcontrollers must provide the timestamp for the 10 Hz data dumps so that they can be correlated to the positional data that the telescope generates. A figure showing the OTF data sequence is shown below.
Figure 2: On-The-Fly Mapping Sequence Receiver Calibration and Characterization: These measurements need be performed even in the absence of the telescope; they are useful for laboratory measurements too! Here, we need to software-control the spectrometer to perform basic integrations of a specified duration, typically less than 10 seconds. No external synchronization is needed this will be performed by software that is upstream of the spectrometer. This is similar to the interface provided by the prototype spectrometer, where all synchronization was set in software.