Introduction to QScan Shourov K. Chatterji SciMon Camp LIGO Livingston Observatory 2006 August 18
QScan web page Much of this talk is taken from the QScan web page http://www.ligo.caltech.edu/~shourov/q/qscan/ Overview Example scan Interpretation of scans Quick start guide Configuration details Source code Installation and testing Existing installations
QScan overview QScan is a tool to perform a detailed study of a very large number of auxiliary and environmental channels around a specific time of interest Detector glitch Hardware injection Candidate gravitational-wave event It currently provides support for a number of activities Commissioning Detector characterization Veto search Astrophysical searches
QScan overview (cont.) QScan is based upon the Q Pipeline burst search algorithm Multiresolution time-frequency search for statistically significant excess signal energy Projects whitened data onto the space (time, frequency, and Q) of Gaussian windowed complex exponentials Equivalent to a matched filter search for waveforms that are sinusoidal Gaussians after whitening Significance is characterized by normalized energy Z 2Z is equivalent to the squared signal to noise ratio of a matched filter search Details provided in http://ligo.mit.edu/~shourov/thesis/
QScan overview (cont.) The search space is covered by Logarithmically spaced Q planes Logarithmically spaced frequency rows Linearly spaced tiles in time
Example Q transform 20% loss, Q of 4 1% loss, Q of 4 poor match 20% loss, Q of 8 1% loss, Q of 8 best match
Example QScan QScan produces web based reports that include event time in GPS, UTC, PST/PDT, CST/CDT science and injection mode segments data quality segments links to detector logs for day of event and next day Index of channels by subsystem channel names link to channel wiki time series and time-frequency spectra quantitative summary file for automated followups log file for containing debug information space for additional information Example QScan of a inspiral hardware injection
Interpreting QScans Channel name links to ChannelWiki Display time series, spectrograms, or eventgrams Most significant tile properties in search window
Interpretation (cont). Only statistically significant channels are displayed Significance level defined in configuration file Channels tested for significance in a finite time window Time window duration defined in configuration file Typically on the order of one second Accurate timing is required before running a QScan Absence of a channel does not imply no significant content Presence of a channel does not imply correlation with signal in the gravitational-wave channel QScans at random times can give some guidance QScan is not a substitute for a continuous search
Interpretation (cont.) Multiple image products are produced Time series (raw, high pass filtered, whitened) Spectrograms (raw, whitened, autoscaled) Eventgrams (raw, whitened, autoscaled) Each image type has its advantages and disadvantages It is important to use all types to correctly interpret scans
Time series images Raw, high pass filtered, and whitened time series Useful for diagnosing saturation, bit level noise, or otherwise broken channels. Raw data may be downsampled Downsampling in reduced data sets Downsampling in QScan May obscure otherwise obvious saturations High pass filter frequency specified by configuration file Whitening performed by zero-phase linear prediction Parameters determined from configuration file May introduce artifacts (particularly echoes)
Time series examples Inspiral and pulsar hardware injections before and after whitening Bit level noise in broken channel Saturating channel
Spectrogram images Spectrograms are produced at constant Q All tiles have same bandwidth to central frequency ratio QScan displays the Q plane with the most significant tile This choice of Q may obscure features that best match other values of Q Spectrograms are produced with and without zero phase linear predictive whitening Linear predictive whitening may produce artifacts Even without linear predictive whitening, QScan normalizes by the mean energy in each frequency row Spectrograms are produced with both fixed and autoscaled colormaps to see both very significant structure as well as weaker but perhaps still siginificant nearby features
Spectrogram examples raw whitened autoscaled
Eventgram images Eventgrams display information from all Q planes Identify significant tiles by thresholding on significance Project all tiles onto the same time-frequency plane Remove the less significant of overlapping tiles Produces a mosaic like tiling of the signal Gives Q independent picture of signal But it is more difficult to identify subtle structure Similar to spectrograms Eventgrams are produced with and without zero-phase linear predictive whitening Eventgrams are produced with both fixed and autoscaled colormaps
Eventgram examples raw whitened autoscaled
Different time scales Specified by configuration file Identify small time-scale features near time of event Larger time scales provide context information How isolated is the glitch? How significant is the glitch relative to nearby structure? Is signal coincident with the gravitational-wave channel or is it glitching all the time? QScan provides guidance for more extensive veto studies Note that the finite resolution of images limits the ability to resolve short transients in long time scale images
Quick start The quickest way to get started is to use the existing QScan installation and default configuration at one of the LIGO Laboratory computing clusters: grid-proxy-init ssh ldas-pcdev1.ligo-wa.caltech.edu ~qonline/qscan/bin/qscan.sh 816335770.0 & The QScan launch script can also be given a custom configuration file, frame cache file, and output directory: qscan.sh gpstime <configuration> \ <framecache> <outputpath>
Configuration Automatic configuration utility qconfigure.sh Uses FrChannels to scan an example frame file Makes educated guess at configuration parameters based on channel name and sample frequency End users can edit the resulting congifuration files A large number of standard configuration files (@gw, @H0H1H2-RDS_R_L1-selected, etc.) are provided Gravitational-wave channels only Level one reduced data set Raw data set Time-domain calibrated data Seismic channels only Different detector networks etc.
Configuration example [Gravitational,Gravitational wave data] { channelname: L1:LSC-DARM_ERR frametype: RDS_R_L1C samplefrequency: 4096 searchtimerange: 64 searchfrequencyrange: [32 Inf] searchqrange: [4 64] searchmaximumenergyloss: 0.2 whitenoisefalserate: 1e0 searchwindowduration: 0.5 plottimeranges: [1 4 16] plotfrequencyrange: [] plotmaximumenergyloss: 0.2 plotnormalizedenergyrange: [0 25.5] }
Data discovery Data is located using framecache files Framecache files produced by createframecache.pl createframecache.pl framecache.txt \ framefiledirectory Predefined framecache files are provided for commonly used data sets (@gw, @S5-RDS_R_L1, etc.) LHO and LLO scans for recent events automatically find raw frame data from /frames S5 framecache files are currently updated once per day Working on LSCdataFind or LDAS frame cache interface
Segments and data quality Science mode, injection mode, and data quality segment information is taken from daily dumps of the LSCsegFind database There is a ~1 day latency on this information Shows up as unknown if not yet available Data quality segment definitions also change over time The qupdate.sh script can update segment and data quality information without rerunning the entire scan qupdate.sh ~/public_html/qscans/ 816335770.0
QScan is written in Matlab Infrastructure Shares common code base with Q Pipeline Uses FrameL based access to frame files Uses Matlab compiler to produce a stand-alone executable that is free from license restrictions Currently uses Matlab version R13 Requires X server to produce figures Supports multiple platforms Fedora Core 3 and 4 Solaris 9 and 10 Installed on CIT, LHO, LLO, MIT, PSU, and UWM clusters Installed on CIT, LHO, LLO, and MIT general computing Can be run under condor on LSC computing clusters
S5 glitch search Current use Loudest single detector inspiral events Loudest single detector blocknormal events Loudest H1H2 kleinewelle events S5 burst search Produced for candidate WaveBurst events S5 inspiral search Incorporating into automated follow-up of candidates S5 operations and commissioning Available in LIGO control rooms In use by both operators and commissioning teams
Room for improvement QScan is computationally intensive Requires a while to run (~1 hour for raw data scan) Requires a lot of disk space (~? GB for raw data scan) Slow to load (~1 minute for raw data scan) Data discovering could be more automated Currently updating frame cache once per day Planned interface to LSCdataFind and LDAS disk cache Requires accurate knowledge of event time Implementing initial prescan to identify glitch time Provide graphical user interface to run QScans Provide calibrated output for channel coupling studies Provide easier access in control rooms Provide larger collection of specialized configurations