QSched v0.96 Spring 2018) User Guide Pg 1 of 6 QSched v0.96 D. Levi Craft; Virgina G. Rovnyak; D. Rovnyak Overview Cite Installation Disclaimer Disclaimer QSched generates 1D NUS or 2D NUS schedules using computed quantiles of probability density functions. A 3D scheduler is in progress. The distribution is: qsched_096.py If you use QSched please cite: D. Levi Craft, Reilly E. Sonstrom, Virginia G. Rovnyak, David Rovnyak, Nonuniform sampling by quantiles, J. Magn. Reson. 288, 109-121 (2018). The python script is ready to run. Place in your path or other convenient location. The script operates correctly under a variety of cases. Experiments with 1D and 2D QSched schedules have yielded good results. It is entirely possible that the program, and this manual, may contain un-detected, even serious bugs or omissions. QSched is extremely flexible, and it is possible to create schedules that are not appropriate for your application. There is no substitute for visually inspecting the schedules you generate. Changes since v0.95: QSched should run on any python on any linux/unix/macos; QSched now one file; examples in the header of the python script have been tweaked; added comment on one-sizefits-all scheduling to this document; this document edited for length and clarity. FAQ 1. Is there a one-size-fits-all schedule? There are many ways to select and optimize schedules, but if you want an everyday schedule that can be used in a variety of settings, experiences suggests that using 25-33% reduction, a quant-sin or quant-poly weighting, x=1.5 and e=2.0 generally results in widely applicable schedules. 2. How do I get a 1D point spread function (PSF) from qsched_0.96.py? For 1D-NUS schedules only, qsched will automatically produce the PSF. 2D-NUS PSF s are not implemented. 3. How do I specify a random, unweighted probability density, e.g. for a constant time period? For 1D NUS, use the noweight option with x=0.3 to 0.4 jitter. For a 2D NUS schedule, set an unweighted dimension to be exponentially weighted with x = 1 and e = 0.1 since this will be jittered elsewhere in the algorithm. 4. What is a PDF? The Probability Density Function specifies when samples will be chosen more or less densely. A PDF can be similar (but not necessarily identical to) the signal envelope. 5. What is a PSF? The Point Spread Function is the Fourier Transform of a data set in which every sampled point is set to 1 and every skipped point is set to 0. This gives the sampling spectrum, which must be deconvolved from the actual spectrum in reconstruction. A high peak-to-sidelobe ratio, and low noise around the central peak are desirable. 6. What jitter level should I use for 2D NUS Schedules? We currently recommend 0.7, 0.75 or 0.8. Empirical tests support that any value between roughly 0.6-0.8 results in robust schedules that compromise randomness with adherence to the PDF. 7. Does weighted NUS broaden my line shapes? No, and yes. The line shape depends on T 2 * (informally T 2 in this manual). In many cases of NUS, line shapes may be indistinguishable from those obtained by FFT of uniform data. Sometimes NUS can cause slight or potentially significant broadening. 8. What is a harmonic tract? It is to be avoided. NUS schedules are ultimately selected from the grid of samples defined by the Nyquist interval: the dwell time. We define a harmonic tract as a sequence of samples that has a constant gap that is some multiple of the dwell time.
QSched v0.96 Spring 2018) User Guide Pg 2 of 6 1. 1D NUS Schedule The 1D quantile-based NUS scheduler generates deterministic schedules that conform closely to the PDF and satisfy other criteria such as minimized gaps and favorable PSF characteristics. The noweight option generates non-deterministic schedules. Usage at the shell prompt, the command line parser requires: -t (type), the PDF you would like to use; options are:* quant-exp Exponential matched or arbitrary bias quant-sin Sinusoids (unpublished) quant-poly Polynomial (unpublished); quadratic, cubic, or quartic gauss Gaussian may be matched, or arbitrarily biased linear Linear decay from 1 to x (where 0 <= x < 1) noweight Randomized, unweighted (1D NUS only) -d (dims) total points comprising the Nyquist grid -b (bins) number of points you would like to select -x (bias) choose characteristics of the particular PDF -x (jitter if the type is noweight) -e (evolution) the degree of T 2 that your PDF spans (enter a number between 0-pi.) If using linear, then e is not used so give e a dummy value such as 1. For more on e see Figure 1. *CAUTION: for 1D NUS, using e < 0.5 for weighted sampling IS RISKY; use unweighted sampling instead of a small e. -o (out) name of output file -ot (output_type) varian (first index 1), jeol (first index 1), bruker (first index 0) -lw (linewidth) required for guassian option -l (linear) percent of linear sampling Quick Start python qsched_096.py -t quant-sin -d 1024 -b 256 -o test.txt -x 1.5 -e 2.0 -ot varian Generates 256 samples selected from a 1024 Nyquist grid. PDFs Sine (x=1) Portion of a sinusoid from (pi to 3pi/2) *Sine1.8 (x=1.5) (unpublished) similar sensitivity as 1.5X exp. biased PDF *SineSq (x=2.) (unpublished) similar sensitivity as 2X exp. biased PDF *Quadratic (x=1) (unpublished) similar sensitivity as matched exponential *Cubic (x=1.5) (unpublished) similar sensitivity to 1.5X exponential *Quartic (x=2.0) (unpublished) similar sensitivity to 2X exponential Exponential enter x = 1 for matched, x = 2 for two-fold biased, etc. Gaussian enter x = 1 for matched, etc. noweight unweighted; x specifies jitter level (see below); but e not used * Developed as alternatives to exponential PDFs, they somewhat improve the PDF at intermediate evolution times
QSched v0.96 Spring 2018) User Guide Pg 3 of 6 General Comments Sensitivity While the application of NUS is widespread and robust, there can be complex inter-play among the sampling parameters. We recommend non-sparse NUS (ca. 25%-50% reduction) with generally no less than 20-30 samples in a given dimension. Conservative schedules will use matched or 1.5X bias. With experience, 2X biased sampling schedules also deliver robust results. The quant-sin and quant-poly PDF s are very similar and are also the most commonly employed in our own work and experiments. QSched takes the view of organizing schedules according to their intrinsic signal to noise ratio (isnr). A matched exponential schedule gives the same isnr (the SNR of the raw time domain data prior to any manipulation) as the Sine schedule and the Quadratic schedule when the evolution is pit 2. Sensitivities will be similar for other evolution times. isnr Enhancement for evolution to pi*t 2 1.7 2.0 2.15 Exponential Matched 1.5X biased 2X Biased Sine Family Sine Sine 1.8 Sine Squared Polynomial Family Quadratic Cubic Quartic x=1 x=1.5 x=2.0 Sinusoid Polynomial Linear This family of three schedules will choose more samples in the region 1-2T 2 than the comparable exponential PDF, and will have about the same intrinsic sensitivity as their exponential counterparts. Experiments show that these sinusoids can slightly improve peak bases in noisy/crowded regions. This family of three schedules (quadratic or cubic or quartic) will also choose more samples in 1-2T 2 than the comparable exponential PDF, and will also have the same intrinsic sensitivity as their exponential counterparts. The results of corresponding polynomial and sinusoidal schedules may be indistinguishable. Not widely used, but is a conservative approach that may be desired in some cases. (note the T2 parameter e is not used but qsched parses it anyway so enter a dummy number) No weighting (noweight) Selecting NUS samples for constant-time evolution still requires more investigation. One route is to use a weighted schedule (e.g. x=1.5 and e=2.0) since it will have good PSF s and lead to good reconstructions. Another possibility is to use the noweight option but comparisons of CT signals with weighted and noweighted NUS have not yet been conducted. specifying x = 0.3-0.4 jitter.
QSched v0.96 Spring 2018) User Guide Pg 4 of 6 Figure 1. The evolution parameter e is used to choose the portion of the function that will be the PDF. The pdf used by QSched is given by the red or blue solid lines line. The examples in the figure above illustrate the use of this parameter. Top left: to use matched exponential NUS and to sample up to pit 2, then choose x = 1 (matched) and e = pi (the PDF spans pit 2 ). The other panels follow by example. Example schedules choosing 16 out of 64 samples are shown in the four panels. (adapted from Craft et al. 2018) On the use of the evolution parameter e : For many reasons it may be desirable to use a portion of a PDF. That is accomplished with the e parameter. If, for example, you can sample to about 1T 2 and you wish the sampling to be matched, then still choose x =1 (matched) but specify an evolution of e = 1, which leads to the final PDF displayed in the bottom left panel of Figure 1. Examples for biased sampling are shown in the righthand panels of Figure 1.
QSched v0.96 Spring 2018) User Guide Pg 5 of 6 2. 2D NUS Schedule The 2D quantile-based NUS scheduler generates semi-deterministic schedules that conform closely to the PDF in each dimension, and which achieve good coverage, reduced gaps, and good PSF characteristics. Some advanced features are available including backfilling, jitter control and edge-forcing. Usage Supply the required parser commands as for the 1D case, but instead of one value for each input, you will enter two values, with the first relating to t1 and the second to t2. In addition you will supply (required): -j (jitter) the length (between 0.01-0.99) of each edge of a box that is centered in each quantile; thus specifying 0.7 means that the sample will be jittered in a (-.7 x 0.7) ---> 50% centered box --inclusion (edge forcing): options are 0 or 1 (default 1) 1 ensures edge forcing, 0 removes this feature --backfill: options are 0 or 1 (default 1) 1 ensures backfilling, 0 removes this feature --appendcorner: options are 0 or 1 (default 1) 1 appends top right corner, 0 removes this feature Quick Start python qsched_096.py -t quant-exp quant-sin -d 64 64 -b 16 16 -o test.txt -j 0.7 -x 1.5 1.5 -e 2.0 2.0 -ot bruker Generates 32 x 16 = 512 + 1 = 513 samples from a 64 x 64 = 4096 Nyquist grid. Note that the +1 is the appended (t1 max, t2 max) sample. The first dimension is a matched (x =1 ) exponential PDF that spans πt 2, while the second dimension is a matched (x = 1) sinusoidal PDF which also spans πt 2. Caution Constant Time Do NOT use the noweight option, which is intended only for 1D schedules. Since the noweight option performs jitter, and the 2D NUS also performs a jitter, the schedule would be over-jittered. As above, it is not yet determined how best to sample CT periods. One route is to use a universal PDF such as (x=1.5 and e=2.0) Another route is to use quant- exp with a very short evolution (such as e = 0.1) and negligible decay (x =0.1). The typical jitter (0.7 to 0.8) will randomize this while preserving coverage. General Comments: In developing and testing 2D NUS schedules, an interesting interplay emerged between the two augmentations. For very sparse sampling, inclusion should be disabled. For very dense sampling, backfilling should be disabled. Sparsity is not trivial to define, but here are rough suggestions: Coverage inclusion backfill ----------------------------------------------------------------------------------------- non-sparse (roughly > 20%) 1 0 intermediate (roughly 10-20%) 1 or 0 1 sparse (roughly <= 10% ) 0 1
QSched v0.96 Spring 2018) User Guide Pg 6 of 6 Discussion of Parameters for 2D NUS Jitter The amount of jitter in each dimension implemented onto the points selected. Note that 0.7 is equivalent to 70% jitter in each dimension, resulting in an area of 50% jitter, centered in each quantile. Preliminary results suggest that 0.7 x 0.7 = 50% is a good choice, but 0.75 and 0.8 show very good results also; users are advised to choose one of these. Output Two columns of sampling points; each row is a pair of sampling points. The first column point represents the t1 dimension, and the second column is the t2 dimension. Column one is presented as the slow dimension; column two is the fast dimension; a toggle will be added in a later version to switch these. For now the user can manually swap them in a spreadsheet program if needed. Back-Filling Qsched has an option that once each quantile contains one jittered sample, then any duplicates can be backfilled in order of closest to the origin. This provides advantages for sensitivity[3]; it also provides burstiness near the beginning of the evolution periods. [1] Backfilling is NOT recommended for dense NUS (e.g. 50% reduction in each dimension). For dense NUS, the backfilling will be excessive. But backfilling is strongly suggested for sparser sampling where even ultra-sparse NUS can result in a few duplicates that backfill and which are extremely helpful for burstiness and good reconstructions. Edge Forcing (aka inclusion) Qsched will force the edges of the sampling schedule to be their respective 1D quantiles; remaining samples in the 2D grid are chosen as jittered quantiles. It is strongly recommended to disable (--inclusion 0) for sparse sampling. Append Corner QSched takes the view that the top right corner (maximal evolution time in each dimension) should be included, and this point is appended by default (appendcorner = 1) if it is not already there. For example, if you request a 2D NUS schedule of 200 points, Qsched will return a schedule with 201 points. ----End of QSched v0.96 User Guide