Accuracy of the Computational Experiments Called Mike Steinberger Lead Architect Serial Channel Products SiSoft Time Domain Simulations Evaluation vs. Experimentation We re used to thinking of results that come from computers as being completely accurate and much more precise than we need. Many times, this leads to a false sense of security due to any of three possible problems: 1. Wrong Computation: The computation performed wasn t the correct one to begin with. For example, the boundary conditions imposed were unrealistic (3D field solver users beware) or the equations chosen did not apply to the problem at hand. 2. Numerical Inaccuracy: The algorithms used to solve the equations were not perfect (See [1] for the definitive practical treatment of this subject). 3. Incomplete Coverage: Not all relevant cases were considered. If none of these problems occurred, then we could call that computation an evaluation. Otherwise, we should consider the computation to be a computational experiment subject to the same uncertainties as a physical experiment. That is, the computational experiment can have sources of both random and systematic error, and there are confidence limits which apply to the results. One should be able to draw the error bars around the results and account for these error bars when making engineering decisions. This article considers time domain simulations of high speed serial channels as computational experiments, and explores the confidence limits that should be applied to such experiments. For the experiments considered here, the most critical problem is incomplete coverage. Serial channel performance is strongly affected by intersymbol interference and, as demonstrated in [2], all messages of length 64 or longer should be included in the experiment in order to obtain consistently accurate results. Suffice it to say that no time domain simulation will ever come close to the more than 10^19 bits required. While the results shown in this article may be of some direct value, the goal is to demonstrate some techniques that can be used to determine confidence limits for time domain simulations in general. 1
While the results shown in this article may be of some direct value, the goal is to demonstrate some techniques that can be used to determine confidence limits for time domain simulations in general. Experimental Approach The channel simulated was 5 Gb/s data transmitted over 1.5m of PC board trace in a low loss dielectric. There was no equalization at the transmitter and linear equalization at the receiver. The experimental approach taken was to vary the data pattern used in the time domain simulation as well as the length of the time domain simulation. To make sure that the data patterns were independent, they were drawn from different starting positions in the same 263-1 linear feedback shift register (LFSR) pattern. This LFSR pattern has the advantage that it is much longer than any of the time domain simulations in the experiment. If a data pattern were to be repeated Table 1: The data patterns used. Data Pattern Definitions Pattern Number Pattern Seed 1 2^63-1 LFSR 8191 2 2^63-1 LFSR 8291 3 2^63-1 LFSR 8391 4 2^63-1 LFSR 8491 5 2^63-1 LFSR 8591 6 2^63-1 LFSR 8691 7 2^63-1 LFSR 8791 8 2^63-1 LFSR 8891 9 2^63-1 LFSR 8991 10 2^63-1 LFSR 8091 over the course of a simulation, then the data patterns would no longer be independent. Rather than choosing different seeds for the same LFSR pattern, we could have chosen different LFSR patterns. If the different data patterns were long enough to produce a representative sample of the intersymbol interference, that would have been a valid choice. An alternating 1/0 pattern or a 27-1 LFSR would not have provided an adequate sample of the intersymbol interference, however. This approach was applied to simulations of three different lengths: one million bits, ten million bits, and one hundred million bits. These results can be used to estimate how much the confidence limits can be improved by running longer time domain simulations. Statistical analysis was also applied to the same channel. Statistical analysis is entirely different from time domain simulation in that it computes the statistics of the eye diagram directly rather than compiling them from samples of a time domain waveform. This computation has the advantage that it directly accounts for a statistically significant sample of the intersymbol interference, and the disadvantage that it is only rigorously applicable to linear, time invariant channels. Since the channel used in this study was truly linear and time invariant, this statistical analysis can be considered to be an evaluation rather than a computational experiment, and its results are what the average of the time domain simulation results should be. For the purposes of this study, the statistical analysis results are the right answer. Results A performance analysis of a high speed serial link produces a lot of results offering many different ways to look at the behavior of the channel. It is not the goal of this article to explore the many ways in which channel performance can be presented. Rather, the goal is to show how the results of time domain simulations vary. We will therefore use three different outputs as examples: 1. Inner eye contours: The shape of the inside of the eye diagram at a particular probability. The probabilities shown are 10-3, 10-6, 10-9, and 10-12. 2. Bathtub curves: Plots of the probability of error as a function of sampling time. These curves are called bathtub curves because they often resemble the cross section of a bathtub. 3. Eye width: The width of the open portion of the eye diagram. This value loosely correlates with timing margin. Figure 1 is an example eye diagram for the channel. All the eye diagrams in this study look very similar to each other. The following figures show the inner eye contours for the three different lengths of time domain simulation. Note that as the length of the time domain simulation progresses from one million bits to one hundred million bits, the 10-12 contour becomes clearly 2
Volts (v) 0.60 0.40 0.20-0.20-0.40-0.60 Figure 1: Example eye diagram. distinct from the 10^-6 contour, and it s almost possible to discern the 10^-9 contour. Notice also that the lower probability contours have considerably more variance than the higher probability contours. The following figures show the bathtub curves for the same sets of simulations, along with the bathtub curve for the statistical analysis (shown in red) and the clock PDFs for the time domain simulations. Note that this way of viewing the data makes it much easier to see the variation due to the different data patterns. Figure 8 is an expanded view of Figure 5, Bathtub curves for one million bit simulations and statistical analysis, on page 5, showing how the bathtub curves diverge for the ten different data patterns. Note that the bathtub curves are nearly the same for the higher error probabilities, but then diverge for the lower probabilities. Finally, Table 2 summarizes the mean and standard deviation of the eye width for the time domain simulations and statistical analysis. Persistent Eye Diagram 1.5m low loss PCB trace -5 5 Volts (mv) 15 5-5 -0-150 Eye Diagram Contours One million bit simulations with ten different data patterns -5 5 Figure 2: Inner eye contours for one million bit simulations. Volts (mv) 15 5-5 -0-150 Eye Diagram Contours Ten million bit simulations with ten different data patterns -5 5 Figure 3: Inner eye contours for ten million bit simulations. Volts (v) 15 5-5 -0-150 Inner Eye Diagram Contours One hundred million bit simulations with ten different data patterns -5 5 +Sensitivity 25 0mV Sensitivity 25 0mV Figure 4: Inner eye contours for one hundred million bit simulations. _ 2.4E-2 _ 7.1E-3 _ 6E-4 _ 3.7E-7 _ 4.8E-12 _ 1. 1. 1. _ 3
BLACK: Ten million bit time domain simulations using ten different data parameters Eye Width x1: (-62.150ps) x2: (65.040ps) dx: 127.20ps -5 5 Figure 5: Bathtub curves for one million bit simulations and statistical analysis. Figure 6: Figure 7: BLACK: Ten million bit time domain simulations using ten different data parameters -5 5 Bathtub curves for ten million bit simulations and statistical analysis. BLACK: One hundred million bit time domain simulations uisng ten different data patterns -5 5 Bathtub curves for one hundred million bit simulations and analysis. Expanded BLACK: One million bit simulations with ten different data patterns Note that as the time domain simulation gets longer, the eye width approaches the statistical analysis result. Note also that increasing the length of the simulation doesn t reduce the standard deviation very much. Discussion and Conclusions The accumulation of a persistent eye from a time domain simulation is an event counting experiment very much like counting radioactive particles with a Gieger counter. That is, for any particular bin in the eye diagram, the expected number of events is equal to the probability density for that particular bin times the number of bits simulated. Also, as in the Gieger counter experiment, the variance of the even count is equal to the square root of the number of events counted [3]. Therefore, as the number of expected events goes down, the variance of the count becomes a larger percentage of the count. In the limit that only one event is expected (for example, along the inner contour of the eye diagram), the variance is also one, meaning that maybe there will be an event counted and maybe there won t. One simple conclusion from the above reasoning is that the number of bits in a time domain simulation should be greater than the reciprocal of the probability of error. That is, if the target bit error rate is 10-12, the time domain simulations should be at least 10-12 bits long. That s not an experiment I m anxious to try. Figure 8: 131.0 132.0 133.0 134.0 135.0 136.0 137.0 138.0 139.0 Expanded view of one million bit bathtub curves. The more important conclusion, however, is that there is a statistical variation associated with the results of any time domain simulation 4
of a high speed serial channel. It s important that the user has a reasonable estimate of that variance so that they can use the simulation results to make reliable engineering decisions. This article has demonstrated one approach for obtaining such an estimate. References [1] Press, Teukolsky, Vetterling and Flannery, Numerical Recipes in C++, second edition, Cambridge University Press, 2002. [2] Steinberger, Exploration of Deterministic Jitter Distributions, DesignCon2008. [3] Bevington, Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill, 1969. About the Author Michael Steinberger, Ph.D., is responsible for leading SiSoft s ongoing tool development effort for the design and analysis of serial links in the 5-30 Gbps range. Dr. Steinberger has over 30 years experience in the design and analysis of very high speed electronic circuits. Dr. Steinberger began his career at Hughes Aircraft designing microwave circuits. He then moved to Bell Labs, where he designed microwave systems that helped AT&T move from analog to digital long-distance transmission. He was instrumental in the development of high speed digital backplanes used throughout Eye Width Results Simulation Duration Mean Eye Width (ps) Eye Width Standard Deviation (ps) One million bits 142.5 0.99 Ten million bits 138.92 0.86 One hundred million bits 137.81 0.75 Statistical analysis 137.5 0 Table 2: The mean and standard deviation of the eye width for the time domain simulations and statistical analysis. Lucent s transmission product line. Prior to joining SiSoft, Dr. Steinberger led a group of over 20 design engineers at Cray Inc. responsible for SerDes design, high speed channel analysis, PCB design and custom RAM design. 5