Supplementary Course Notes: Continuous vs. Discrete (Analog vs. Digital) Representation of Information Introduction to Engineering in Medicine and Biology ECEN 1001 Richard Mihran In the first supplementary notes of the semester, we examined the nature of sound, and observed that sound information, in the physical sense, is a time-varying perturbation of the pressure or density in a medium - in this case air. The graph of such a pressure variation that we can draw is simply a representation of this physical process on paper, and is by no means the only possible form of representation. One such representation is shown below: Pressure This particular graphical form of information representation utilizes a line whose height on the vertical axis of the paper varies over the horizontal axis in a manner which is analogous with the pressure variation over time of the physical system - in this case, sound. Both time and amplitude are represented as a continuum. Similarly, rather than drawing this relationship between pressure and time on paper, we might "encode" it on a long strip of magnetic film - i.e. magnetic tape. In this case, we would represent or encode the pressure variation in air as a function of time by analogously varying the degree of magnetization of the tape as a function of position. What I am describing is, of course, audio tape, such as is found in a common cassette tape. During playback, the magnetization changes along the length of the tape are converted to a time-varying electric current in a tape "head" by moving the tape in space at a controlled velocity. This would be another example of an analog form of representation. Getting back to a sound such as a simple A-note, imagine for a moment that I were to ask you to provide to me the sound "information" you have recorded when striking an A-note on a piano. If you were to simply hand to me a copy of the pressure vs. time graph, you would be providing me with an analog representation of the information. Alternatively, you might provide me with a set of data points from which I could construct my own graph. Thus for the A-note, the set of data points might look something like: Time time (msec) Pressure amplitude 0 0 0.1 0.27 0.2 0.53 0.3 0.74 0.4 0.90 0.5 0.98 0.6 0.99 0.7 0.93 0.8 0.80 0.9 0.60 1.0 0.36 etc. R. Mihran: Introduction to Engineering in Medicine and Biology Page 94
Note that the pressure information in this latter case is represented as discrete numbers, of finite resolution, at discrete points in time. This form of information representation illustrates the basic concept of digital representation. The physical system (in this case pressure) is sampled (measured) at discrete (typically periodic) time intervals, and the state of that system at any given discrete point in time is recorded as a number. Intuitively it may seem that in representing information in this way, some information would be lost: After all, one might assume that we would have no way of knowing the state of the system in the time which passes between respective samples. This is in contrast to analog representation, in which the information is represented continuously. This is not actually the case, however. As we have seen in class, as long as we sample at a high enough rate relative to the highest frequency component in the signal, it is actually possible to reconstruct the original analog signal perfectly. You might now wonder why we would go to the trouble of "digitizing" and subsequently reconstructing an information signal, when we could just represent and record it in an analog form right from the beginning. One of the primary reasons for recording and transmitting information in a discrete or digital format is an increased reliability in terms of maintaining (i.e. not distorting) the information content of the signal. To illustrate this point, let us return to the earlier scene where I ask you to provide me with a copy of the sound pressure information associated with the pure A-note. Assume for the moment that you recorded this pressure information in graphical, or analog form, on a piece of paper which you folded in half and placed in your file. You now proceed to make a photocopy of your graph, but the crease down the middle of the paper imparts a slight "wave" to the line depicting pressure amplitude on the copy. You now give that copy to me without mentioning the crease in the original. I might subsequently incorrectly interpret this distortion as being representative of the physical state of the system. In contrast, if you had the information recorded on your paper as a table of data points instead, the distortion to the physical forms of the numbers lying along the crease would likely not be so great as to prevent me from interpreting their intended values. In other words, the fact that the system is restricted to a discrete set of states allows me to "overlook" slight distortions and still interpret the correct value. This is one of the great advantages of digital information representation and processing. This is not a new concept, by any means. Nature has recognized this inherent advantage of discrete (digital) representation of information, and utilizes what can be thought of as a form of digital encoding in the storage and transfer of genetic information in the form of DNA. Genetic information is encoded using patterns of the bases adenine, guanine, thymine and cytosine. These bases are paired across the double stranded DNA molecule such that adenine (A) always pairs with thymine (T), and cytosine (C) always pairs with guanine (G). The genetic code consists of triplets of bases, such that each segment of three pairs represents a genetic code word. Figure 3-6 from the text illustrates an example of a small segment of DNA showing three such code words. If one considers each of the 4 bases to represent a discrete digital value, e.g. 0-3, then you can conceptualize the DNA information as being encoded by a very long pattern of base 4 numbers. This is not unlike the digital representation of information we use in computers, which are long patterns of base 2, or binary numbers i.e. ones and zeros. R. Mihran: Introduction to Engineering in Medicine and Biology Page 95
Fundamentals of Digitizing Analog Signals One of the central concepts presented in various forms throughout this course is the representation of information in different forms. Earlier in the course we introduced this concept with the example of representing voice information by a time varying signal whose amplitude varied analogously with the air pressure variation of the sound: As we observed above, it is possible to represent the same information as a set or series of numbers, or discrete states, corresponding to the amplitude of the sound pressure waveform at discrete points in time. This is the essence of digital representation of information. In the following sections we will briefly survey some of the fundamental considerations in transforming an analog, time varying signal to a digital form of representation. Digitizing Analog Signals: the Nyquist Sampling Theorem Assume that the analog voltage signal shown below represents a brief interval of music: R. Mihran: Introduction to Engineering in Medicine and Biology Page 96
The basic process of "digitizing" this signal is to "sample" the state of the amplitude of the signal at regular intervals, and represent this amplitude at a given time by a discrete numeric or digital value. Thus for our music signal, we may record the value of the amplitude at regular intervals, as shown below: One of the basic questions which needs to be addressed in proceeding to digitizing the signal just how fast or frequently we need to sample or measure the analog waveform. It is clear that if we do not take samples frequently enough, we will be losing some of the information contained in the original analog signal - but just how fast do we need to sample? The very simple and elegant theoretical answer to this question is provided by the Nyquist sampling theorem, which essentially states: A band-limited signal which has no frequency components above fmax can be uniquely represented by its sampled values at uniform intervals that are not more than 1/2fmax seconds apart. Thus in the current example, if the highest frequency component of our music signal is 20,000 Hz, we can capture all the information in that signal (and therefore reconstruct it later in an analog form with no error) if we sample it at at least 2 times 20kHz, or 40 khz. The consequences of not sampling at a high enough rate, called "undersampling", can be rather severe. Upon reconstructing the sampled signal, some of the higher frequency components which were "undersampled" will be "folded over" into lower frequencies, causing a bad form of distortion known as aliasing. Thus, not only do we lose some part of the original information, but we actually inappropriately add energy from the undersampled higher frequency components back into the digitized signal in a distorted fashion. The process for selecting the sampling rate for a general signal would necessarily first involve a determination of its highest frequency component. In practice, this can be done using an instrument known as a spectrum analyzer, which you have seen demonstrated in class lectures. This instrument performs a fast fourier transform (FFT) on the signal and generates a frequency domain representation of the signal, known as a power spectrum. Thus, if the power spectrum of the signal you wish to digitize looked like this, R. Mihran: Introduction to Engineering in Medicine and Biology Page 97
Relative Power Frequency (khz) 1 2 3 4 The Nyquist sampling rate would be 2 times the highest frequency component, or about 6000 samples per second in this case. In practice, you would typically choose a sampling rate somewhat higher than the theoretical minimum, to give some margin for error, since the distortion from aliasing can be significant. The bandwidth of commercially recorded music is approximately 20-20kHz, with CD's representing this information with samples that were taken at approximately 44kHz, somewhat over twice the Nyquist rate. In addition, the analog signals should also be filtered appropriately to insure that they are in fact band-limited. One reason for doing this is to remove any unanticipated high frequency noise from the analog signal before digitizing, again to prevent aliasing. R. Mihran: Introduction to Engineering in Medicine and Biology Page 98