Jaggies as aliasing or reconstruction phenomena: a tutorial

Transcription

1 Jaggies as aliasing or reonstrtion phenomena: a ttorial Isaa Amidror Roger D. Hersh Downloaded From: on //4 Terms of Use:

2 Jornal of Eletroni Imaging 3(), 8 (Jan Mar 4) TUTORIAL Jaggies as aliasing or reonstrtion phenomena: a ttorial Isaa Amidror Roger D. Hersh Eole Polytehniqe Fédérale de Lasanne (EPFL) IC-LSP, Station 4 5 Lasanne, Switzerland isaa.amidror@hotmail.om Abstrat. Jaggies (stairasing effets) along slanted lines or rved edges are omnipresent in digital imaging. They are so widespread in digital display devies that very often they are assoiated with the modern ompterized world (and sometimes even intentionally introded into artworks sh as logos, advertisements, et. to onvey a modern pielized look). Althogh this sbjet is not new, it still remains an important isse in the design of modern digital display and printing devies. In the lassial literatre, jaggies are often onsidered as aliasing artifats; and yet some other referenes onsider them instead as reonstrtion artifats. The present ttorial revisits this qestion and tries to elidate the real natre of this phenomenon sing Forier-based onsiderations. It shows that the jaggies an be either aliasing artifats de to poor sampling in aptre, or the reslt of poor reonstrtion; and it eplains the impliations thereof on the elimination of nwanted jaggies. The Athors. Pblished by SPIE nder a Creative Commons Attribtion 3. Unported Liense. Distribtion or reprodtion of this work in whole or in part reqires fll attribtion of the original pbliation, inlding its DOI. [DOI:.7/.JEI.3..8] Introdtion It is well known that the approimation of ontinos lines or shapes by piels on a disrete raster grid may ase the appearane of jagged edges (also known as jaggies or stairasing effets ) on the shape s borders (see Refs. and, p. 4). Sh jaggies often or dring the sampling proess (analog-to-digital onversion) along slanted or rved lines. They are most prominent on sharp edges sh as blak/white transitions, as in Fig., bt they are also visible in smoother gray-level transitions, as in Fig.. These jaggies are partilarly objetionable at low resoltions (i.e., low sampling rates), bt they may still be present even in modern higher-resoltion devies. Ths, althogh jaggies are not new, they are still an important isse in the design of today s digital devies, as one an jdge, for eample, by searhing for the term jaggies in the U.S. patent database; see, for eample, the reent U.S. patents 8,35,967, 8,345,63, 8,94,73, 8,6,89, and 6,37,566 (Refs. 3 7), to mention jst a few. It is therefore important to have a good nderstanding of this phenomenon and its origins. In the lassial ompter graphis literatre, and in partilar, in the field of digital typography, jaggies are often onsidered as aliasing effets (see, for eample, Ref., pp. 4 5; Ref. 8; or Ref. 9, pp ). Moreover, the Paper 34T reeived Jl. 9, 3; revised mansript reeived Nov., 3; aepted for pbliation Nov. 9, 3; pblished online Jan., methods devised for reding the visibility of these jaggies are often alled antialiasing methods. On the other hand, other referenes onsider jaggies as artifats that are de to the reonstrtion proess (see, for eample, Ref., pp. 7 8). Or aim in this ttorial is to elidate this qestion based on simple Forier onsiderations: It is lear that if jaggies are indeed aliasing phenomena, they mst be represented in some way or another in the Forier domain (i.e., in the freqeny spetrm of the sampled image), too. We will see that in many ases, jaggies are indeed aliasing phenomena, bt in other ases they are only generated in the reonstrtion stage, i.e., when the display devie rereates a ontinos-world signal from the sampled one. Note that throghot this ttorial, we assme that the jaggies in qestion are sffiiently big to be seen by the naided eye, and we do not disss isses related to the hman visal system, modlation transfer fntions, et. Also, this ttorial does not intend to provide formal theorems and proofs, bt rather favors an informal approah sing Forier-based onsiderations and pitorial illstrations. Bakgrond Aliasing is a well-known phenomenon whih may or in the disretization (sampling) of an nderlying inpt fntion. Disretizing an analog signal reqires that the signal s vale be sampled often enogh to define the waveform nambigosly. Aording to the lassial sampling theorem (see, for eample, Ref., p. 5; Ref., Se. 8 7; or Ref. 3, p. 593), a sampling freqeny of at least twie the highest freqeny present in the signal is sffiient for its waveform to be defined ompletely, and for allowing its orret reonstrtion. (Note that lower sampling freqenies may sffie for some speial types of fntions, as eplained in Appendi A; bt the lassial sampling theorem addresses the general ase.) If the sampling points are not taken as densely as reqired (a sitation often alled ndersampling), they will fail to follow the high-freqeny fine details of the signal, ths leading to aliasing. In the signal domain, this is epressed by the eistene of a smoother, lower-freqeny signal, known as alias, whih an be traed throgh all the sampled points and mimi or masqerade the behavior of the original signal on its sampled vales (see Fig. ). Only based on these too far-apart sampling points, the original signal is ndistingishable from its lower-freqeny aliased signal, sine both of them oinide on all of these sampling points. In other words, aliasing means that high-freqeny Jornal of Eletroni Imaging 8- Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

3 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial of this sampling implse omb is itself an implse omb with implse interval of f s ¼ Δ and implse height of Δ (see, for eample, Ref. 4, pp. 7 8; Ref. 5, p. 577): F ½ð ΔÞIIIð ΔÞŠ ¼ IIIðΔÞ: () Fig. Jaggies in disrete-world images: Along sharp edges (here: slanted blak lines forming the letter A). Along smooth gray-level shapes (here: a irlar osinsoidal wave). omponents in the original signal appear inorretly as lower freqenies in the sampled reslt. This is the interpretation of aliasing in terms of the signal domain, bt the aliasing phenomenon an be also interpreted from the viewpoint of the spetral domain. Remember that sampling in the signal domain is eqivalent to mltiplying the original ontinos signal gðþ by ð ΔÞIIIð ΔÞ, a periodi nit-height implse omb (implse train) having an implse interval of Δ (see Fig. 3). The Forier transform [Note that de to the implse property δð aþ ¼jajδðÞ (Ref. 5, p. 8) the implse height of the omb IIIð ΔÞ is Δ (Ref. 5, p. 577), and its nit-height onterpart is ð ΔÞIIIð ΔÞ. Similarly, the implse height of IIIðΔÞ is Δ]. Therefore, by virte of the onvoltion theorem, the effet on the spetral domain of sampling gðþ is that GðÞ, the original spetrm of gðþ, is now onvolved with the implse omb (). This means that the spetrm of the sampled version of gðþ onsists of infinitely many idential replias of the spetrm GðÞ, whih are saled by Δ (in terms of the amplitde) and entered abot all the integer mltiples of the sampling freqeny f s ¼ Δ (see, for eample, Ref. 4, pp. 7 9 or Ref. 6, pp. 79 8): F ½ð ΔÞIIIð ΔÞgðÞŠ ¼ F ½ð ΔÞIIIð ΔÞŠ F ½gðÞŠ ¼ IIIðΔÞGðÞ; () where is the onvoltion operator. Spetral domain g() G() d f G III( ) III( ) III( ) g() f G III( ) * G() f G d f G f s > f G f s = f s = 4 4 Fig. Illstration of the signal-domain interpretation of aliasing: Sampling a ontinos signal [here: a osine fntion gðþ ¼ osðπfþ with freqeny f ¼ ] at a rate higher than twie the maimm signal freqeny gives a orret disrete representation of the original signal. Sampling the same ontinos signal at a rate lower than twie the maimm signal freqeny gives a false, aliased lower-freqeny signal (here: a osine fntion with freqeny f ¼.5) that mimis the original signal on its sampled vales. Both signals are drawn with ontinos lines and their sampled vales are indiated by blak dots. ' III( ' )g() ' () (d) f ' s III( ' ) *G() f ' s d ' f s f s > f G f = s ' ' f s' f s ' < f G Fig. 3 Shemati illstration of the Forier-domain interpretation of aliasing: An original ontinos-world signal gðþ and its spetrm GðÞ. The sampling implse train and its spetrm. () The sampled signal is the prodt of the signals and, and therefore its spetrm is the onvoltion of their spetra. Note the replias of the original spetrm that are loated at all integer mltiples of the sampling freqeny f s. (d) If the sampling rate is lower than twie the maimm signal freqeny f G (say, f s), then eah two neighboring replias of GðÞ overlap (additively), giving false aliased freqenies in the overlapped zones. The aliased parts of the spetrm in (d) are shown by thik dashed lines, whih are simply the sm of the original overlapped replias. The horizontal arrows in () and (d) indiate the freqeny range that is obtained by disrete Forier transform (DFT), spanning between mins half and pls half of the sampling freqeny. Jornal of Eletroni Imaging 8- Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

4 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial Note that if the spetrm GðÞ of the original fntion gðþ is ontinos, so is the periodi spetrm () of the sampled version of gðþ. Let s now denote by f G the highest freqeny in GðÞ. As long as the sampling freqeny f s ¼ Δ is at least twie f G, as reqired by the lassial sampling theorem, the replias of GðÞ are sffiiently far from eah other to avoid overlapping [Fig. 3()]. Bt if the sampling freqeny is lower than twie f G, every two neighboring replias of GðÞ will somewhat overlap [Fig. 3(d)]; note that this overlapping is additive de to the properties of onvoltion. This means in partilar that freqenies from the neighboring replias will penetrate into the main replia (the replia whih is entered abot the origin) and appear in the spetrm as parasite, false lower freqenies, known as aliased freqenies. Note that in this ase, too, all of the replias are idential; however, none of them (inlding the main replia) remains idential to the original spetrm GðÞ itself, sine they have all been orrpted (additively overlapped) by freqenies (positive or negative) belonging to their neighboring replias [see Fig. 3(d)]. The above disssion abot aliasing an also be etended to the two-dimensional (-D) or mltidimensional ases. In sh ases, sampling in the signal domain and the reslting onvoltion in the spetral domain [Eq. ()] are simply etended to two or M-dimensions (we assme here that the same sampling interval Δ is sed along all the M dimensions, bt if reqired it is also possible to se along eah dimension a different sampling interval): F½ð ΔÞ M IIIð ΔÞgðÞŠ¼F½ð ΔÞ M IIIð ΔÞŠF½gðÞŠ ¼IIIðΔÞGðÞ; (3) where, R M, IIIð ΔÞ is the M-dimensional sampling implse omb (having an implse interval of Δ along eah dimension), and is the M-dimensional onvoltion operator. 3 Jaggies, Aliasing, and Reonstrtion Having reviewed the image-domain and the spetral-domain manifestations of aliasing, how an this phenomenon be related to jaggies? Let s onsider, as an illstrative eample, a slightly rotated periodi line grating, i.e., a periodi seqene of -valed lines with period T and width < τ < T on a -valed bakgrond (see Fig. 4, where blak is negative, white is positive, and midgray is zero). Its original spetrm, before sampling, is a slightly rotated infinite omb whose implse interval is eqal to the freqeny of the line grating, and whose implse strengths are modlated by a sin fntion [whih is T of the spetrm of a single line of width τ; see Ref. 7, pp. 3 for the one-dimensional (-D) ase]. This spetrm is obviosly not band limited. After sampling, as we have seen in Eq. (3), the ontinos-world spetrm of the reslting sampled grating onsists of the original omb pls infinitely many replias of this omb that are entered abot eah implse of a nailbed (whih is itself the Forier transform of the sampling nailbed). This is shown in Fig. 4 [note that the spetra are obtained by disrete Forier transform (DFT), and they are only shown between the freqenies ð Þf s and ð Þf s along both of the aes, where f s is the sampling freqeny]. This means that the spetrm of the sampled grating ontains many new implses belonging to the new replias. If any of these new implses falls lose enogh to the spetrm origin, as in Fig. 4() (note the two enirled implses), a new low-freqeny strtre (moiré effet) beomes visible in or sampled grating, as shown by the arrows in the image domain (for more details on the sampling moiré effets see, for eample, Ref. 7, Se..3). Bt even if none of the new implses falls lose to the spetrm origin and ases a moiré effet [see Fig. 4], it is lear that these new implses still mst orrespond to some new strtres in the sampled grating, whih did not eist in the original, ontinos grating. Indeed, it trns ot that these new implses represent new freqenies that orrespond to the jaggedness of the sampled grating. To see this, onsider Fig. 5, whih is intentionally drawn at a lower resoltion so that the individal jaggies as well as the individal implses in the spetrm an be learly visible. Figre 5 shows the sampled grating and its spetrm as obtained by DFT. As we an see, the spetrm ontains the main, original implse omb (the slightly rotated omb passing throgh the origin, whih is indiated by the arrows) pls implses belonging to its new replias de to the sampling. In order to see the inflene of these new implses, let s zero all of them, leaving in the spetrm only the implses belonging to the original omb. The orresponding strtre bak in the signal domain is obtained by taking the inverse DFT of this spetrm. The reslts of this maniplation are shown in Fig. 5. As we an see in this figre, the effet on the signal domain of eliminating the implses of the new replias (the aliased implses) onsists of smoothing ot the jagged edges of the sampled grating (see the gray level piels along the line edges). Note, however, that this does not yet ompletely eliminate the jaggies; as we will see later, the residal jaggies that we still see in Fig. 5 are reonstrtion artifats that or de to the sqare piels being sed to draw the sampled signal. Finally, Fig. 5() shows what happens if we zero in the spetrm of Fig. 5 the implses of the original omb and keep only the new implses that are de to the sampling. One again, the orresponding strtre bak in the signal domain is obtained by taking the inverse DFT of this spetrm. Note that the signal-domain strtre in Fig. 5() is simply the differene between the jagged grating of Fig. 5 and its smoothed-ot version of Fig. 5; this differene orresponds, indeed, to the net effet of the jaggies themselves on the line edges. Althogh these Forier-based onsiderations are most easily illstrated in periodi strtres as in Fig. 5 (sine their spetra are prely implsive), it is lear that jaggies do not only or in periodi strtres. To illstrate a simple aperiodi ase onsider Fig. 6, whih shows a single slightly rotated line of width τ >. The spetrm of this aperiodi strtre onsists of a ontinos infinitely long line implse (a blade ) passing throgh the origin, whose amplitde is modlated by a sin fntion, and whih is slightly rotated by the same angle (Ref. 9, pp ). In this ase, too, the spetrm is obviosly not band limited, so that the sampled signal will sffer from aliasing. After sampling, the spetrm of the reslting sampled line onsists of the original line implse pls infinitely many replias thereof. Figre 6 shows the sampled line and its DFT spetrm. Figre 6 shows the main line implse, after zeroing all its replias (the aliased elements) in the DFT spetrm, Jornal of Eletroni Imaging 8-3 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

5 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial 8. Spetral domain () Fig. 4 Aliasing in the two-dimensional (-D) ase: A simple eample where the spetrm onsists of a one-dimensional (-D) implse omb. The original ontinos-world fntion in the signal domain is a vertial line grating that has been slightly rotated by angle θ. Its spetrm is a horizontal infinite implse omb throgh the spetrm origin that is slightly rotated by the same angle θ, and whose implse interval is eqal to the freqeny of the grating (see, for eample, Ref. 7, pp. 3 5). After sampling, the spetrm of the reslting sampled grating onsists of the original omb pls infinitely many replias of this omb, that are entered abot the points (kf s, lf s ), where f s is the sampling freqeny in both diretions and k, l Z. In eah of the rows of this figre the left-hand olmn shows the sampled, disrete-world signal and the right-hand olmn shows its spetrm (as obtained by DFT) between the freqenies ð Þf s and ð Þf s in both diretions. The only differene between the three rows is in the rotation angle θ: θ ¼ deg; θ ¼ 4 deg; and () θ ¼ 3 deg. In ase (), de to the small rotation angle, the replias of the original omb form a sharper angle θ with the horizontal ais of the spetrm, and ths they fall mh loser to the original omb itself [ompare the spetra in rows and ()]. In sh ases, implses of the replias may fall very lose to the spetrm origin (see the two implses marked by irles). Indeed, these implses orrespond to a new low-freqeny parasite strtre (sampling moiré) whih is learly visible in the sampled line grating (see the arrows in the signal domain). Bt even when none of the new implses falls lose to the spetrm origin [as in row ], it is lear that these new implses still mst orrespond to some new strtres in the sampled gratings; and as we will see below, they orrespond indeed to the jaggedness of the sampled grating. The ross-like osillations, whih srrond implses in the spetra of and (), are de to the leakage artifat of the DFT (see, for eample, Ref. 6, Se. 6.4). Note that in this and the following figres gray levels represent the displayed vales: blak is negative, white is positive, and midgray is zero. Jornal of Eletroni Imaging 8-4 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

6 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial Spetral domain () Fig. 5 Jaggies along the edges of a sampled, slightly rotated line grating, and their spetral representation. The sampled grating and its spetrm (as obtained by DFT) between the freqenies ð Þf s and ð Þf s in both diretions; the arrows in the spetrm show the main omb. The implses of the main omb alone, after zeroing all the remaining aliased implses, and the orresponding signaldomain strtre obtained by inverse DFT. () The aliased implses otside the main omb, after zeroing the implses of the main omb, and the orresponding signal-domain strtre obtained by inverse DFT. As shown in, the effet on the signal domain of eliminating the implses of the other replias (the aliased implses) onsists of smoothing ot the jagged edges of the sampled grating (see the gray level piels along the line edges). The signal-domain strtre in () is simply the differene between the jagged grating of and its smoothed-ot version of ; this differene orresponds, indeed, to the net effet of the jaggies themselves on the line edges. Note that for the sake of simpliity, the grating shown in is perfetly symmetri abot its origin, so that the reslting spetrm is prely real-valed. If the original ontinos-world grating is slightly shifted before sampling, the jaggies in may vary and lose their perfet symmetry; in this ase, eah spetral omponent will also have an imaginary-valed part as predited by the -D shift theorem (Ref. 8, p. 56), bt eept for having in that ase omple-valed implses, nothing in or disssion will be hanged. Note also that in the present figre, there are no leakage artifats bease the figre is perfetly wraparond (see Se. 6.4 in Ref. 6 for the -D ase or Se. 6.6 and Remark 6.7 in Ref. 9 for the -D ase). Jornal of Eletroni Imaging 8-5 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

7 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial Spetral domain () Fig. 6 Same as in Fig. 5, eept that this time the original ontinos-world signal is not a slightly rotated line grating, bt only a single line taken from this grating (the entral one). The spetrm of this line onsists of a ontinos line implse passing throgh the origin whose amplitde is modlated by a sin fntion, and whih is slightly rotated by the same angle. After sampling, the spetrm of the reslting sampled line onsists of the original line implse (indiated by arrows) pls infinitely many replias of this line implse, that are entered abot the points (kf s, lf s ), where f s is the sampling freqeny in both diretions and k, l Z. The sampled line and its spetrm (as obtained by DFT) between the freqenies ð Þf s and ð Þf s in both diretions. The main line implse, after zeroing all its replias in the DFT spetrm, and the orresponding signal-domain strtre obtained by inverse DFT. () The replias of the line implse, after zeroing the main line implse, and the orresponding signal-domain strtre obtained by inverse DFT. As in Fig. 5, the effet on the signal domain of eliminating the aliased elements from the spetrm onsists of smoothing ot the jagged edges of the sampled line. The ripple effet in the DFT spetrm is de to the leakage; leakage artifats are present here bease in this ase the signal-domain figre is not wraparond (see, for eample, Se. 6.6 in Ref. 9). Jornal of Eletroni Imaging 8-6 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

8 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial 8. Spetral domain Fig. 7 A sampled vertial line grating may have a sampling moiré effet (note the periodially repeating thiker vertial lines) de to the new aliased low freqenies that appear de to an interation with the sampling freqeny. Compare with Fig. 4, where no sampling moiré ors. In both ases, no jaggies are present. and the orresponding signal-domain onterpart obtained by inverse DFT. Finally, Fig. 6() shows only the replias of the line implse, after zeroing the main line implse itself, and the orresponding signal-domain onterpart obtained by inverse DFT. As in Fig. 5, the effet on the signal domain of eliminating the aliased elements from the spetrm onsists of smoothing ot the jagged edges of the sampled line. Note that in both Figs. 5 and 6, the jaggies or on slanted edges. In both ases, if the lines in the signal domain are prely vertial (or horizontal) no jaggies appear on their edges. Indeed, in sh ases, the new replias in the DFT spetrm de to the sampling fall on top of the original replia, and no new freqenies are generated that orrespond to jaggies [see Figs. 4 and 7]. As we an see, there is indeed strong evidene in favor of onsidering jaggies as aliasing phenomena. Frthermore, jst like the previosly disssed faets of aliasing (masqerading lower freqenies in the signal domain or overlapping of replias in the spetral domain), jaggies, too, tend to beome less prominent when we inrease the sampling resoltion or when we apply low-pass filtering prior to the sampling, so that sharp transitions beome smoother., On the other hand, there are also good reasons against onsidering jaggies as aliasing phenomena. For eample, there eist many strtres that present strong aliasing, bt have no jaggies at all (for instane, as shown in Fig. 7, a sampled horizontal or vertial line grating may have a sampling moiré effet de to new aliased low freqenies withot having jaggies). This means, indeed, that aliasing does not neessarily ase jaggies. Bt frthermore, jaggies may also eist when there is no aliasing at all for eample, in a -D plot of a slightly rotated osinsoidal grating, whose freqeny is far below half of the sampling freqeny (see Fig. 8). How an this be eplained? The key for nderstanding this qestion resides in the ambigity of the term aliasing. This term is sed in the literatre for two different effets: () the distortion that is proded by poor sampling, and whih ases the sampled. Spetral domain Fig. 8 Low-resoltion figre showing a slightly rotated osine wave of freqeny f ¼.88 and its spetrm. Note that jaggies are visible althogh the sampling freqeny f s ¼ 4 is higher than f, and no aliasing ors in the sampling proess. Jornal of Eletroni Imaging 8-7 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

9 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial signal to beome indistingishable from a lower-freqeny signal (an alias); and () the distortion that is proded by poor reonstrtion, whih ases the signal that is reonstrted from the samples to be different from the original ontinos signal (again, an alias). Note that both of these problems originate from failres in the orret appliation of the sampling theorem: the failre to flfil the reqired ondition on the freqenies leads to sampling aliasing; while the failre to approimate ideal reonstrtion leads to reonstrtion error. To resolve this terminologial ambigity, we will heneforth all the first, lassial effet aliasing de to the sampling or sampling aliasing, while the seond effet will be alled reonstrtion error [other terms being sometimes sed in the literatre are prealiasing and postaliasing, respetively (see Ref., p. )]. Let s eplain this in more detail. Sampling is the proess that onverts a ontinos signal to a disrete one, while reonstrtion is the proess that rereates a ontinos signal from its samples (Ref., Se..5; Ref. ). Note that theoretially, a sampled signal onsists of zero-width implses (of varying heights), and not of real-world piels having sqare or irlar shapes. It is preisely the reonstrtion proess that brings bak the flesh arond eah of the sampled bones. Now, aording to the lassial sampling theorem, all the information in the original ontinos signal is preserved in its sampled version, if the sampling freqeny is at least twie the highest freqeny ontained in the signal. Under this ondition, the theorem garantees that the original ontinos signal an be perfetly reonstrted by mltiplying the spetrm of the sampled signal with a ret fntion that ts off all the freqenies beyond half of the sampling freqeny, or eqivalently, by onvolving the sampled signal with the Forier transform of this ret fntion, i.e., with the orresponding narrow sin fntion (Ref. 6, p. 83). This is easier to nderstand in the spetral domain, as is learly shown in the right-hand olmn of Figs. 3 and 9. To formlate the perfet reonstrtion proess shown in Fig. 9 mathematially, let s denote or sampled signal by sðþ¼ð ΔÞIIIð ΔÞgðÞ and its spetrm by SðÞ¼ IIIðΔÞGðÞ [see row () in Fig. 3, whih has been opied into row of Fig. 9]. We also denote the ideal ret fntion that ts off all the freqenies above ð Þf s and below ð Þf s by Δ retð f s Þ, where f s ¼ Δ is the sampling freqeny [see Fig. 9; the onstant fator Δ ¼ f s is reqired to normalize the heights]. Then, the prodt of the spetra in rows and of Fig. 9 gives bak, as we an see in Fig. 9(), the original spetrm GðÞ: SðÞΔ retð f s Þ¼GðÞ: (4) Therefore, in the signal domain of Fig. 9(), the onvoltion of the sampled signal sðþ with F ½Δ retð f s ÞŠ ¼ sinðf s Þ yields, indeed, a perfet reonstrtion of or original signal gðþ: sðþsinðf s Þ¼gðÞ: (5) However, in pratie, the reonstrtion of a sampled signal is never done by onvolving its implses with an infiniterange sin fntion, as stiplated by the lassial sampling theorem and shown in Fig. 9. Instead, eah implse of the sampled signal is typially represented (i.e., onvolved) by a plse fntion pðþ, whose width eqals the distane between two onsetive samples (see Fig. ). [This is, of orse, a theoreti idealization, sine sally pðþ is not a perfet retangle.] Stating this mathematially, in the ase of real-world reonstrtion we get in the spetral domain the prodt [see Fig. ()]: SðÞPðÞ GðÞ; (6) s() = III( ) g() Spetral domain S() = III( ) * G() d s() = III( ) g() S() = III( ) * G() Spetral domain d sin( f s ) f G f G f s ret(/ f s ) p() = ret( ) f G f G f s P() = sin( ) s()* sin( f s ) = g() S() ret(/ f s ) = G() d f s f s = s()* p() g() f s = S()P() G() d () () f s Fig. 9 Contination of Fig. 3 showing shematially the signal and spetral domain representations of an ideal reonstrtion proess, whih gives bak eatly the original ontinos signal we had before sampling. The sampled signal, as in Fig. 3(). The ideal reonstrtion fntion aording to the lassial sampling theorem (a narrow sin fntion of height ) and its spetrm, whih is a ret fntion of height f s ¼ Δ that etends from ð Þf s to ð Þf s. () The perfetly reonstrted signal [onvoltion of the signals and ] and its spetrm [prodt of the spetra and ]. Note that the reonstrted signal in row () is indeed idential to the original ontinos signal shown in row of Fig. 3. Fig. Contination of Fig. 3 showing shematially the signal and spetral domain representations of a nonideal reonstrtion proess, that ases reonstrtion artifats. The sampled signal, as in Fig. 3(). A nonideal reonstrtion fntion (a ret fntion representing a sqare piel) and its spetrm, whih is a sin fntion that etends ad infinitm. () The reonstrted signal [onvoltion of the signals and ] and its spetrm [prodt of the spetra and ]. Row () is not idential to row in Fig. 3, and it ontains reonstrtion artifats (note the debris from the neighboring replias that give new high-freqeny noise in the reslting reonstrted signal). Jornal of Eletroni Imaging 8-8 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

10 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial [where PðÞ is the spetrm of pðþ] rather than the perfetly reonstrted spetrm of Eq. (4). Indeed, in the signal domain of Fig. (), the onvoltion of the sampled signal sðþ with the piel pðþ does not yield a preise opy of or original signal gðþ bt rather a pielized or jagged version thereof: sðþ pðþ gðþ: (7) Similarly, in the -D ase (like in ompter displays or in digital printing devies) eah implse of the sampled signal sð; yþ to be displayed is onvolved with a single piel shape pð; yþ (whih may be, depending on the devie, a sqare dot, a irlar dot, et.). The reslting reonstrted signal may therefore have highly visible jaggies along its borders (see Fig. 8), and possibly also sharp transitions between the vales of neighboring piels. These elements orrespond to new high freqenies in the reonstrted signal that are not present in the samples, and whih do not eist in the original, ontinos signal gð; yþ, either (see the -D spetral domain eplanation of Fig. ). This proess reslts in reonstrtion errors, sine when the resoltion of the display devie is not sffiient the reonstrted image differs signifiantly from the original ontinos image (it is pielized and jagged), and what we atally see is an alias. This kind of aliasing has nothing to do with ndersampling and it is solely de to poor reonstrtion; indeed, as already mentioned above, this phenomenon may or even in ases whih are ompletely alias-free (in the lassial sense), as in Fig. 8. Note that in some ases the individal display-devie piel pð; yþ is smoother and has the shape of a narrow -D Gassian; this is often the ase in CRT displays. In sh ases reonstrtion errors still eist (althogh they look smoother), sine the spetrm of a Gassian is itself a Gassian (Ref. 8, p. 49), whih is obviosly not a band-limited spetrm. Let s now retrn to or original qestion. As we an see in Figs. 5 and 6, in many ases jaggies indeed reslt from the sampling aliasing, and they are represented in the spetrm by aliased elements (this is partilarly evident when sampling a -D signal involving slanted sharp transitions, whih is not band limited and hene sffers from aliasing). Clearly, jaggies are frther amplified by the reonstrtion errors (poor reonstrtion of the signal). However, they an even be generated by the reonstrtion errors if they have not already been generated de to the sampling aliasing. This is the ase, for eample, in a slightly rotated low-freqeny osinsoidal grating, whih is learly band limited and has no sampling aliasing (the implse samples along the orrgations of the sampled osinsoidal grating are gradally attenated in sh a way that no jaggies are apparent). In ases like this, if the reonstrtion is orretly done, as stiplated by the lassial sampling theorem, no jaggies shold appear in the reonstrted signal; and if jaggies do appear as in Fig. 8 this is only de to the reonstrtion error. Indeed, this observation led some referenes to say that jaggies are not aliasing artifats as often laimed in the literatre, bt rather reonstrtion artifats (see, for eample, Ref., pp. 7 8). In onlsion, we see that jaggies have mied origins: in some ases as in Figs. 5 and 6, they are de to the sampling aliasing (althogh even then they may be frther amplified by poor reonstrtion); bt in other ases, like in a low-resoltion osinsoidal grating (Fig. 8), they are a pre prodt of poor reonstrtion. In the first ase, jaggies learly manifest themselves as aliased elements in the spetrm of the sampled signal, bt in the latter ase they do not, sine they only or in a later stage, dring the reonstrtion of the ontinos signal (hene the name postaliasing). Of orse, they wold have appeared in the ontinos-world spetrm of the reonstrted signal, as shown in Fig. (), had we ared to prode sh a spetrm. Finally, let s retrn to the qestion with whih we opened this setion: What is the onnetion between the sampling-inded jaggies and the lassial manifestations of aliasing (masqerading lower-freqenies in the signal domain or spetral repliations in the spetral domain)? Indeed, the most notorios signal-domain manifestation of aliasing onsists of ases where new very low-freqeny strtres appear de to the sampling proess, as shown in Fig. for the -D ase and in Fig. 4() for the -D ase. These new very low-freqeny strtres are simply sampling moirés. Bt the spetral-domain replias de to the sampling proess may also introde new aliased freqenies that are higher than this. These new higher freqenies ontribte to the mirostrtre details of the sampled signal Signal domain s() = III( ) ' ' g() Spetral domain f' ' f ' s s f' s < f ' G f G f G sin( f' ) s ' ret(/ f' ) ' s s()* sin( f' s ) g() () S() = III( ) ' *G() f' s ' S() ' ret(/ f' s ) G() ' d d f s ' f' = s ' Fig. Same as Fig. 9, bt this time we take as or starting point row (d) of Fig. 3 (i.e., a poorly sampled and slightly aliased signal) rather than row () of Fig. 3 (a orretly sampled signal). In sh ases, even a onvoltion with a narrow sin reonstrtion fntion, as we did in Fig. 9, will not be able to reover from the sampled signal the original jagless, ontinos-world signal, sine a mltipliation in the spetral domain with the orresponding ret fntion will not be able to eliminate the aliased freqenies. The poorly sampled and aliased signal of Fig. 3(d). The narrow sin reonstrtion fntion whose spetrm is a ret fntion etending from mins half to pls half of the sampling freqeny f s. () The reonstrted signal [onvoltion of the signals and ] and its spetrm [prodt of the spetra and ]. Note that the reonstrted signal in row () is ontaminated by new high freqenies, and is not idential to the original ontinos signal shown in row of Fig. 3, even thogh we have sed here a good reonstrtion fntion. Bease in this ase the onditions reqired by the sampling theorem are not met in the sampling stage, the reslts stiplated by the theorem for the reonstrtion stage are no longer garanteed (see Appendi A). This figre illstrates the fat that jaggies or other parasite phenomena that are de to poor sampling annot be eliminated by a general reipe improving the reonstrtion proess (for eample, by sing a narrow sin-shaped element, as done in this figre, rather than the narrow ret piel sally sed in the display devies). Jornal of Eletroni Imaging 8-9 Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

11 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial and give it its typial jagged look, as shown for eample in Fig Disssion The fat that jaggies an reslt either from poor sampling or from poor reonstrtion (or both) has a diret impliation on the methods to be sed for their elimination. Clearly, jaggies that are generated by aliasing (poor sampling) will not be eliminated by improvements in the reonstrtion stage (see a spetral-domain illstration in Fig. and a signaldomain illstration in Fig. ). Similarly, jaggies that are only de to poor reonstrtion (as in Fig. 8) will not be eliminated by improvements in the sampling stage, sh as prefiltering of the original signal before its sampling or other antialiasing methods. Ths, whenever it is needed to eliminate jaggies, for eample, dring the developing proess of a new display devie, it is important to first preisely nderstand the origin of the jaggies in qestion before trying to set p an adeqate soltion. The optimal way for eliminating (or at least reding) jaggies on a display devie depends, of orse, on the speifi properties of the devie in qestion, as we an see from the mltitde of patents that ontine to be issed on this sbjet year by year. Of orse, as a general rle, inreasing the devie resoltion whenever this is feasible, will also derease the size of the jaggies, and hene rede their visibility. On very high resoltion devies display artifats sh as jaggies will no longer be visible to the naided eye Fig. Looking at Fig. 8, it may be tempting to say that pielization and jaggies originate from improper reonstrtion (for eample, de to the se of sqare piels rather than narrow -D sin-shaped elements). However, this is not always tre, and sometimes jaggies originate from improper sampling, and no general reipes for improvements in the reonstrtion stage an eliminate them. The present figre illstrates this more intitively, diretly in the signal domain, showing the sampled versions of a slightly rotated binary ( -valed) line in, and of a low freqeny osinsoidal fntion (whose vales vary between and ) in. Both of the sampled signals and are shown before reonstrtion, i.e., as pre implses before their onvoltion with the piel fntion pð;yþ. Note that in the sampled osinsoidal strtre onsists of implses with gradally varying vales, whereas in the sampled slanted line onsists of only - and -valed implses, so that sharp transitions are omnipresent. In the osinsoidal ase, where the signal varies slowly, a onvoltion with a narrow -D sin-shaped piel an yield a smooth, ontinos reonstrted strtre, as stiplated by the lassial sampling theorem. Bt this is hopeless in the ase of the binary line, whih is not band limited and therefore does not meet the onditions of the sampling theorem.. 5 Conlsion Digital devies sh as smartphone displays, ompter displays, printers, et., are being onstantly developed and improved. Rendering of digital tet and graphis on sh devies involves onversions from analog to digital and bak, yielding jaggies as navoidable artifats. Althogh this phenomenon is not new, it still remains an important isse in the design of modern digital display and printing devies. The present ttorial sheds some light on this phenomenon. Based on simple Forier onsiderations it shows that jaggies an be, depending on the ase, either the otome of aliasing, i.e., a sampling artifat, or a reonstrtion artifat. This also has a diret impat on the methods to be sed for the elimination of ndesirable jaggies, sine antialiasing methods, sh as prefiltering, will not eliminate jaggies that are reonstrtion artifats and vie versa. Understanding the real natre of the jaggies in eah ase may therefore help in their elimination (when they are indeed ndesired). Note that or disssion here is ompletely general, and applies also to the three-dimensional (3-D) ase, where jaggies are generated by 3-D printing devies. Appendi A As mentioned earlier, the lassial sampling theorem says that all the information in the original ontinos signal is preserved in its sampled version if the sampling freqeny is at least twie the highest freqeny ontained in the signal. Under this ondition, the theorem garantees (see Fig. 9) that the original ontinos signal an be perfetly reonstrted by mltiplying the spetrm of the sampled signal with a ret fntion that ts off all the freqenies beyond half of the sampling freqeny, or eqivalently, by Jornal of Eletroni Imaging 8- Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use:

12 Amidror and Hersh: Jaggies as aliasing or reonstrtion phenomena: a ttorial onvolving the sampled signal with the Forier transform of this ret fntion, i.e., with the orresponding narrow sin fntion (Ref. 6, p. 83). Figres and show that when the onditions reqired by the lassial sampling theorem are not met in the sampling stage (i.e., when the sampling freqeny is not at least twie the highest freqeny ontained in the original nsampled signal), the reslts stiplated by the theorem for the reonstrtion stage are not garanteed. In sh ases, the narrow sin reonstrtion fntion is not neessarily optimal (see Fig. ). Moreover, in sh ases, there may eist no general reipe for reonstrtion fntions that an reover the original ontinos-world signal (and hene eliminate all jaggies). Yet, the mere fat that the onditions of the lassial sampling theorem (known as Nyqist onditions) are not met does not yet imply that no optimal reonstrtion fntion may eist. Indeed, for ertain families of signals that do not meet the Nyqist onditions there still may eist partilar variants of the sampling theorem that allow to reover the original signal from its samples sing some partilar reonstrtion fntions (see, for eample, Refs. and 3). Bt sh reonstrtion fntions are obviosly not general, and they only work for the respetive families of inpt signals. In onlsion, we see that nothing general an be said abot ases in whih the onditions of the lassial sampling theorem are not met, and the sitation then shold be stdied on a ase-by-ase basis. Readers who wish to deepen their nderstanding of the generalized sampling theory and to break ot of the nrealisti ase of band-limited signals may refer to Ref. 3, whih goes mh beyond the theory reqired for the present ttorial. Aknowledgments This ttorial is an etended version of material that appeared reently in Se. 8.5 of the book, Ref. 9 pblished by Springer. Referenes. F. C. Crow, The aliasing problem in ompter-generated shaded images, Commn. ACM (), (977).. J. D. Foley et al., Compter Graphis: Priniples and Pratie, nd ed., Addison-Wesley, Reading, Massahsetts (99). 3. R. H. Wyman and B. Shoner, Method and system for reding the appearane of jaggies when deinterlaing moving edges, U.S. Patent 8,35,967 (Janary 8 3). K. Iorha et al., Filtering method and apparats for anti-aliasing, U.S. Patent 8,345,63 (Janary 3). 5. Ch. Tremblay, Anti-aliasing of a graphial objet, U.S. Patent 8,94,73 (Otober 3 ). 6. A. V. Tonisson et al., Orientation adapted aliasing anellation, U.S. Patent 8,6,89 (September 4 ). 7. W. Hill et al., Methods and apparats for performing image rendering and rasterization operations, U.S. Patent 6,37,566 (Otober 3 ). 8. T. Dff, Polygon san onversion by eat onvoltion, in Raster Imaging and Digital Typography, J. André and R. D. Hersh, Eds., pp , Cambridge University Press, Cambridge (989). 9. R. Rbinstein, Digital Typography: An Introdtion to Type and Composition for Compter System Design, Addison-Wesley, Reading, Mass. (988).. G. Wolberg, Digital Image Warping, IEEE Compter Soiety Press, California (99).. R. W. Ramirez, The FFT: Fndamentals and Conepts, Prentie Hall, Englewood Cliffs, New Jersey (985).. J. D. Gaskill, Linear Systems, Forier Transforms, and Optis, Wiley, New York (978). 3. E. W. Weisstein, CRC Conise Enylopedia of Mathematis, CRC Press, Boa Raton (999). K. R. Castleman, Digital Image Proessing, Prentie Hall, Englewood Cliffs, New Jersey (979). 5. R. N. Braewell, The Forier Transform and its Appliations, 3rd ed., MGraw-Hill, Boston (). 6. E. O. Brigham, The Fast Forier Transform and its Appliations, Prentie-Hall, Englewood Cliffs, New Jersey (988). 7. I. Amidror, The Theory of the Moiré Phenomenon, Volme I: Periodi Layers, nd ed., Springer, New York (9). 8. R. N. Braewell, Two Dimensional Imaging, Prentie Hall, Englewood Cliffs, N.J. (995). 9. I. Amidror, Mastering the Disrete Forier Transform in One, Two or Several Dimensions: Pitfalls and Artifats, Springer, Berlin (3).. S. Gpta and R. F. Sproll, Filtering edges for gray-sale displays, Compt. Graph. 5(3), 5 (98).. D. P. Mithell and A. N. Netravali, Reonstrtion filters in ompter graphis, Compt. Graph. (4), 8 (988).. M. Mishali and Y. C. Eldar, Sb-Nyqist sampling, IEEE Signal Proess. Mag. 8(6), 98 4 (). 3. M. Unser and A. Aldrobi, A general sampling theory for nonideal aqisition devies, IEEE Trans. Signal Proess. 4(), (994). Isaa Amidror reeived a BS degree in mathematis from the Hebrew University of Jersalem, Israel, and an MS degree in ompter siene from the Weizmann Institte of Siene in Rehovot, Israel. He reeived a Japanese government sholarship for a -year researh period in the Compter Siene Department of the Toyohashi University of Tehnology in Japan. After having worked for a few years in indstry (notably in the fields of laser printing and digital typography), he reeived the PhD degree from the Swiss Federal Institte of Tehnology (EPFL), Lasanne, Switzerland. He is the athor of three books (two volmes on the moiré phenomenon and one on the disrete Forier transform). He also pblished nmeros sientifi papers and is inventor or oinventor of several patent appliations. His researh interests inlde the mathematial fondations of moiré phenomena, serity printing, and image proessing in general. Roger D. Hersh reeived the engineering and PhD degrees from ETH Zrih in 975 and from EPFL in 985, respetively. He is a professor of ompter siene and head of the Peripheral Systems Laboratory at the Eole Polytehniqe Fédérale de Lasanne (EPFL), Switzerland. He has pblished more than 5 sientifi papers and is inventor or oinventor in many patent appliations. His inventions are present in serity doments sh as passports, ID ards, and train tikets. He is interested in novel imaging tehniqes related to olor predition, olor reprodtion, artisti imaging, and serity printing. He is a member of the IEEE Compter Soiety and a fellow of IS&T (Soiety for Imaging Siene and Tehnology). Jornal of Eletroni Imaging 8- Jan Mar 4 Vol. 3() Downloaded From: on //4 Terms of Use: