CONSIDER the problem of transmitting two correlated

Transcription

1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE Separate Source Channel Coding for Transmitting Correlated Gaussian Sources Over Degraded Broadcast Channels Yang Gao Ertem Tuncel, Member, IEEE Abstract The problem of transmitting a pair of correlated Gaussian sources over degraded broadcast channels using optimal separate source channel codes is studied. Upper bounds are derived for the rate penalty (in terms of channel uses per source symbol) the power loss endured by separate coding compared to joint coding. Although source channel separation is suboptimal in general, it is demonstrated that the performance of separate coding comes close to that of optimal joint coding, especially for low distortion pairs. In fact, in some cases, separate coding performs better than the best known joint schemes so far. It is also shown analytically that separate coding is optimal when either of the sources is to be reconstructed in a near-lossless fashion. Index Terms Channel uses per source symbol, correlated sources, degraded broadcast channels, joint source channel coding, separate source channel coding. I. INTRODUCTION CONSIDER the problem of transmitting two correlated Gaussian sources over a broadcast channel with two receivers, each of which desires to reconstruct (in a lossy manner) only one of the sources. Previous work on this problem exclusively focused on the case the broadcast channel is also Gaussian. In [5], it was shown that for the case of matched source channel bwidths, 1 uncoded transmission, which is the simplest possible scheme, is actually optimal when the signal-to-noise ratio (SNR) is below a certain threshold. Then, various hybrid digital/analog (HDA) schemes have been proposed in [1], [2], [3], [10], [11]. In fact, the HDA scheme in [11] operates with transmission rate 1 (i.e., matched bwidth) achieves optimal performance whenever uncoded Manuscript received October 25, 2011; revised November 13, 2012; accepted February 02, Date of publication February 12, 2013; date of current version May 15, This work was supported in part by the National Science Foundation CAREER Grant CCF The material in this paper was presented in part at the 2011 IEEE International Symposium on Information Theory. Y. Gao was with the Department of Electrical Engineering, University of California, Riverside, CA USA. He is now with Google, Inc., Mountain View, CA USA ( yangg@google.com). E. Tuncel is with the Department of Electrical Engineering, University of California, Riverside, CA USA ( ertem.tuncel@ucr.edu). Communicated by T. Uyematsu, Associate Editor for Shannon Theory. Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /TIT In the sequel, we will refer to the bwidth expansion/compression ratio as the transmission rate measure it in channel uses per symbol. Thus, matched bwidth will translate to rate 1. transmission in [5] does not, thereby leading to a complete characterization of the achievable power-distortion tradeoff. For other transmission rates, the HDA schemes proposed in [1] [3] comprise of different combinations of known schemes using either superposition or dirty paper coding. In all the aforementioned work, authors also compared achieved performances with that of separate source channel coding. Since the Gaussian broadcast channel is degraded, source coding boils down to sending a common message to both decoders a refinement message to the decoder at the end of the better channel. In bothofthetwosource coding schemes proposed in [10], thefirst source is encoded as the common message, but one scheme encodes (as the refinement message) the second source independently, the other after decorrelating it with the first source. In the source coding scheme of [11], on the other h, the second source is encoded after it is decorrelated with the reconstruction of the first source. Although this approach provably yields a better performance than the schemes in [10], it is still not optimal. In [7], it was shown that the optimal rate-distortion (RD) tradeoff in this source coding scenario is in fact achieved by a scheme called successive coding, by both common refinement messages are generated by encoding both sources jointly, instead of using any kind of decorrelation. Although successive coding is a special case of successive refinement for appropriate distortion measures, computation of the RD tradeoff, even for Gaussian sources, turned out to be nontrivial. A Shannon-type lower bound derived for the problem was rigorously shown to be tight in [7], yielding an analytical characterization of the RD tradeoff. In this paper, we investigate the performance of separate source channel coding for any transmission rate when the broadcast channel is constrained to be degraded. Our approach takes two directions. 1) We analyze the rate penalty of optimal separate coding compared to optimal joint coding, defined as the ratio between the minimum required transmission rates of the two regimes. We introduce a channel-independent upper bound on the rate penalty, numerically demonstrate with an example that the rate penalty can be close to 1 (its trivial lower bound) for considerably large regions on the distortion plane. In fact, we analytically show that separate source channel coding is optimal (the rate penalty 1) when either of the sources is to be reconstructed in a near-lossless fashion. We also specialize the channel to 1) the binary symmetric broadcast channel (BSBC) 2) the /$ IEEE

2 3620 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 Gaussian broadcast channel, show that the rate penalty can be better bounded when we know the channel. 2) We analyze the power loss between separate joint coding when the channel is Gaussian, the loss is defined as a ratio of minimum required power levels for fixed transmission rates. Similar to the rate penalty case, separate source channel coding is not only demonstrated to be very competitive, but it is also shown to be optimal when one source is to be reconstructed with vanishing distortion the other with small enough distortion. Furthermore, for transmission rates other than 1, the optimal (joint) coding scheme is not known, optimal separate coding can outperform the joint coding schemes laid out in [1] [3], which are the best known schemes so far. Crucial to both directions above is the distortion outer bound introduced in [10] for transmission rate 1 generalized in [3] to other rates. The outer bound is based on the fictitious genieaided scenario the better receiver has a lossless copy of the source that is to be reconstructed by the other receiver. For Gaussian broadcast channels, it was shown in [10] that this genie-aided scenario is separable, thus allowing a straightforward characterization of the achievable distortion region ( hence of the outer bound to the original problem since there is in fact no such copy available). We show here that the genie-aided problem is in fact separable for all degraded channels, allowing us to proceed with direction 1. Our results are in the same spirit with those in [12], which proves optimality near-optimality of separate source channel coding in certain families of scenarios. In fact, as suggested by Tian [13], the present scenario is a special case of what the authors of [12] termed joint source channel multiple multicast with distortions. To see that connection, the two sources must be viewed as a single vector source, as was done in [8], the distortion region with individual distortion criteria must be characterized as a union of distortion regions under the covariance distortion measure. We do not pursue this connection further in this paper for two reasons: First, since we are studying one of the few network scenarios the performance of separate source channel coding is analytically known, we want to fully enjoy this knowledge by producing the more specific results indicated above in directions 1 2. Second, we want to base our comparison between separate joint coding on a single number (i.e., rate penalty power loss), as in [12], the loss in performance is indicated either in terms of genie-aided bits or distortion loss at each receiver. The rest of this paper is organized as follows. In Section II, we start with the preliminaries background for the problem. We then provide a simplified characterization for the successive coding RD region in Section III, revisit the genie-aided scenario in Section IV. Rate penalty of separate coding is analyzed in Section V, for Gaussian broadcast channels, the power loss is discussed in Section VI. Section VII concludes this paper. II. PRELIMINARIES AND BACKGROUND As depicted in Fig. 1, we are interested in the communication scenario a pair of correlated Gaussian sources Fig. 1. General communication scenario a correlated Gaussian source pair is transmitted to two receivers using a degraded broadcast channel. Each receiver is interested in only one source. generated in an i.i.d. fashion using is broadcast to two receivers, receiver, is to reconstruct only. The broadcast channel is memoryless degraded, 2 is governed by, i.e., receiver 2 is stronger than receiver 1. Let denote the channel input alphabet denote the channel output alphabet at receiver. Without loss of generality, we let with. The encoder at the transmitter maps the source sequences to decoder,the at each receiver reconstructs. We refer to as the transmission rate in terms of channel uses per source symbol. The reconstruction quality at each receiver is measured by the mean square error. Significant part of our results is about Gaussian broadcast channels, in which case average input power is also constrained. Definition 1: Atuple is achievable if for every, there exists such that for large enough. If there is no input power constraint, achievability of is defined similarly without the last inequality above. We will drop the superscript from the encoders decoders whenever there is no room for confusion. 2 Using stard arguments, it can be shown that we can safely assume that the channel is physically degraded even if it is only stochastically degraded.

3 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3621 A. Separate Source Channel Coding In this paper, we focus exclusively on separate source channel coding. Naturally, in this setting, it suffices to analyze schemes in the source coder maps the source sequences to a common message to be decoded by both receivers a private message to be decoded only by the strong receiver. More formally, the encoder becomes, with for, Similarly, the decoders can be written as similarly without the last inequality above, the achievability region is denoted simply as. It should be clear that (or, respectively, ) is achievable using separate source channel coding if only if (1) for any region, is the union of all for which. For (1) to be useful, we need to underst both the capacity region the RD region. Sections II-B II-C will be devoted to these regions. B. Capacity Region for the Degraded Broadcast Channel The capacity region for the degraded broadcast channel has been characterized in [4], [6] as the union of all such that there exists satisfying We denote by,themessage in chosen by the source encoder to be fed to the channel encoder. We also use the notation for the decoded message at receiver (leaving undefined). Definition 2: A source coding RD tuple is achievable if for every, there exists such that forms a Markov chain. There are two widely known well-investigated examples of degraded broadcast channels. The first one is the Gaussian case, in which each receiver observes, with. The capacity region in this case can be computed as the set of all pairs such that there exists satisfying (2) (3) weusethenotation for any scalar. The second example is the BSBC, the two receivers have crossover probabilities, respectively, with. The capacity region of the BSBC can be computed as the set of all pairs such that there exists satisfying for large enough. The set of all achievable for given is denoted by. Definition 3: A channel coding rate-power tuple is achievable if for every, there exists such that is the binary entropy function for large enough. The set of all achievable (i.e., the capacity region) for given is denoted by.ifthereis no input power constraint, achievability of is defined C. Source Coding Schemes the RD Region Before we start discussing source coding schemes that have appeared in the literature, we pointthatinatwo-layeredscheme like the one we are studying here, one can always transfer some rate, say, from the private message to the common message. That is, if is an achievable source coding rate pair (with other parameters such as distortion fixed), then so is. This is because the decoder that receives only the common message can always ignore the transferred bits, the one that receives both messages can simply transfer those bits back to the private message before decoding.

4 3622 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 This observation implies that, in general, we should characterize the source coding rate region in terms of cumulative rates, i.e., in terms of all achievable. However, as proved in [14], when the two-layered source code is to be used in conjunction with a degraded broadcast channel code, we can always focus on marginal rates without losing generality. Several separation-based schemes have been previously proposed for the Gaussian channel case, differing only in their source coding strategy. In the first scheme (Scheme A in [10]), sources are encoded as if they are independent, resulting in the source coding rate region for any given by In Scheme B of [10], the second source is written as,, are treated as two new independent sources. Hence, we obtain 3 Note that by construction Scheme B can only achieve the distortion pairs satisfying. Also, Scheme B achieves asmaller ifonlyif. In the separation-based scheme introduced in [11], which we call Scheme C, is quantized to is then encoded conditioned on (in a Wyner Ziv fashion [15]). The resultant source coding rate region becomes (4) (5) (6) (7) Theorem 1: The RD tradeoff for the successive coding problem is defined parametrically with respect to as 4 with,, is the unique root of (10) (11) (12) in the interval. Remark 1: By close inspection, one can conclude that (10) (11) degenerates to (8) (9) when.thus,scheme C is a special case of successive coding. D. A Related Source Coding Problem In a closely related source coding problem introduced in [16], the source pair is to be transmitted over a point-topoint channel. The quality of the reconstruction at the receiver is then measured using individual squared-error distortion criteria for. It was shown in [16] that the behavior of the RD function varies in different regions on the -plane shown in Fig. 2(a). These regions are defined as (8) (9) Of the three schemes, it can easily be shown that Scheme C achieves the best performance. However, it is also obvious that the optimal scheme should make use of all the information available at the encoder. That is, instead of using any kind of decorrelation between the sources, the transmitter should encode the sources jointly both at the common the refinement layers. But this exactly coincides with what is known as the successive coding problem, which was introduced completely solved for Gaussian sources in [7]. In contrast with Schemes A C for which the RD characterization has all but one corner point describing the whole region as evidenced in (4) (9), the RD region for successive coding is best characterized parametrically as in the following theorem (cf.,[7,th.2,eqs.(76)(77)]. 3 When, the optimal strategy degenerates into sending only a common message estimating solely from for Schemes B C, as well as for successive coding. This trivial case is excluded from the discussion in the sequel. areshowninfig.2(a)for. These regions also play a role in our problem setting, as will be explained in the next section. E. The Genie-Aided Problem the Corresponding Outer Bound Consider the scenario shown in Fig. 3, which will be referred to as the genie-aided problem. The difference with the original scenario in Fig. 1 is that the second (strong) receiver has a lossless copy of. Obviously, the optimum performance here (in terms of the -tradeoff) provides us with an outer bound to that of the original (which we refer to as the genie-aided outer bound), because extra information can never degrade the reconstruction quality. The performance of this scenario was computed for the Gaussian channel case in [10] when 4 Again, this is ignoring the possibility of a rate transfer.

5 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3623 Fig. 4. Rate regions of various source coding schemes, namely, Schemes A C, successive coding, the genie-aided scheme, shown together with the capacity region of the Gaussian broadcast channel with noise levels for various. More specifically, starting from, either or is increased until intersects with rate regions of the genie-aided successive coding schemes. (14) We will come back to this problem later in Section IV show that it is in fact a separable problem for any arbitrary degraded channel. Fig. 2. (a) Regions on the -plane for which the source coding problem of [16] shows different behavior. (b) Feasible ranges of in those regions. The significance of the dashed line is that above the line, in the genie-aided problem discussed in Section IV. Fig. 3. Genie-aided system. The better receiver has a free lossless copy of the source destined for the worse receiver., was later generalized in a straightforward fashion to any in [3, Lemma 1] as follows. Lemma 1: The tuple is achievable in the genie-aided problem if only if there exists such that (13) F. Comparison of Source Channel Rate Regions Examples of rate regions for the source coding schemes mentioned in Section II-C are shown in Fig. 4, together with the channel capacity region for various.atarget is chosen for a Gaussian source pair with. Note that although Scheme B can achieve this distortion pair (since ), its source coding rate is too large to display in the figure. Asshowninthefigure, the boundary of the successive coding source rate region consists of three segments, they are the segment, ;thecurve for as given in Lemma 2; the line connecting to.thefirst segment is degenerate the third segment is merely obtained by rate transfer. The single achievable points shown in the figure for the genie-aided problem, Scheme A, Scheme C can similarly be extended to a region whose boundaries consist of segments of the first the third kind. On the channel coding side, a Gaussian broadcast channel with is chosen to exemplify how grows with increasing or (starting from ), how much of a gap occurs between the genie-aided outer bound the successive coding scheme in terms of the minimum required or. As can be seen from Fig. 4, when the power is fixed at,theminimum necessary to make intersect with the genie-aided rate region is 0.7, as is the minimum required transmission rate for successive

6 3624 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 coding. On the other h, when is fixed, the required power jumps from to. The latter gap seems more significant, which is simply because is a multiplicative factor as power is a logarithmic one. G. Joint Source Channel Coding With Rate For, i.e., when,itwasshownin[5]thatthe uncoded transmission scheme of can be found by varying power allocation scaling coefficients. For, an HDA scheme from [9] for broadcasting a common source was adapted for the problem of broadcasting correlated sources was termed the Reznic Feder Zamir (RFZ) scheme in [2]. In the RFZ scheme, is achievable for a fixed if only if there exists a power allocation such that (15) is optimal (with a proper choice of ) in a certain low-snr regime. Later, Tian et al. [11] showed that in all other cases, the HDA scheme given by is optimal, is the quantized version of, are also chosen properly. For fixed, the set of achievable was summarized in the following theorem from [11]. Theorem 2: If,then (16) In addition, a scheme, termed the HWZ scheme, containing an analog layer two digital layers each with a Wyner Ziv coder a channel coder, was also proposed in [2] [3]. In the HWZ scheme, is achievable for a fixed if only if there exists a power allocation such that (17) for, is defined as in the equation at the bottom of this page. On the other h, if,then (18) In this theorem, is the performance of uncoded transmission, is that of HDA coding. By close inspection, can be shown to coincide precisely with the genieaided outer bound in (13) (14) with. It was in fact this observation in [11] that lead to the optimality of HDA coding. H. Joint Source Channel Coding With Rate In [1] [3], a group of HDA schemes were proposed for, analog, digital, hybrid schemes are layered with superposition or dirty paper coding, the achievable region It was argued using a numerical example in [2] that the HWZ scheme performs the same as (or similar to) the RFZ scheme. Itis,infact,notsodifficult to see that they perform exactly the same analytically: Comparing (17) (18) to (15) (16), it suffices to show that for any, (19) that equality is satisfied for some. But it is easy to see that (19) is always true with satisfying equality. Therefore, in Section VI-B, wecomparetheperformance of separate coding to joint source channel coding, we will simply use the RFZ region in (15) (16). For, special attention was paid to in [1] [3], out of three proposed schemes, the best one was observed to have the following characterization: For a fixed

7 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3625, is achievable if only if there exists a power allocation parameters such that The numerator above is a quadratic function of centered at with a maximum value of.thus, in implying that is monotonically decreasing in the same interval. Similarly, when,wehave.thistime, since is decreasing, we examine obtain is the same as above. III. SIMPLIFIED CHARACTERIZATION OF THE SUCCESSIVE CODING RD REGION Our first contribution is the following lemma, which shows that the RD region of successive coding can be simplified by eliminating both the parameter the need to find the relevant root of the cubic polynomial. Lemma 2: For any distortion pair, the boundary of the achievable source coding rate region is given by The numerator is centered at achieves the maximum value. Hence, is monotonically decreasing in the interval. We finish the proof by two observations: 1) in the corner case, the interval degenerates into, is indeed a root; 2) when, the value of is irrelevant because of (12), it suffices to consider only. Remark 2: The regions described in Section II-D determine the feasible range of in Lemma 2. The corresponding ranges are as shown in Fig. 2(b). (20) IV. GENIE-AIDED PROBLEM REVISITED with (21) Using the capacity of the Gaussian broadcast channel given in (2) (3), the region in Lemma 1 may be put in the alternative form (22) Proof: Since (20) (21) are the same as (10) (11), respectively, it suffices to show that there is a one-to-one mapping between.theresult then follows from the relation between in (12). Toward that end, we rewrite as denotes the stard RD function is the conditional RD function which can be computed as Observing that, the proof will be complete after showing that is a decreasing function in the interval. Now, when,wehave.since is already decreasing in, we only examine differentiation of which yields (23) (24) The alternate expression (22) reveals something that may have gone unnoticed in [3] [10]: The genie-aided problem is actually separable. That is because the seemingly naive scheme of sending bits/source sample to the first receiver bits/source sample to the second receiver using an appropriate channel code is the best one can do even when joint source channel coding is allowed. Interestingly, if this scheme is used, although the second receiver can also decode

8 3626 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 whatever is sent to the first receiver, it can simply choose not to throw away the decoded bits. We next prove that the genie-aided problem is actually separable for any any degraded channel. We defer the proof to Appendix A. Lemma 3: For any degraded broadcast channel with capacity region, the distortion pair is achievable in the genieaided scenario if only if In the following sections, we will analyze in more detail the transmission rate penalty the power loss of successive coding with respect to the genie-aided scheme. The rate penalty power loss between the two will then naturally become upper bounds for those between separate joint source channel coding schemes. The separability of the genie-aided problem will prove crucial for our results in Section V, because it will allow us to bound the performance loss of separate coding with respect to joint source channel coding in terms of transmission rate for an arbitrary degraded channel. V. RATE PENALTY OF SEPARATE CODING In this section, we first give a channel-independent upper bound of the transmission rate penalty of optimal separate source channel coding over optimal joint source channel coding. We then analyze the specific rate penalties for BSBC Gaussian broadcast channels. A. Upper Bound on the Rate Penalty When we have the freedom to choose the transmission rate, we have the following relation for any given source pair desired distortion levels, ( power constraint if any): (25) is the minimum achievable transmission rate for joint source channel coding,, respectively, denote the minimum achievable for separate coding (i.e., such that (1) is satisfied) for the genie-aided problem (i.e., such that (13) (14) are satisfied). For example, in the case shown in Fig. 4,,, is an unknown value between the two. We define the rate penalty of separate coding over joint coding as the ratio, which can be bounded using (25) as Fig. 5. Illustration of the upper bound on the rate penalty. For this specific example, the intersection point is on the second segment of the successive coding curve, i.e., for. falls on the boundary of for any channel. 5 We then connect the origin to the point withaline extend the line until it intersects the successive coding rate curve, say at, as shown in Fig. 5. Now, define the ratio (27) Since, by construction, will intersect with the successive coding curve, we have which means that (26) can be completed to Since only depends on the sources the desired distortion values, it constitutes an upper bound which is universal, i.e., independent of the actual channel ( of course the power constraint, if any). The value of is summarized in the following lemma. Lemma 4: For any degraded channel, the upper bound on is given by (26) Since is unknown in general, we instead rely on (26) to underst the rate penalty. Both can be computed for any degraded channel with a known capacity region. However, we can further upper bound even without knowing anything about the channel. To obtain this upper bound, we first notice using Lemma 3 that the genie-aided source rate pair 5 will be denoted here as,respectively, for simplicity.

9 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3627 is the solution of (28) see that as achieves. Remark 4: In, we always have because Proof: If, the line connecting the origin intersects with the second segment of the successive coding curve, i.e., at a point with some, as shown in Fig. 5. Then, using (20), (21), (23), (24), the equality can be rewritten as (29) (30) Now, can be eliminated to obtain (28) by first subtracting (30) from (29) to write (31) then by using (29) to substitute into (31). When, the line from the origin to intersects with the third segment of the successive coding curve, which is also a line obtained using rate transfer starting from the point. Therefore, the intersection point is the solution of Solving for substituting into (27) yields As shown in the following corollary to Lemma 4, in some cases, approaches 1, implying separate coding is actually optimal. Corollary 1: Separate source channel coding achieves the optimum performance when either of the sources needs to be recovered almost losslessly, i.e.,. Proof: According to the regions in Fig. 2, we divide the two conditions to 4 cases as follows: 1) 2) 3) 4) Both conditions 1 2 occur in region,thus, implying that for some such that, It is then easy to see that the upper bound on above converges to 1 for either or. Condition 3 is in the degenerate region discussed in Remark 3. Condition 4 falls in region at bottom right of the distortion plane,, as indicated in Lemma 2, either or.if, the same exact argument for condition 2 applies. If,since,wehave When, first observe that. Second, as indicated in Fig. 2,. Finally, from (21),. It then follows that the intersection point is given by.thus When, we still have but this time, as shown in Fig. 2. Also, implying that the intersection point is again on the third segment of the successive coding curve is given by.butthen Remark 3: The case is excluded due to its degeneracy. But for the sake of completeness, one can immediately whichalsogoesto1when. Remark 5: Note that when, the optimality of separate coding actually follows from previous results because in this case, the strong receiver can also losslessly recover the first source, the problem effectively reduces to the genie-aided setup, which is separable. In Fig. 6, an example of over the distortion plane is shown with. Even though the maximum value of is higher than 50, it is very close to 1 in the entire region less than 1.2inmostof, i.e., separate coding requires no more than 20% excess channel uses. However, since is only an upper bound of the rate penalty, even in other regions, the actual rate penalty might be much smaller than. In the following sections, we will evaluate the upper bound on the rate penalty for BSBC Gaussian broadcast channel demonstrate how the upper bound might be tightened when we know the channel.

10 3628 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 Fig. 6. Channel-independent rate penalty upper bound over the distortion plane. The color-value mapping is made nonlinear intentionally to emphasize the values close to 1. Fig. 7. Rate penalty over the distortion plane for BSBC. for reliable trans- B. Binary Symmetric Broadcast Channel For BSBC, the condition mission of source bits can be translated into for some, or equivalently, Since the two expressions inside the maximum above have opposite monotonicity with respect to, the minimum transmission rate can be found by solving for the value that equates thetwo.morespecifically, to find,wesolve to find,wefirst solve Fig. 8. Rate penalty over the distortion plane for Gaussian broadcast channel. for some,thus for every then find. As depicted in Fig. 7, for the given BSBC, the rate penalty has a maximum value of about 30, which is lower than the more general rate penalty bound showninfig.6. C. Gaussian Broadcast Channel Similar to the BSBC case, for a Gaussian broadcast channel characterized by, translates into (32) (33) Similar to the BSBC case, we can compute both using the opposite monotonicity of the two expressions with respect to. The resultant rate penalty is depicted in Fig. 8. Again, the rate penalty is smaller than, as expected. VI. POWER LOSS FOR THE GAUSSIAN BROADCAST CHANNELS In separate coding with a fixed transmission rate,theregion of all achievable triplets can be determined using one of two methods. The conventional method fixes searches for the lower envelope of all whose source rate region intersects with the capacity region given in [4]. Alternatively, we can fix search for the minimum

11 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3629 whose corresponding capacity region intersects with the source rate region given in Lemma 2. We find this alternative both more convenient more meaningful. More specifically, it is easier to compare schemes based on the minimum power they need to achieve the same distortion pair, the ratio of minimum powers yields a single number as a quality measure. As illustrated in [11] for, the genie-aided outer bound in (13) (14) is not always tight. However, we can still utilize it to upper bound the power loss for any as For the case of, the minimum power of separate coding can actually be computed analytically for any. Lemma 6: For, the minimum required power for any is given by (38) Toward this end, we first compute the minimum required power for the genie-aided problem. Note that when,(14) will hold for any, hence, the minimum power is obtained solely from (13), as when,theminimum power satisfies equality in both (13) (14). Combining the two cases, we obtain the concise expression (34) To compute,wefirstneed to find the minimum required power for any given source coding rate pair. Lemma 5: For any source coding rate pair,theminimum required power is given by (35) Proof: Inequalities (32) (33) imply that is achievable if only if there exists such that The proof of the lemma is deferred to Appendix B. In Section VI-A, we analytically evaluate bound, thereby further bounding the power loss, for some special cases. In Section VI-B, we numerically compare the performance of successive coding scheme with various other schemes (based on separate joint coding) over the entire distortion plane. A. Analytical Results on Power Loss The following theorem is the counterpart of Corollary 1 for Gaussian broadcast channels in terms of power loss instead of rate penalty. Theorem 3: Separate source channel coding achieves optimal power-distortion tradeoff when satisfies either of the following conditions: 1), 2). Proof: From (37), we have (36) Since the terms in the maximum exhibit opposite monotonicity with respect to with asymptotes at, the minimum power is achieved when the two terms are equal, that is, when Using from (34), it is easy to see that when Substituting into either expression on the right-h side of (36) yields the desired result. Now, by substituting (20) (21) of Lemma 2 into (35), we obtain the minimum power required for separate coding as a function of : Since is feasible, the minimum power of separate coding satisfies. Therefore, sufficiency of condition 1 follows. Similarly, by setting,wehave (37) when,. Note that when,, which again implies, thus proving the second part of the theorem.

12 3630 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 Remark 6: Similar to the rate penalty analysis, sufficiency of condition 1 naturally follows as discussed in Remark 5. Remark 7: Here, we proved that the genie-aided outer bound is tight in the region when either or goes to 0, the performance of separate coding approaches the outer bound. The condition that either or goes to 0 translates to infinite channel SNR. In fact, as we show in the following theorem, separate coding is approximately optimal for the entire region,inthesense that the power ratio can be upper-bounded universally in. Theorem 4: When, Proof: The first half of the bound is true because Fig. 9. Upper bounds of the power loss in decibels. Bound 1 in the figure is the first half of the bound in Theorem 4 Bound 2 is the second. Thus, using (34) follows since is in the feasible range, follows by, follows since, since. Since is also feasible in (because of ), the second half of the bound can be obtained in a similar way as follows. First, we have the last inequality comes from the fact that maximizes the right-h side. Combining the two bounds, we have the result in the statement of the theorem. A typical example of the performance of the upper bound of Theorem 4 is shown in Fig. 9. In addition to the two halves of the bound as a function of, what is also shown is an envelope for all values of for varying. As can be seen from the figure, for this specific example, there is about less than 1 db gap between the upper bound the envelope. In general, the first half of the bound is tighter for smaller, as the second half is tighter for larger. B. Numerical Comparisons for Power Loss We begin by discussing how much improvement is made by adopting the optimal separate source channel coding scheme,

13 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3631 Fig. 10. Power loss between (a) optimal separate coding the genie-aided outer bound (b) Scheme C optimal separate coding. The parameters are set to,,,. namely successive coding, instead of Scheme C. The minimum required power of Scheme C can be obtained from (8) (9) as (39) For an example setting, we show the power ratio between optimal separate coding the genie-aided outer bound, i.e.,, in Fig. 10(a), that between Scheme C optimal separate coding, i.e.,, in Fig. 10(b), both in decibel scale. As implied by Theorem 4, the region exhibits a small. Also, the optimal separate coding scheme does not require too much extra power in most of. Again, since the genie-aided outer bound is not always tight, the large power difference in may be dramatically reduced when is replaced by the (yet) unknown (as does happen for the case discussed later below). There is also noticeable power loss between the optimal separate coding scheme Scheme C, we numerically observed the largest decibel values near the point. Analytically, we can evaluate how large can become at that point. Lemma 7: When Proof: Straightforward using Theorem 3 to conclude then comparing (34) (39). We next demonstrate the exact power loss how it compares to for the special case of. Note that in this caseweknow exactly due to Theorem 2. For an example setting, Fig. 11(a) shows Fig. 11(b) shows.as seen from Fig. 11(b), the maximum exact power loss is 0.45 db for this example. Also observe how loose can become in region compared to the actual loss. For, we compare our separate coding scheme with the gene-aided outer bound the RFZ/HWZ scheme in Fig. 12. For this comparison, we first revert to the more familiar plot for the exact same as those used in the examples in [2] [3]. As seen in Fig. 12(a), when is small, the separate coding scheme almost coincides with the outer bound outperforms RFZ/HWZ schemes. When the sources are highly correlated as in Fig. 12(b), the separate coding scheme is still better than the RFZ/HWZ schemes when is lower than a certain value, also provides competitive performance when it is higher. We also conduct power loss comparison for the entire distortion plane for the same set of parameters between RFZ/HWZ scheme separate coding. As can be seen from Fig. 12(c) (f), when, separate coding is very close to the genie-aided outer bound is better than RFZ/HWZ scheme in most of the distortion plane, at some distortion pairs by 9 db. Even in the region RFZ/HWZ scheme performs better (marked with white dashed lines), the difference is rather small. When, RFZ/HWZ scheme outperforms separate coding in a larger region, but again not by much. On the contrary, when separate coding is better, it can be better by as large as 5 db. VII. CONCLUSION We analyzed the performance difference between optimal separate joint source channel coding schemes for broadcasting two correlated Gaussians to two receivers. We first defined the rate penalty of optimal separate coding compared to optimal joint coding. Then, using a channel-independent upper bound we defined on the rate penalty, we were not only able to demonstrate that it can be close to 1 for very large regions on the distortion plane, but we also showed analytically that it approaches 1 as either of the two distortion levels approach 0, implying optimality of separate coding. For Gaussian channels, we then defined the power loss between separate joint coding obtained similar results as in the rate penalty case: Separate coding is very competitive in

14 3632 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 Fig. 11. Power loss between (a) optimal separate coding the genie-aided outer bound (b) optimal separate coding optimal joint coding. The parameters are set to,,,. most of the distortion plane can be shown to be optimal when one source is to be reconstructed with vanishing distortion the other with a small enough distortion. We also constructed an upper bound on the power loss in a certain low distortion region which is universal in channel quality distortion levels. Finally, for transmission rates other than 1, we demonstrated that optimal separate coding can outperform HDA-based joint coding schemes which are the best performing so far. APPENDIX A PROOF OF LEMMA 3 It suffices to show that is achievable only if with, follows from the Markov chain, from the fact that conditioning reduces entropy. Notice that forms a Markov chain. Also (43) Toward that end, we will use the stard relations that for any, (40) (41) if is achieved by. Now, we have the following chain of inequalities: (44) follows because given, forms amarkovchain, follows because conditioning reduces entropy, follows because is a Markov chain, follows because is a Markov chain. Bringing (40) (44) together, we have (42)

15 GAO AND TUNCEL: SEPARATE SOURCE CHANNEL CODING FOR TRANSMITTING CORRELATED GAUSSIAN SOURCES 3633 Fig. 12. Comparison between the genie-aided outer bound, RFZ/HWZ scheme in [2] [3], separate coding.,,.in(c)(e),theareawithinthewhite-dashed lines correspond to.(a) (b) (c) for (d) for (e) for (f) for. Since for all,since is a convex region, we have the desired result. When APPENDIX B PROOF OF LEMMA 6, (37) becomes

16 3634 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 59, NO. 6, JUNE 2013 After some algebra, the derivative can be found as in the equation at the top of the page, using which we can conclude that 1),2) has only two solutions given by with givenasinthelemma,finally 3) since the coefficient of in the numerator of is negative, must be transitioning from negative to positive at from positive to negative at,orinotherwords,, thus only is relevant. The proof will then be complete once we show that,,, because then the minimum would be achieved by Towards that end, let rewrite Now, as isthesameas which is granted. Similarly, obviously true statement can be rewritten as the REFERENCES [1] H. Behroozi, F. Alajaji, T. Linder, Hybrid digital-analog joint source-channel coding for broadcasting correlated Gaussian sources, presented at the IEEE Int. Symp. Inf. Theory, Seoul, Korea, Jun [2] H. Behroozi, F. Alajaji, T. Linder, Broadcasting correlated Gaussian sources with bwidth expansion, presented at the IEEE Inf. Theory Workshop, Taormina, Italy, Oct [3] H. Behroozi, F. Alajaji, T. Linder, On the performance of hybrid digital-analog coding for broadcasting correlated Gaussian sources, IEEE Trans. Commun., vol. 59, no. 12, pp , Dec [4] P. Bergmans, A simple converse for broadcast channels with additive white Gaussian noise, IEEE Trans. Inf. Theory, vol. IT-20, no. 2, pp , May [5] S.Bross,A.Lapidoth,S.Tinguely, BroadcastingcorrelatedGaussians, presented at the IEEE Int. Symp. Inf. Theory, Toronto, ON, Jul [6] T. Cover, Broadcast channels, IEEE Trans. Inf. Theory, vol. IT-18, no. 1, pp. 2 14, Jan [7] J. Nayak E. Tuncel, Successive coding of correlated sources, IEEE Trans. Inf. Theory, vol. 55, no. 9, pp , Sep [8] J. Nayak, E. Tuncel, D. Gunduz, E. Erkip, Successive refinement of vector sources under individual distortion criteria, IEEE Trans. Inf. Theory, vol. 56, no. 4, pp , Apr [9] Z. Reznic, M. Feder, R. Zamir, Distortion bounds for broadcasting with bwidth expansion, IEEE Trans. Inf. Theory, vol. 52, no. 8, pp , Aug [10] R. Soundararajan S. Vishwanath, Hybrid coding for Gaussian broadcast channels with Gaussian sources, presented at the IEEE Int. Symp. Inf. Theory, Seoul, Korea, Jun [11] C. Tian, S. Diggavi, S. Shamai, The achievable distortion region of sending a bivariate Gaussian source on the Gaussian broadcast channel, IEEE Trans. Inf. Theory, vol. 57, no. 10, pp , Oct [12] C. Tian, J. Chen, S. Diggavi, S. Shamai, Optimality approximate optimality of source-channel separation in networks, IEEE Trans. Inf. Theory, submitted for publication. [13] C. Tian, Private Communication. [14] E. Tuncel, The rate transfer argument in two-stage scenarios: When does it matter?, presented at the IEEE Int. Symp. Inf. Theory, Seoul, Korea, Jun [15] A. D. Wyner J. Ziv, The rate-distortion function for source coding with side information at the decoder, IEEE Trans. Inf. Theory, vol. IT-22, no. 1, pp. 1 10, Jan [16] J. -. Xiao Z. -. Luo, Compression of correlated Gaussian sources under individual distortion criteria, in Proc. 43rd Allerton Conf. Commun. Control Comput., Sep. 2005, pp Finally, is always true because it is the same as Yang Gao received the B.S. degree in electrical engineering from Tsinghua University, China, in 2004, M.S. degree from the Chinese Academy of Sciences in 2007, Ph.D. at the University of California, Riverside 2012 with the supervision of Prof. E. Tuncel. He is currently with Google Inc. which is also readily granted. ACKNOWLEDGMENT The authors would like to thank the reviewers the Associate Editor for their suggestions that greatly improved the paper in readability. Ertem Tuncel (S 99 M 04) received the Ph.D. degree in electrical computer engineering from University of California, Santa Barbara, in In 2003, he joined the Department of Electrical Engineering, University of California, Riverside, he is currently an Associate Professor. His research interests include rate-distortion theory, multiterminal source coding, joint sourcechannel coding, zero-error information theory, content-based retrieval in high-dimensional databases. Dr. Tuncel received the National Science Foundation CAREER Award in 2007.