Browsing by Subject "Information Theory"
Now showing 1 - 4 of 4
Results Per Page
Sort Options
Item Aspects of Interface between Information Theory and Signal Processing with Applications to Wireless Communications(2012-10-05) Park, Sang WooThis dissertation studies several aspects of the interface between information theory and signal processing. Several new and existing results in information theory are researched from the perspective of signal processing. Similarly, some fundamental results in signal processing and statistics are studied from the information theoretic viewpoint. The first part of this dissertation focuses on illustrating the equivalence between Stein's identity and De Bruijn's identity, and providing two extensions of De Bruijn's identity. First, it is shown that Stein's identity is equivalent to De Bruijn's identity in additive noise channels with specific conditions. Second, for arbitrary but fixed input and noise distributions, and an additive noise channel model, the first derivative of the differential entropy is expressed as a function of the posterior mean, and the second derivative of the differential entropy is expressed in terms of a function of Fisher information. Several applications over a number of fields, such as statistical estimation theory, signal processing and information theory, are presented to support the usefulness of the results developed in Section 2. The second part of this dissertation focuses on three contributions. First, a connection between the result, proposed by Stoica and Babu, and the recent information theoretic results, the worst additive noise lemma and the isoperimetric inequality for entropies, is illustrated. Second, information theoretic and estimation theoretic justifications for the fact that the Gaussian assumption leads to the largest Cramer-Rao lower bound (CRLB) is presented. Third, a slight extension of this result to the more general framework of correlated observations is shown. The third part of this dissertation concentrates on deriving an alternative proof for an extremal entropy inequality (EEI), originally proposed by Liu and Viswanath. Compared with the proofs, presented by Liu and Viswanath, the proposed alternative proof is simpler, more direct, and more information-theoretic. An additional application for the extremal inequality is also provided. Moreover, this section illustrates not only the usefulness of the EEI but also a novel method to approach applications such as the capacity of the vector Gaussian broadcast channel, the lower bound of the achievable rate for distributed source coding with a single quadratic distortion constraint, and the secrecy capacity of the Gaussian wire-tap channel. Finally, a unifying variational and novel approach for proving fundamental information theoretic inequalities is proposed. Fundamental information theory results such as the maximization of differential entropy, minimization of Fisher information (Cramer-Rao inequality), worst additive noise lemma, entropy power inequality (EPI), and EEI are interpreted as functional problems and proved within the framework of calculus of variations. Several extensions and applications of the proposed results are briefly mentioned.Item Coding Schemes for Physical Layer Network Coding Over a Two-Way Relay Channel(2013-08-09) Hern, Brett MichaelWe consider a two-way relay channel in which two transmitters want to exchange information through a central relay. The relay observes a superposition of the trans- mitted signals from which a function of the transmitted messages is computed for broadcast. We consider the design of codebooks which permit the recovery of a function at the relay and derive information-theoretic bounds on the rates for reliable decoding at the relay. In the spirit of compute-and-forward, we present a multilevel coding scheme that permits reliable computation (or, decoding) of a class of functions at the relay. The function to be decoded is chosen at the relay depending on the channel realization. We define such a class of reliably computable functions for the proposed coding scheme and derive rates that are universally achievable over a set of channel gains when this class of functions is used at the relay. We develop our framework with general modulation formats in mind, but numerical results are presented for the case where each node transmits using 4-ary and 8-ary modulation schemes. Numerical results demonstrate that the flexibility afforded by our proposed scheme permits substantially higher rates than those achievable by always using a fixed function or considering only linear functions over higher order fields. Our numerical results indicate that it is favorable to allow the relay to attempt both compute-and-forward and decode-and-forward decoding. Indeed, either method considered separately is suboptimal for computation over general channels. However, we obtain a converse result when the transmitters are restricted to using identical binary linear codebooks generated uniformly at random. We show that it is impossible for this code ensemble to achieve any rate higher than the maximum of the rates achieved using compute-and-forward and decode-and-forward decoding. Finally, we turn our attention to the design of low density parity check (LDPC) ensembles which can practically achieve these information rates with joint-compute- and-forward message passing decoding. To this end, we construct a class of two-way erasure multiple access channels for which we can exactly characterize the performance of joint-compute-and-forward message passing decoding. We derive the processing rules and a density evolution like analysis for several classes of LDPC ensembles. Utilizing the universally optimal performance of spatially coupled LDPC ensembles with message passing decoding, we show that a single encoder and de- coder with puncturing can achieve the optimal rate region for a range of channel parameters.Item Direct Information Exchange in Wireless Networks: A Coding Perspective(2011-10-21) Ozgul, DamlaThe rise in the popularity of smartphones such as Blackberry and iPhone creates a strain on the world's mobile networks. The extensive use of these mobile devices leads to increasing congestion and higher rate of node failures. This increasing demand of mobile wireless clients forces network providers to upgrade their wireless networks with more efficient and more reliable services to meet the demands of their customers. Therefore, there is a growing interest in strategies to resolve the problem and reduce the stress on the wireless networks. One strategy to reduce the strain on the wireless networks is to utilize cooperative communication. The purpose of this thesis is to provide more efficient and reliable solutions for direct information exchange problems. First, algorithms are presented to increase the efficiency of cooperative communication in a network where the clients can communicate with each other through a broadcast channel. These algorithms are designed to minimize the total transmission cost so that the communication will be less expensive and more efficient. Second, we consider a setting in which several clients exchange data through a relay. Our algorithms have provable performance guarantees. We also verify the performance of the algorithms in practical settings through extensive simulations.Item Interference Channel with State Information(2012-10-19) Zhang, LiliIn this dissertation, we study the state-dependent two-user interference channel, where the state information is non-causally known at both transmitters but unknown to either of the receivers. We first propose two coding schemes for the discrete memoryless case: simultaneous encoding for the sub-messages in the first one and super-position encoding in the second one, both with rate splitting and Gel'fand-Pinsker coding. The corresponding achievable rate regions are established. Moreover, for the Gaussian case, we focus on the simultaneous encoding scheme and propose an active interference cancellation mechanism, which is a generalized dirty-paper coding technique, to partially eliminate the state effect at the receivers. The corresponding achievable rate region is then derived. We also propose several heuristic schemes for some special cases: the strong interference case, the mixed interference case, and the weak interference case. For the strong and mixed interference case, numerical results are provided to show that active interference cancellation significantly enlarges the achievable rate region. For the weak interference case, flexible power splitting instead of active interference cancellation improves the performance significantly. Moreover, we focus on the simplest symmetric case, where both direct link gains are the same with each other, and both interfering link gains are the same with each other. We apply the above coding scheme with different dirty paper coding parameters. When the state is additive and symmetric at both receivers, we study both strong and weak interference scenarios and characterize the theoretical gap between the achievable symmetric rate and the upper bound, which is shown to be less than 1/4 bit for the strong interference case and less than 3/4 bit for the weak interference case. Then we provide numerical evaluations of the achievable rates against the upper bound, which validates the theoretical analysis for both strong and weak interference scenarios. Finally, we define the generalized degrees of freedom for the symmetric Gaussian case, and compare the lower bounds against the upper bounds for both strong and weak interference cases. We also show that our achievable schemes can obtain the exact optimal values of the generalized degrees of freedom, i.e., the lower bounds meet the upper bounds for both strong and weak interference cases.