Lower Bounds on Mutual Information for Linear Codes Transmitted over Binary Input Channels, and for Information Combining

It has been known for a long time that the mutual information between the input sequence and output sequence of a binary symmetric channel (BSC) is upper bounded by the mutual information between the same input sequence and the output sequence of a binary erasure channel (BEC) with the same capacity. Recently, Samorodnitsky discovered that one may also lower bound the BSC mutual information in terms of the mutual information between the same input sequence and a more capable BEC. In this paper, we strengthen Samorodnitsky's bound for the special case where the input to the channel is distributed uniformly over a linear code. Furthermore, for a general (not necessarily binary) input distribution PX and channel WY|X, we derive a new lower bound on the mutual information I(X;Yn) for n transmissions of X∼PX through the channel WY|X.