In section 7.5 we discuss the implications of the channel coding theorem. The channel coding theorem says that an indefinitely long message can be communicated reliably through the channel when the block length n tends to infinity. This is much stronger than the requirement that the bit error rate tends to zero, because for long block length, even though the bit error rate can be very small, the probability of error of the whole message can very large. The direct part of the channel coding theorem is an existence proof, as opposed to a constructive proof. A randomly constructed code has the following issues. First, encoding and decoding are computationally prohibitive. And second, high storage requirements for the encoder and the decoder. Nevertheless, the direct part of the channel coding theorem implies that when n is large, if the codewords are chosen randomly, most likely the code is good. This is a consequence of the Markov lemma, please see the textbook for the details. The direct part of the theorem also gives much insight into what a good code would look like. In particular the repetition code we discussed at the beginning of the chapter is not a good code because the number of zeros and ones in the codewords are not roughly the same. This is an illustration of a good code. On the right-hand side, there are approximately 2 to the power n times H(Y) typical y sequences. For each codeword that we choose on the left hand side from the typical x set, it is jointly typical with approximately 2 to the power n times H(Y|X) y sequences. This set of y sequences, is represented by a cone. Now we want to pack as many codewords as possible, such that the cones are basically non-overlapping. Therefore, the number of codewords cannot exceed 2 to the power n times the entropy of Y, which is the number of typical y sequences, 2 to the power n, times entropy of Y given X, which is the size of a cone. And this is equal to 2 to the power n times the mutual information between X and Y. And by letting p(x) be the input distribution, that achieves the channel capacity, this is equal to 2 to the power n times C. The Channel Coding Theorem although guarantees the existence of a good code, does not show how to construct a code that is practical. Construction of codes with efficient encoding and decoding algorithms falls in the domain of channel coding theory, which is beyond the scope of this course. The performance of a code is measured by how far the rate is away from the channel capacity. All channel codes used in practice are linear for efficient encoding and decoding in terms of computation and storage. [BLANK_AUDIO] Channel coding has been widely used in home entertainment systems, for example audio CD and DVD, computer storage systems, for example, CD ROM, hard disk, floppy disk, and magnetic tape, computer communication, wireless communication and deep space communication. The most popular channel codes used in existing systems include Hamming code, Reed-Solomon code, BCH code, turbo code, and LDPC code. In particular, turbo code and LDPC code are almost capacity achieving.