We are going to skip section 3.2 because the discussions therein are subsumed by the discussion in section 3.3. In this section, we are going to make rigorous the analogy between information theory and set theory by the construction of a sign measure called the I-measure, denoted by mu star. Let tilde{X} be a set corresponding to a random variable X. Fix n and let N_n be the set of integers 1, 2, up to n. Now we define the universal set omega to be the union of all i, tilde{X_i}. Then the atom A_0, defined as the intersection of all i, tilde{X_i} complement, which by de Morgan's law can be written as the union of all i tilde{X_i} complement. Now we see that the union of all i tilde{X_i} complement is simply equal to the universal set that we have defined. And so, this can be written as omega complement, which by definition is equal to the empty set. For this reason, we call the atom A_0 the empty atom of F_n. We let A be the set of all the other atoms of F_n, called the non empty atoms of F_n. Note that the size of A is equal to 2 to the power n minus 1, because we exclude the empty atom. We have seen that a signed measure mu on F_n is completely specified by the values of mu on the atoms of F_n, but because of our special definition of the universal set, the atom A_0 is always empty, and so mu(A_0) is always equal to 0. For this reason, a signed measure mu on F_n is completely specified by the values of mu on the set of all non-empty atoms of F_n, namely the set A. We now look at the simplest example with n equals 2. Here omega is equal to tilde{X_1} union tilde{X_2}. The empty atom A_0 is equal to tilde{X_1} complement intersect tilde{X_2} complement. This is shown in the Venn diagram at the bottom. Now because A_0 is always empty, we can simplify the Venn diagram to the information diagram shown on the right. For simplicity, the set variable tilde{X_1} is shown as X_1 on the information diagram, so on and so forth. In this information diagram the non empty atoms are tilde{X_1} intersect tilde{X_2}, tilde{X_1} complement intersect tilde{X_2}, and tilde{X_1} intersect tilde{X_2} complement. These are indicated in the information diagram. Note that the size of A, namely the number of non-empty atoms, is equal to 2 to the power 2 minus 1, which is 3. Then, a signed measure mu on F_2 is completely specified by the values of mu on the atoms of A, namely, mu on tilde{X_1} intersect tilde{X_2}, mu on tilde{X_1} complement intersect tilde{X_2}, and mu on tilde{X_1} intersect tilde{X_2} complement. We now introduce some useful notations. For nonempty subset G of the index set N_n let X_G be the collection of random variables X_i, i in G. And tilde{X}_G be the union of all tilde{X_i} i in G. Theorem 3.6 is a very important theorem. Let script B be the set of all tilde{X}_G such that G is a nonempty subset of the index set N. Then a signed measure mu on F_n is completely specified by mu(B), B in script B, which can be any set of real numbers. Now, this is the significance of theorem 3.6. We have previously seen that a signed measure mu on F_n is completely specified by a set of numbers mu(A) A in script A, the set of values of mu on the nonempty atoms. This theorem says that mu can instead be specified by mu(B), B in script B, the set of values of mu on the unions. In appendix 3A, we show that for any A in script A, the set of non-empty atoms of F_n, mu(A) can be expressed as a linear combination of the values of mu on the unions of tilde{X_i}. First it is easy to check that for a set-additive function mu, mu(A_1 union A_2) is equal to mu(A_1) plus mu(A_2) minus mu(A_1 intersect A_2). Now by rearranging the terms we obtain mu(A_1 intersect A_2) equals mu(A_1) plus mu(A_2) minus mu(A_1 union A_2). Notice that 2 can be obtained from 1 by exchanging the unions, and the intersections. Here we replace the union by an intersection, and here we replace the intersection by the union. We also notice that equation 1 is a special case where m is equal to 2 of the inclusion exclusion formula which says that mu of the union of A_k from k equals 1 to m is equal to summation mu(A_i) for all i, minus the summation of mu(A_i intersect A_j), for all distinct pairs i and j, plus, all the way to minus 1 to the power m plus 1, mu(A_1 intersect A_2 intersect ... intersect A_m) Theorem 3.19 in the appendix is a variation of the inclusion-exclusion formula. The proof of this theorem is elementary, for the details please see the textbook. To understand this variation of the inclusion-exclusion formula, we cover all the minus B in the formula and observe that this formula is just an inclusion-exclusion formula with all the unions replaced by intersections and all intersections replaced by unions. We are now ready to prove the claim in Appendix 3.A. That is, for any non-empty atom A of F_n, mu A can be expressed as a linear combination of the values of mu on the unions of tilde{X_i}'s. First, for a nonempty atom A in A, A is equal to the intersection of Y_i for i equals 1 up to n, where Y_i is either tilde{X_i} or tilde{X_i} complement, and there exists at least one i such that Y_i is equal to tilde{X_i}. Otherwise, A is equal to the empty atom. Then we can write A as the intersection of two intersections. The first intersection is the intersection of all tilde{X_u}'s where u is such that Y_u is equal to tilde{X_u}. Basically, what we have done is that we group all the X tildes and put them in the front and we group all the X tilde complements and put them at the back. For the second intersection, it is the intersection of all tilde{X_v}'s complement, for all v such that Y_v is equal to tilde{X_v} complement. By applying de Morgan's Law to the second intersection, we can write this as the union of tilde{X_v} for all v and then we take the complement. Equivalently, we can write this as minus the union of all tilde{X_v} for all v. This form is exactly the same as the one on the left-hand side of theorem 3.19, where A_k is equal to tilde{X_u} for some u and B is equal to the union of all B tilde{X_v}. By applying theorem 3.19, we can express mu(A) as a linear combination of terms of the forms on the right-hand side of the theorem. We now invoke an elementary identity in equation 2 that can easily be verified. It says that mu(A minus B) is equal to mu( A union B) minus mu(B). Then apply 2 to each term on the right hand side of equation 1. For example, mu(A_i union A_j minus B) is equal to mu(A_i union A_j union B) minus mu(B). Since B is a union of tilde{X_i}'s, the right hand side above is a linear combination of the values of mu on the unions of tilde{X_i}'s. Thus we have expressed mu A of a non empty atom A as a linear combination of the values of mu, on the unions of tilde{X_i}'s. We are now ready to prove theorem 3.6, which says that a sign measure mu, on F_n, is completely specified by mu(B), B in B, that is, we can specify a sign measure mu on F-n by specifying the values of mu on the unions. The proof goes as follows. First, recall that the number of nonempty atoms in F_n is equal to 2 to the power n minus 1. And a signed measure mu on F_n is completely specified by mu(A), A in A. That is we specify the value of mu on the non-empty atoms. And this set of numbers can be any set of real numbers. The set B is indexed by all non-empty subsets of the index set script N. So, the size of script B is equal to the size of script A, which is equal to 2 to the power n minus 1. And we call this quantity k. Now we let u, be a column k vector of mu(A), A in A. And we let n be a column k vector of mu(B), B in B. For each B in B, mu(B), which is the value of mu on a union, can obviously be expressed as a linear combination of mu(A) , A in A, by set additivity. In fact, this linear combination is a summation. Therefore, we can write, h is equal to C_n times u where C_n is a unique k by k matrix. On the other hand, we have shown in Appendix 3.A that for each A in A, mu(A) can be expressed as a linear combination of mu(B), B in B. Therefore, we can write, u is equal to D_n times h, where D_n is a k by k matrix. Combining equation one and equation two, we have u is equal to D_n times h, and h is equal to C_n times u. So we can write it as D_n times C_n times u, which is equal to D_n times C_n together, and then times u. Then obviously D_n times C_n is equal to I_n, the k by k identity matrix, showing that D_n is in fact equal to the inverse of C_n. Moreover since C_n is unique, D_n is also unique. Therefore mu(B), B in B, is related to mu(A), A in A, through an invertible linear transformation. Hence mu(B), B in B, completely specifies a signed measure mu on Fn.