Bayesian Balls

Suppose there are three balls, each either black or white with
presumed equal probability, in a covered container.  Based on this
premise, the probability of all three balls being white is 1/8.  Now
suppose we blindly reach in and pull out a ball - it is white.  We 
return that ball to the container and make another blind selection - 
it again is white.  We return the ball and make another pick - white
again.  So, we have made three random selections and come out with 
three white balls.  What, now, is the probability that all three balls
are white?

Bayes Theorem is the easiest way of computing the answer.  For any 
two events X and Y, the probability of both X and Y being true is 
clearly equal to the probability of {X is true} times the probability 
of {Y is true given that X is true}.  This can be written as   
Pr{X and Y} = Pr{X} Pr{Y|X},  where the symbol "|" is read as "given".
From this it follows that

                               Pr{X and Y}
                    Pr{Y|X} = -------------                 (1)
                                  Pr{X}
 
So if we define Y as the event {All three balls are white} and X as 
the event {All three of our selections are white}, then obviously the 
probability that all 3 balls are white AND all 3 selections are white 
is just equal to the probability that all 3 balls are white, which is 
1/8.  So we only need to determine Pr{X}, the probability that all 
3 selections are white.  

As already noted, the unconditional probability that all 3 balls are 
white is 1/8, and we are certain to have all 3 selections white in 
that case.  The unconditional probability that exactly 2 balls are 
white is 3/8, in which case we have (2/3)^3 probability of all 3
selections being white.  The unconditional probability of exactly
1 white ball is 3/8, in which case we have (1/3)^3 probability of 
all 3 selections being white.  Therefore, the overall probability 
of all 3 selections being white is

         (1/8)*(1) + (3/8)*(2/3)^3 + (3/8)*(1/3)^3  =  1/4

So, we've determined that, of all possible outcomes, 1/4 result in 
all 3 selections being white, and 1/8 have all 3 balls white AND 
all 3 selections white.  It follows that of the cases when all 3 
selections are white, exactly half are cases with all 3 balls being 
white.  Hence the conditional probability of all three balls being
white, given that all three of our selections was white, is 1/2.

Now, sometimes students think that the probability of all the balls 
being white should be 2/9, based on the notion that it equals the
probability of selecting a different ball on each of our three trials,
which equals (1)(2/3)(1/3)= 2/9.  It's true that if we select a 
different ball each time, and if all 3 selections are white, then 
all three balls are white, but the converse is not true.  If we do 
NOT select a different ball each time, it's still possible that all
3 balls are white.  So, to work the problem this way, we need to 
consider all the possibilities:  We have 2/9 chance of selecting 3 
different balls, and the probability of all 3 selections being white 
is 1/8.  We have 6/9 chance of selecting one ball twice and another 
ball once, and in this case our chances of all 3 selections being 
white are 1/4.  We have 1/9 chance of selecting the same ball all 
three times, and in this case our chance is 1/2 of all three 
selections being white.  Combining these, the total probability of 
all 3 selections being white is 

            (2/9)(1/8) + (6/9)(1/4) + (1/9)(1/2)  =  1/4

so again the probability of all three balls being white, given that 
all 3 selections are white, is (1/8)/(1/4) = 1/2.

For a related problem, suppose a box contains six white balls and an 
unknown number of blue balls, but not more than six.  Three balls are 
removed successively without replacement, and all are blue. If a 4th 
ball is removed, what is the probability that it is white?  This 
problem, again, calls for some Bayesian analysis, to infer the 
probability of a prior condition from some subsequent results.  In 
this case we are trying to figure out the probabilities that the 
original number of blue balls was 0,1,2,..,or 6, given the fact that 
we drew 3 blue balls in a row.  Again we use a form of Bayes theorem

                                    Pr{A}
                  Pr{A|B} = Pr{B|A} -----
                                    Pr{B}

which is really just a consequence of the intuitive fact that 
Pr{A and B} = Pr{A}Pr{B|A}, and by symmetry we also have 
Pr{A and B} = Pr{B}Pr{A|B}, so we can equate these two to give the
above version of Bayes' theorem.

The probability that there were originally 5 blue balls, given that 
we drew 3 blue balls in a row, is equal to [the probability that we 
would draw 3 in a row GIVEN that there were originally 5 blue balls] 
times [the probability that there were originally 5 blue balls] 
divided by [the probability that we would draw 3 in a row].

Notice that if we assume the "a priori" probability of 5 blue balls 
originally is the same as 4 or 3 or 2 or 1 or 6, then that factor is 
the same for all the possibilities.  Also, the overall probability
of drawing 3 blue balls (unconditionally) is the same for all cases.
Thus, the probabilities are PROPORTIONAL to the respective values of
Pr{B|A}, i.e., the probability that we would draw 3 blue in a row 
GIVEN that there were originally k blue balls (k=3,4,5,6).  The 
results are

         3 2 1           4 3 2          5  4 3           6  5 4
   p3  = - - -     p4 = -- - -    p5 = -- -- -     p6 = -- -- --
         9 8 7          10 9 8         11 10 9          12 11 10

Letting Pt = p3 + p4 + p5 + p6, the overall result is found by 
multiplying the probability of drawing a white on the next ball given 
each of these four possible priors, so we have

     p3         p4           p5           p6           677
     -- (1)  +  -- (6/7)  +  -- (6/8)  +  -- (6/9)  =  ---
     pt         pt           pt           pt           909

which is about 74.5%.  

Incidentally, suppose we re-worded the problem like this:

   A box contains from 9 to 12 balls, 6 of which are white, 
   and the remainder red.  If someone randomly draws four
   balls (without replacement) and the first three of those
   four are red, what is the probability that the fourth 
   one is white?

The answer is the same as with the original wording.  We can make 
the original problem explicitly symmetrical over all seven of the 
equally likely prior conditions (which consist of there being 0, 
1, 2,..., or 6 blue balls at the start) by letting pk denote the 
probability that we would draw 3 blue balls in a row GIVEN that 
there were originally k blue balls.  Then define 

                 pt = p0+p1+p2+p3+p4+p5+p6

to give the "weights" (pk/pt) of the seven possible conditions.  Of 
course, it turns out that p0 = p1 = p2 = 0, so the solution reduces 
to the same form as given originally.  In other words, the problem 
has the same answer with the original wording as with the alternate 
wording.  The only assumption that's needed (in either case) is that 
the prior probabilities of having 3, 4, 5, or 6 blue balls are all 
equal.  It doesn't matter if these priors are each 1/4 (meaning they 
are are only four possible priors) or each 1/7 (meaning there could 
have been 0, 1, or 2 blue balls also).  

The reason is that the subsequent condition of drawing 3 blue balls 
in a row automatically sets the "post" probabilities of the 0,1,2 cases 
to zero.  The priors of 1/7 or 1/4 are just used to determine the 
PROPORTIONALITY between the resulting non-zero weights.  As long as 
their priors are all the same (all 1/4 or all 1/7) the final answer 
will be the same.