Solution:
Let A,B,C,P be the centers of the circles with radii 1,2,3, and r, respectively. Then, ABC is a 3-4-5 right triangle. Using the law of cosines in △PAB yields
cos∠PAB=2⋅3⋅(1+r)32+(1+r)2−(2+r)2=3(1+r)3−r
Similarly,
cos∠PAC=2⋅4⋅(1+r)42+(1+r)2−(3+r)2=2(1+r)2−r
We can now use the equation (cos∠PAB)2+(cos∠PAC)2=1, which yields 0=23r2+132r−36=(23r−6)(r+6), or r=6/23.