Harvard-MIT Math Tournament

Solution:

Using a little trig, we have $BC = 2 \sin 18$, $AC = 2 \sin 36$, and $AB = 2 \sin 30$ (see left diagram). Call these $a, b$, and $c$, respectively. By the law of cosines, $b^{2} = a^{2} + c^{2} - 2 a c \cos ABC$, therefore
$$\cos ABC = \frac{\sin^{2} 18 + \sin^{2} 30 - \sin^{2} 36}{2 \sin 18 \sin 30}.$$
In the right diagram below we let $x = 2 \sin 18$ and see that $x + x^{2} = 1$, hence $\sin 18 = \frac{-1 + \sqrt{5}}{4}$. Using whatever trig identities you prefer you can find that $\sin^{2} 36 = \frac{5 - \sqrt{5}}{4}$, and of course $\sin 30 = \frac{1}{2}$. Now simplification yields $\sin^{2} 18 + \sin^{2} 30 - \sin^{2} 36 = 0$, so $\angle ABC = \mathbf{90}^{\circ}$.

Note that this means that if a regular pentagon, hexagon, and decagon are inscribed in a circle, then we can take one side from each and form a right triangle.