HMMT February

Consider a right triangle $A O C$ with right angle at $O, A O=16$ and $C O=x$. Moreover, let $B$ be on $A O$ such that $B O=9$. Then $\tan ^{-1} \frac{x}{9}=\angle C B O$ and $\tan ^{-1} \frac{x}{16}=\angle C A O$, so their difference is equal to $\angle A C B$. Note that the locus of all possible points $C$ given the value of $\angle A C B$ is part of a circle that passes through $A$ and $B$, and if we want to maximize this angle then we need to make this circle as small as possible. This happens when $O C$ is tangent to the circumcircle of $A B C$, so $O C^{2}=O A \cdot O B=144=12^{2}$, thus $x=12$, and it suffices to compute $\sin (\alpha-\beta)$ where $\sin \alpha=\cos \beta=\frac{4}{5}$ and $\cos \alpha=\sin \beta=\frac{3}{5}$. By angle subtraction formula we get $\sin (\alpha-\beta)=\left(\frac{4}{5}\right)^{2}-\left(\frac{3}{5}\right)^{2}=\frac{7}{25}$.