We know $r+\frac1r=\sqrt3$. Cubing this gives $r^3+3r+\frac3r+\frac1{r^3}=3\sqrt3$. But $3r+\frac3r=3\left(r+\frac1r\right)=3\sqrt3$, so subtracting this from the first equation gives
$r^3+\frac1{r^3}=\boxed{0\textbf{ (C)}}$.
(Actually, $r+\frac1r$ could have been equal to $-\sqrt3$ instead of $\sqrt3$, but this would have led to the same answer. Also, this answer implies that $r^6=-1$, which means that $r$ is a complex number.)