Denote $u = \sqrt[6]{xyz}$. Then by the given condition and mean inequality,
$$\begin{aligned} 2u^3 &= 2\sqrt{xyz} = \frac{1}{\sqrt{3}}\sum \sqrt{x(3-3x)} \\ &\le \frac{1}{\sqrt{3}}\sum \frac{x+(3-3x)}{2} = \frac{3\sqrt{3}}{2} - \frac{1}{\sqrt{3}}(x+y+z) \\ &\le \frac{3\sqrt{3}}{2} - \sqrt{3} \times \sqrt[3]{xyz} = \frac{3\sqrt{3}}{2} - \sqrt{3}u^2. \end{aligned}$$
Therefore, $4u^3 + 2\sqrt{3}u^2 - 3\sqrt{3} \le 0$, i.e.
$$(2u - \sqrt{3})(2u^2 + 2\sqrt{3}u + 3) \le 0,$$
and thus $u \le \frac{\sqrt{3}}{2}$. Following this, we have $xyz \le \frac{27}{64}$, and equality holds when $x = y = z = \frac{3}{4}$. Hence, the maximum is $\frac{27}{64}$.