Problem 564

When $(3+2x+x^2)(1+mx+m^2x^2)$ is expanded, the terms that include an $x^2$ will come from multiplying a constant with a term that includes $x^2$ or multiplying two terms that includes $x$.
In other words, the term that includes $x^2$ will be $$3 \cdot m^2x^2 + 2x \cdot mx + x^2 \cdot 1 = 3m^2x^2 + 2mx^2 + x^2 = (3m^2+2m+1)x^2$$ From the condition that the coefficient of this term equals 1, we see that $3m^2+2m+1=1$ which gives $3m^2+2m=0$ or $m(3m+2) = 0$, which means that $m=0$ or $m = -\frac{2}{3}$.
The sum of these possible values of $m$ is $-\frac{2}{3}$.