The zero polynomial is obviously a valid solution. Let p be a non-zero polynomial and write p(x)=anxn+an−1xn−1+⋯+a0, where an=0. The leading term on the left-hand side of the equality is equal to an(anxn)n=ann+1xn2, and the leading term on the right-hand side is x2⋅anxn=anxn+2. The degrees of the two terms must be equal, so n2=n+2, or (n−2)(n+1)=0. Since n is a non-negative integer, we conclude that n=2. Hence, p(x)=ax2+bx+c, where a=0. We have:
p(p(x))=a3x4+2a2bx3+(ab2+2a2c+ab)x2+(2abc+b2)x+(ac2+bc+c)(1)
(x2+x+1)p(x)=ax4+(b+a)x3+(c+b+a)x2+(c+b)x+c.(2)
From the given equality we obtain the system of equations
a32a2bab2+2a2c+ab2abc+b2ac2+bc+c=a,=b+a,=c+b+a,=c+b,=c.
Since a=0, the first equation implies a=1 or a=−1. If a=1 the second equation becomes 2b=b+1, so b=1. Inserting both into the third equation we find that 2c+2=c+2, so c=0. It is easy to check that these three numbers also satisfy the final two equations. If a=−1, then the system has no solutions. There are two solutions to the problem, namely p(x)=0 and p(x)=x2+x.