Solution:
Adding 1 to both sides, the equation reduces to
[(x+1)2]=([x+1])2
We have used [x]+m=[x+m] for every integer m.
Suppose x+1≤0. Then [x+1]≤x+1≤0. Thus
([x+1])2≥(x+1)2≥[(x+1)2]=([x+1])2
Thus equality holds everywhere. This gives [x+1]=x+1 and thus x+1 is an integer. Using x+1≤0, we conclude that
x∈{−1,−2,−3,…}
Suppose x+1>0. We have
(x+1)2≥[(x+1)2]=([x+1])2
Moreover, we also have
(x+1)2≤1+[(x+1)2]=1+([x+1])2
Thus we obtain
[x]+1=[x+1]≤(x+1)<1+([x+1])2=1+([x]+1)2
This shows that
x∈[n,1+(n+1)2−1)
where n≥−1 is an integer. Thus the solution set is
{−1,−2,−3,…}∪{n=−1⋃∞[n,1+(n+1)2−1)}
It is easy to verify that all the real numbers in this set indeed satisfy the given equation.