Substitute y=1 and y=−1:
f(f(x)−1)=f(x2)+f(1)−2f(x) and f(f(x)−1)=f(x2)+f(−1)−2f(−x). (∗)
Combining both equalities, get f(1)−2f(x)=f(−1)−2f(−x). Hence, if x=1: f(1)=f(−1), so f(x)=f(−x), the function f is even.
If x=y=1, f(f(1)−1)=0, in other words there exists such number b that f(b)=0. If x=b in (∗) then
f(f(b)−1)=f(b2)+f(1),f(−1)=f(b2)+f(1),so f(b2)=0.
Let substitute x=b and y=0 in the initial equality: f(f(b))=f(b2)−2f(0), so 3f(0)=f(b2)=0. If x=0 then
f(y2)=y2f(y).(∗∗)
There are two cases.
Case 1: there exists b∈R, b=0 such that f(b)=0. As is shown above, f(b2)=0. Let substitute x=b in the initial equality: