First, setting y=0 to give f(f(x)2)=xf(x).
Then setting x=0 to get f(f(y)f(−y)+y2)=24yf(0). Now change y by −y then the LHS does not change, so we will get
24yf(0)=−24yf(0),∀y⟹f(0)=0.
From f(f(y)f(−y)+y2)=0, one can get f(y)f(−y)=−y2.
Setting y=−x to get f(−24x2)=−24xf(x), change x to −x will implies that f(x) is odd. Thus f(y)2=y2. We have two solutions: f(x)=x for all x and f(x)=−x for all x.
Now we need to handle the point-wise trap. Let A={a∈R∖{0}∣f(a)=a} and B={a∈R∖{0}∣f(b)=−b}, suppose that these sets are non-empty. Take a∈A and b∈B.
Let x=a,y=b−a to give f(x+y)=f(b)=−b and f(x−y)=f(2a−b). Thus
LHS=f(−bf(2a−b)+2a(b−a)+(b−a)2)=±(−b±(2a−b)+b2−a2).
RHS=(b−a)a+a(−b)=−a2.
Thus ±(±(2a−b)(−b)+b2−a2)=−a2. For each pair of a,b, there are 4 ways to choose the sign so for some fixed value a, there are at most 8 ways to choose b which implies that ∣B∣≤8. Similarly, ∣A∣≤8, this is contradiction since A∪B=R∖{0}.
Hence, f(x)=x for all x and f(x)=−x for all x. □