If we choose x=0, then
f(yf(0))=f(f(0)).
If f(0)=0, then for y=f(0)1, we get
f(t)=f(f(0))=c,
i.e. f is a constant function. But then, substituting in (1), the equation gets the form
c=c+cx,x∈R.
As x is arbitrary, we get that c=0. Due to the resulting contradiction, f(0)=0.
II. Suppose that f(x)=0 for some x=0. Then, substituting in the initial equation
xf(y)=0,
for each y∈R. Therefore, f(y)=0 for each y∈R. Due to the resulting contradiction, f(x)=0, x∈R.
It is not hard to check that f(x)=0, x∈R is a solution of the initial equation.
III. If y=0, then f(x)=f(f(x)). Now, substituting in the first equation,
f(x+yf(x))=f(x)+xf(y).
IV. Let f(1)=a=0. Then
x=1,y=1x=1,y=−1x=1+a,y=−1⇒f(1+a)=2a⇒f(1−a)=a+f(−1)⇒f(1−a)=f(1+a−2a)=2a+(1+a)f(−1)
Now from (1) and (2) we get
a+f(−1)−af(−1)f(1−a)=2a+(1+a)f(−1)=af(−1)=−1=a−1
On the other hand
xxx=1−a,y=1=1=−10=f(1−a+a−1)=a−1+(1−a)a=−(a−1)2=a−1=f(1+y)=1+f(y)=f(−1−y)=−1−f(y)
Now from (3) and (4) −f(y)=1+f(−1−y)=f(−y). Therefore, for y=−1 we have
f(x−f(x))=f(x)−x.
Finally,
f(x−f(x))f(x−f(x))x−f(x)f(x)=f(f(x−f(x)))=f(f(x)−x)=−f(x−f(x))=0=0=x
This function satisfies the equation, so the required solutions are: f(x)=0 and f(x)=x.