Substitute x=y=0: this gives
f(0)=f(0)−0+f(f(0))
so f(f(0))=0. Substitute x=y=1: this gives
f(1+f(1))=f(f(1))−1+f(1+f(1))
so f(f(1))=1. Now substitute x=1,y=0 :
f(1)=f(f(0))−1+f(f(1))
We know that f(f(0))=0 and f(f(1))=1, so we find f(1)=0. Since f(f(1))=1, this also gives f(0)=1. Now substitute y=0 :
f(x)=f(x)−x+f(f(x)) for all x∈R
so f(f(x))=x for all x∈R. Substitute x=1 : this gives
f(1)=f(f(y))−1+f(y) for all x∈R
Together with f(1)=0 and f(f(y))=y, we now get 0=y−1+f(y), so f(y)=1−y for all y∈R. We have now found that the only possible solution can be: f(x)=1−x for all x∈R. If we substitute this into the original functional equation, we get on the left
1−(x+y(1−x))=1−x−y+xy
and on the right
1−x(1−y)−x+1−(y+1−x)=1−x+xy−x+1−y−1+x=1−x−y+xy
So this function satisfies and is therefore the only solution.