Substituting x=y=0 in the functional equation, we get f2(0)=1 or f(0)=±1.
If f(0)=1 then substituting x=x,y=0, we get f(0)=f(x)⋅f(0)+f(x)−1 or f(x)=1. So in this case f(x)=1 is a constant function.
If f(0)=−1, then substituting x=y=2, we get f(4)=f2(2)+f(4)−1 or f(2)=±1.
Now suppose that f(2)=−1. Then, substituting x=y=1, we get f(1)=f2(1)+f(2)−1⇔f2(1)−f(1)−2=0. From this quadratic equation we find that f(1)=−1 or f(1)=2.
If f(1)=2 then substituting x=x, y=1 we get f(x)=f(x)f(1)+f(x+1)−1⇔f(x+1)=1−f(x). From here we get f(x+2)=1−f(x+1)=f(x). Also, another way, substituting x=x, y=2 then we get f(2x)=f(x)f(2)+f(x+2)−1=−f(x)+2f(x+1)+1−1=−f(x)+2f(x+1). But from above, f(x+1)=1−f(x), so f(2x)=−f(x)+2(1−f(x))=−f(x)+2−2f(x)=2−3f(x). But for every x∈Q we get that f(x)=−1, but we have x=1 for f(1)=2. Which is a contradiction.
Now if f(1)=−1 then substituting x=x, y=1 we get
f(x)=f(x)⋅f(1)+f(x+1)−1⇔f(x+1)=2f(x)+1.(1)
On the other hand, we get
f(2x)=f(x)⋅f(2)+f(x+2)−1=−f(x)+2f(x+1)+1−1==−f(x)+2(2f(x)+1)=−f(x)+4f(x)+2=3f(x)+2.(2)
By using (1) we get that:
f(m)=f(m−1)+1=f(m−2)+2=⋯=f(2)+m−2=m−1.
for every m∈N, since f(2)=1 (see below).
Now, let's check f(2). If f(2)=−1, as above, we get a contradiction. So f(2)=1.
Now substituting x=y=1, we get f(1)=f2(1)+f(2)−1. So f(1)=0 or f(1)=1. If f(1)=1 then substituting x=x, y=1 we get f(x)=f(x)⋅f(1)+f(x+1)−1 from this f(x+1)=1. Substituting x=−1 then f(0)=1 which is a contradiction with f(0)=−1.
Now we have f(0)=−1, f(1)=0, f(2)=1. For every x∈Q:
f(x)=f(x)⋅f(1)+f(x+1)−1⇔
⇔f(x+1)=f(x)+1.(3)
Then for every m∈N
f(m)=f(m−1)+1=f(m−2)+2=⋯=f(2)+m−2=m−1.
Also,
0=f(1)=f(m⋅m1)=f(m)⋅f(m1)+f(m+m1)−1
By using (3)
f(m+m1)⇔0=f(m−1+m1)+1=⋯=f(m1)+m⇔=(m−1)f(m1)+f(m1)+m−1
So we get f(m1)=m1−1.
Now for every m,n∈N,
f(mn)=f(n)⋅f(m1)+f(n+m1)−1==(n−1)⋅(m1−1)+n+m1−1−1=mn−1.
Therefore, the solutions are: for every x∈Q, f(x)=−1 or f(x)=x−1.