**Step 1. (f is injective)
Claim 1.** For any a≥2, the set {fn(a)∣n∈Z≥0} is infinite.
Proof. First, we have ff(a)(a+1)=P(a,1)(a+1)f(1). Varying a, we see that f(Z≥0) is infinite. Next, we have fbf(a−1)(a)=P(a−1,b)af(b). So, varying b, fbf(a−1)(a) takes infinitely many values. □
Claim 2. For any a≥2 and n∈Z≥0, we have fn(a)=a.
Proof. Otherwise, we would get a contradiction with Claim 1. □
Assume f(b)=f(c) for some b<c. Then we have
(a+1)f(c)=P(a,c)fcf(a)(a+1)=f(c−b)f(a)(fbf(a)(a+1))=P(a,b)f(c−b)f(a)((a+1)f(b))=f(c−b)f(a)((a+1)f(c)),
which contradicts Claim 2. So, f is injective.
Step 2. (f(Z>0)=Z≥2)
Claim 3. 1 is not in the range of f.
Proof. If f(b)=1, then ff(a)(a+1)=a+1 by P(a,1), which contradicts Claim 2. □
We say that a is a descendant of b if fn(b)=a for some n∈Z>0.
Claim 4 For any a,b≥1, both of the following cannot happen at the same time:
- a is a descendant of b;
- b is a descendant of a.
Proof. If both of the above hold, then a=fm(b) and b=fn(a) for some m,n∈Z>0. Then a=fm+n(a), which contradicts Claim 2. □
Claim 5 For any a,b≥2, exactly one of the following holds:
- a is a descendant of b;
- b is a descendant of a;
- a=b.
Proof. For any c≥2, taking m=fcf(a−1)−1(a) and n=fcf(b−1)−1(b), we have
f(m)=fcf(a−1)−1(a)=P(a−1,c)af(c)andf(n)=fcf(b−1)−1(b)=P(b−1,c)bf(c).
Hence
fnf(a−1)(a)=P(a−1,n)af(n)=abf(c)=bf(m)=P(a−1,m)fmf(b−1)(b).
The assertion then follows from the injectivity of f and Claim 2. □
Now, we show that any a≥2 is in the range of f. Let b=f(1). If a=b, then a is in the range of f. If a=b, either a is a descendant of b, or b is a descendant of a by Claim 5. If b is a descendant of a, then b=fn(a) for some n∈Z>0, so 1=fn−1(a). Then, by Claim 3, we have n=1, so 1=a, which is absurd. So, a is a descendant of b. In particular, a is in the range of f. Thus, f(Z>0)=Z≥2.
Step 3. (f(1)=2)
Claim 6 Let a,n≥2, then na is a descendant of a.
Proof. We write n=f(m) by Step 2. We have na=f(m)a=P(a−1,m)fmf(a−1)(a), which shows na is a descendant of a. □
By Claim 6, all even integers ≥4 are descendants of 2. Hence 2=f(2k+1) for some k≥0. Next, we show f(2k+1)≥f(1), which implies f(1)=2. It trivially holds if k=0. If k≥1, let n be the integer such that fn(2)=2k+2. For any b>n/f(1), we have
fbf(1)−n(2k+2)=fbf(1)(2)=P(1,b)2f(b)andfbf(2k+1)(2k+2)=P(2k+1,b)(2k+2)f(b).
By Claim 6, (2k+2)f(b) is a descendant of 2f(b). By Claim 2, we have b(2k+1)>bf(1)−n. By taking b large enough, we conclude f(2k+1)≥f(1).
Step 4. (f(2)=3 and f(3)=4)
From f(1)=2 and P(1,b), we have f2b(2)=2f(b). So taking b=1, we obtain f2(2)=2f(1)=4; and taking b=f(2), we have f2f(2)(2)=2f2(2)=8. Hence, f2f(2)−2(4)=f2f(2)(2)=8 and f3(4)=P(3,1)8 give f(3)=2f(2)−2.
Claim 7 For any m,n∈Z>0, if f(m) divides f(n), then m≤n.
Proof. If f(m)=f(n), the assertion follows from the injectivity of f. If f(m)<f(n), by P(a,m), P(a,n) and Claim 6, we have that fnf(a)(a+1) is a descendant of fmf(a)(a+1) for any a∈Z>0. So mf(a)<nf(a), and m<n. □
By Claim 7, every possible divisor of f(2) is in {1,f(1)=2,f(2)}. Thus f(2) is an odd prime or f(2)=4. Since f2(2)=4, we have f(2)=4, and hence f(2) is an odd prime. We set p=f(2).
Now, f(3)=2f(2)−2=2(p−1). Since p−1 divides f(3), we have p−1∈{1,f(1)=2,f(2)=p} by Claim 7, so p−1=2. Thus, f(2)=p=3 and f(3)=2(p−1)=4.
Step 5. (f(n)=n+1)
Claim 8 For any b≥1, f(2f(b)−1)=2b+2.
Proof. Since f2(2)=4, we have f2b−2(4)=f2b(2)=2f(b), so
ff(2f(b)−1)+2b−2(4)=ff(2f(b)−1)(2f(b))=P(2f(b)−1,1)4f(b)=P(3,b)f4b(4),
which gives us f(2f(b)−1)=2b+2. □
Finally, we prove f(n)=n+1 by induction on n. Suppose f(n)=n+1 for all 1≤n≤2b+1. Replace b by b+1 in f(2f(b)−1)=2b+2 to get
f(2b+3)=f(2f(b+1)−1)=2(b+1)+2=2b+4.
By induction hypothesis, we have fb(b+2)=2b+2. Hence
f(f(2b+2))=fb+2(b+2)=ff(b+1)(b+2)=P(b+1,1)2(b+2)=f(2b+3).
By injectivity, f(2b+2)=2b+3. Then f(n)=n+1 for all n∈Z≥0, which is indeed a solution.