The given relation implies
f(fg(n)(n))<f(n+1) for all n,(1)
which will turn out to be sufficient to determine f.
Let y1<y2<… be all the values attained by f (this sequence might be either finite or infinite). We will prove that for every positive n the function f attains at least n values, and we have (i)n: f(x)=yn if and only if x=n, and (ii)n: yn=n. The proof will follow the scheme
(i)1,(ii)1,(i)2,(ii)2,…,(i)n,(ii)n,…(2)
To start, consider any x such that f(x)=y1. If x>1, then (1) reads f(fg(x−1)(x−1))<y1, contradicting the minimality of y1. So we have that f(x)=y1 is equivalent to x=1, establishing (i)1.
Next, assume that for some n statement (i)n is established, as well as all the previous statements in (2). Note that these statements imply that for all k≥1 and a<n we have fk(x)=a if and only if x=a.
Now, each value yi with 1≤i≤n is attained at the unique integer i, so yn+1 exists. Choose an arbitrary x such that f(x)=yn+1; we necessarily have x>n. Substituting x−1 into (1) we have f(fg(x−1)(x−1))<yn+1, which implies
fg(x−1)(x−1)∈{1,…,n}(3)
Set b=fg(x−1)(x−1). If b<n then we would have x−1=b which contradicts x>n. So b=n, and hence yn=n, which proves (ii)n. Next, from (i)n we now get f(k)=n⟺k=n, so removing all the iterations of f in (3) we obtain x−1=b=n, which proves (i)n+1.
So, all the statements in (2) are valid and hence f(n)=n for all n. The given relation between f and g now reads n+gn(n)=n+1−g(n+1)+1 or gn(n)+g(n+1)=2, from which it
immediately follows that we have g(n)=1 for all n.