By the Cauchy inequality,
(x+1)(y+1)≤4((x+1)+(y+1))2=4(x+y+2)2.
So the required inequality
x+y+11−(x+1)(y+1)1<111,if x>0 and y>0,(1)
is a consequence of the inequality
x+y+11−(x+y+2)24<111,if x>0 and y>0.(2)
Putting x+y+1=t (t>1, since x and y are positive) into (2), we obtain the inequality
t1−(t+1)24<111,if t>1(3)
which is equivalent to (2). This inequality can be represented as
t3−9t2+23t−11>0,if t>1.(4)
Inequality (1) will be proved once we prove (4). We rearrange (4) as
t3−9t2+23t−11=(t−3)3−4t+16=(t−3)3−4(t−3)+4=(t−1)(t−3)(t−5)+4.
Therefore, (4) is equivalent to the inequality
(t−1)(t−3)(t−5)+4>0,if t>1.(5)
We see that for t>1 the polynomial (t−1)(t−3)(t−5)≤0 only if t∈[3;5]. But in this case 0<t−1≤4 and (t−3)(t−5)≥−1, therefore, (t−1)(t−3)(t−5)≥−4 for t∈[3;5]. Thus (5) is proved, which implies that (1) holds.