Bulgarian Mathematical Competitions

Solution:
It is easy to see that if a triangle contains another triangle, then its inradius is greater than the inradius of the second one. So we may consider triangles with vertices on the boundary of the square. Moreover, we may assume that at least one vertex of the triangle is a vertex of the square and the other two vertices belong to the sides of the square containing not the first vertex. So we shall consider $\triangle OAB$ such that $O=(0,0)$, $A=(a,1)$ and $B=(1,b)$, $0 \leq a, b \leq 1$.

Consider also $\triangle OCD$, where $C=(a+b, 1)$ and $D=(1,0)$. Denote by $S$ and $P$ the area and perimeter of $\triangle OAB$, respectively. Set $x=OA=\sqrt{1+a^{2}}$, $y=AB=\sqrt{(1-a)^{2}+(1-b)^{2}}$, $z=OB=\sqrt{1+b^{2}}$, $u=OC=\sqrt{1+(a+b)^{2}}$ and $v=CD=\sqrt{1+(1-a-b)^{2}}$. Note that $OD=1$, $u \geq z \geq 1$, $x \geq 1$ and $v \geq 1$.

Comparing the perimeters of $\triangle OAB$ and $\triangle OCD$ gives
$$\begin{aligned} (u+v+1)-(x+y+z) & =\frac{u^{2}-x^{2}}{u+x}+\frac{v^{2}-y^{2}}{v+y}+\frac{1-z^{2}}{1+z} \\ & =\frac{2ab+b^{2}}{u+x}+\frac{2ab}{v+y}-\frac{b^{2}}{1+z} \\ & \leq \frac{2ab+b^{2}}{1+z}+\frac{2ab}{v+y}-\frac{b^{2}}{1+z} \\ & =2ab\left(\frac{1}{v+y}+\frac{1}{1+z}\right) \leq 3ab \leq (u+v+1)ab \end{aligned}$$
Hence
$$(u+v+1)(1-ab) \leq x+y+z \Longleftrightarrow \frac{1}{u+v+1} \geq \frac{1-ab}{x+y+z}=\frac{2S}{P}=r$$
On the other hand,
$$u+v+1=\sqrt{1+(a+b)^{2}}+\sqrt{1+(1-a-b)^{2}}+1 \geq \min_{x \geq 1} F(x)$$
where $F(x)=\sqrt{1+x^{2}}+\sqrt{1+(1-x)^{2}}+1$. Since $\min_{x \geq 1} F(x)=\sqrt{5}+1$, we get that $r \leq \frac{1}{\sqrt{5}+1}=\frac{\sqrt{5}-1}{4}$.