Solution:

Clearly one vertex of the rectangle will be at the right angle. Position the triangle with the leg of length $4$ along the $x$-axis and the leg of length $3$ along the $y$-axis. Then the hypotenuse is along the line $y = 3 - (3/4)x$.

Suppose the rectangle has a side of length $y$ along the leg of length $3$. Then the area is $y \left(\frac{4}{3}\right)(3-y) = 4y - \frac{4}{3}y^{2}$. The derivative of this is $0$ when $4 - \frac{8}{3}y = 0$, or $y = \frac{3}{2}$, giving an area of $3$.

Or, if you prefer, suppose the rectangle has a side of length $x$ along the leg of length $4$. Then the area is $x(3 - (3/4)x) = 3x - \frac{3}{4}x^{2}$. The derivative of this is $0$ when $3 - \frac{3}{2}x = 0$, or $x = 2$, again giving an area of $3$.