4. $-\frac{3}{2}$.

As shown in Figure 3, the length of $AB$ is determined. To minimize the perimeter of quadrilateral $ABCD$, we need to minimize $AD + DC + CB$. Construct the point $A'$, the reflection of point $A$ over the $y$-axis, and the point $B'$, the reflection of point $B$ over the $x$-axis. Connect $A'B'$, which intersects the $x$-axis and $y$-axis at points $C$ and $D$, respectively. Then the perimeter of quadrilateral $ABCD$ is minimized.
It is easy to see that $A'(-4,-5)$ and $B'(8,3)$.
Let the equation of line $A'B'$ be $y = kx + b$.
From $\left\{\begin{array}{l}-4k + b = -5, \\ 8k + b = 3,\end{array}\right.$ we solve to get
$$k = \frac{2}{3}, b = -\frac{7}{3}.$$

Therefore, the equation of line $A'B'$ is $y = \frac{2}{3}x - \frac{7}{3}$.
When $x = 0$, $y = -\frac{7}{3}$; when $y = 0$, $x = \frac{7}{2}$.
Thus, $m = \frac{7}{2}$ and $n = -\frac{7}{3}$.
Hence, $\frac{m}{n} = -\frac{3}{2}$.