The square's corners must be at $(x, y),(x+1, y),(x+1, y+1)$, and $(x, y+1)$ for some $x$ and $y$. So, $$\begin{aligned} f\left(x_{1}, y_{1}\right) & +f\left(x_{2}, y_{2}\right)+f\left(x_{3}, y_{3}\right)+f\left(x_{4}, y_{4}\right) \\ & =2\left(x^{2}+2 x\right)+2\left((x+1)^{2}+2(x+1)\right)+2\left(y^{2}+4 y\right)+2\left((y+1)^{2}+4(y+1)\right) \\ & =4 x^{2}+12 x+6+4 y^{2}+20 y+10 \\ & =(2 x+3)^{2}-3+(2 y+5)^{2}-15 \\ & \geq-18 \end{aligned}$$ This attains its minimum value of -18 when $x=-\frac{3}{2}$ and $y=-\frac{5}{2}$.