$$x^{2} + x y + \frac{y^{2}}{2} + \frac{2^{6}}{x + y} + \frac{3^{4}}{x^{3}} = \frac{x^{2}}{2} + \frac{(x + y)^{2}}{2} + \frac{2^{6}}{x + y} + \frac{3^{4}}{x^{3}}.$$
By applying AM-GM inequality we have
$$\frac{x^{2}}{2} + \frac{3^{4}}{x^{3}} = \frac{x^{2}}{6} + \frac{x^{2}}{6} + \frac{x^{2}}{6} + \frac{3^{4}}{2 x^{3}} + \frac{3^{4}}{2 x^{3}} \geq 5 \sqrt[5]{\left(\frac{x^{2}}{6}\right)^{3} \left(\frac{3^{4}}{2 x^{3}}\right)^{2}} = \frac{15}{2}$$
and the equality holds when $\frac{x^{2}}{6} = \frac{3^{4}}{2 x^{3}}$. This is when $x = 3$.
By applying AM-GM inequality we have
$$\frac{(x + y)^{2}}{2} + \frac{2^{6}}{x + y} = \frac{(x + y)^{2}}{2} + \frac{2^{5}}{x + y} + \frac{2^{5}}{x + y} \geq 3 \sqrt[3]{\frac{(x + y)^{2}}{2} \left(\frac{2^{5}}{x + y}\right)^{2}} = 24$$
and the equality holds when $\frac{(x + y)^{2}}{2} = \frac{2^{5}}{x + y}$. This is when $x + y = 4$.
Therefore, the minimum of
$$x^{2} + x y + \frac{y^{2}}{2} + \frac{2^{6}}{x + y} + \frac{3^{4}}{x^{3}}$$
is
$$\frac{15}{2} + 24 = \frac{63}{2}$$
and it is reached when $x = 3$ and $y = 1$.