Solution:
Let $d$ be the degree of $p$ and let $k$ be any non-negative integer. We will choose
$$q(x) = \frac{x^{d + k + \ell}p\left(\frac{1}{x}\right) - p(x)}{x - 1},$$
$$r(x) = \frac{p(x) - x^{d + k}p\left(\frac{1}{x}\right)}{x - 1}.$$
First, we must show that both $q$ and $r$ are integer polynomials. Consider the numerator in $q$'s definition, $x^{d + k + \ell}p\left(\frac{1}{x}\right) - p(x)$. This is clearly an integer polynomial. As it is equal to $0$ when evaluated at $x = 1$, $x - 1$ divides it. Furthermore, as $x - 1$ is monic, the quotient has integer coefficients. The argument for $r$ is similar.
Next, we will show that this choice of $q$ and $r$ satisfies the desired equation. Plugging them into the RHS of the equation gives
$$q(x) + x^{\ell}r(x) = \frac{x^{d + k + \ell}p\left(\frac{1}{x}\right) - p(x)}{x - 1} + x^{\ell}\left(\frac{p(x) - x^{d + k}p\left(\frac{1}{x}\right)}{x - 1}\right)$$
$$\qquad = \frac{x^{d + k + \ell}p\left(\frac{1}{x}\right) - p(x) + x^{\ell}p(x) - x^{d + k + \ell}p\left(\frac{1}{x}\right)}{x - 1}$$
$$\qquad = \left(\frac{x^{\ell} - 1}{x - 1}\right)p(x)$$
$$\qquad = (1 + x + \cdots + x^{\ell -1})p(x)$$
as desired.
Finally, we will show that $q$ and $r$ are indeed reflexive. We can re-interpret the reflexive condition as such:
Polynomial $a(x)$ is reflexive iff there is an integer $n \geq \deg (a)$ for which
$$a(x) = x^{n}a\left(\frac{1}{x}\right).$$
We have
$$q(x) = \frac{x^{d + k + \ell}p\left(\frac{1}{x}\right) - p(x)}{x - 1}$$
$$\qquad = x^{d + k + \ell -1}\cdot \frac{p\left(\frac{1}{x}\right) - x^{-(d + k + \ell)}p(x)}{\frac{x - 1}{x}}$$
$$\qquad = x^{d + k + \ell -1}\cdot \frac{x^{-(d + k + \ell)}p(x) - p\left(\frac{1}{x}\right)}{\frac{1}{x} - 1}$$
$$\qquad = x^{d + k + \ell -1}q\left(\frac{1}{x}\right)$$
as desired. Similarly,
$$r(x) = \frac{p(x) - x^{d + k}p\left(\frac{1}{x}\right)}{x - 1}$$
$$\qquad = x^{d + k - 1}\cdot \frac{x^{-(d + k)}p(x) - p\left(\frac{1}{x}\right)}{\frac{x - 1}{x}}$$
$$\qquad = x^{d + k - 1}\cdot \frac{p\left(\frac{1}{x}\right) - x^{-(d + k)}p(x)}{\frac{1}{x} - 1}$$
$$\qquad = x^{d + k - 1}r\left(\frac{1}{x}\right).$$

Solution:
We write degree $n$ polynomial $p$ as
$$p(x):= \sum_{i = 0}^{n}p_{i}x^{i}.$$
Define vector $P\in \mathbb{Z}^{n + 1}$ as
$$P:= \left(p_{0}\ p_{1}\ \dots\ p_{n}\right)^{T}.$$
We also denote $X\in \mathbb{Z}[x]^{N}$ for $N$ some sufficiently high degree (e.g. $N > 2n + \ell$) as the vector of powers of $x$, i.e.
$$X:= \left(1\ x\ x^{2}\dots\ x^{N - 1}\right)^{T}.$$
For a matrix $M\in \mathbb{Z}^{(n + 1)\times N}$, $P^{T}M X$ is an integer polynomial of degree $< N$. Note that if the non-zero entries of matrix $M$ are horizontally symmetric, then the resulting polynomial must be reflexive.

Then their total is the matrix whose entries are $1$ at the parallelogram formed by
$$(0,0),(0,\ell -1),(n,n + \ell -1),(n,n),$$
which is precisely $A$.