Answer: $1 \le a < 2019$.

It is clear that $a \ge 1$, since $a_1$ is a positive integer less than or equal to $a$.

Denote by $\sigma(a_n)$ the remainder of the division of $2019$ by the sum of the digits of $a_n$ in base $4038$.

We consider the sequence $\{b_n : \sigma(b_n) = 0\}_{n=0}^{\infty}$ formed by the numbers such that their sum of digits in base $4038$ is divisible by $2019$. We have $b_0 = 0$, $b_1 = 2019$, etc.

For any positive integer $n$, the numbers $\{\sigma(2019n), \sigma(2019n+1), \dots, \sigma(2019n+2018)\}$ form a complete remainder system modulo $2019$ (because $2019n, \dots, 2019n+2018$ differ only in the last digit). Therefore
$$2019n \le b_n \le 2019n + 2018. \qquad (1)$$
Let us consider arbitrary sequence $a_1, a_2, a_3, \dots$ which does not satisfy the condition, i.e. only finitely many of its terms $a_{i_1}, a_{i_2}, \dots, a_{i_\ell}$ satisfy $\sigma(a_{i_j}) = 0$. Then there exists a positive integer $M$ such that $a_m \in \{b_n\}_0^\infty$ for every $m \ge M$. This means that for every positive integer $N$ we have $a_{M+k} \in \{b_n\}_0^\infty$, $k = 0, \dots, N$.

Assume that $a < 2019$. Since the terms of the sequence are mutually distinct, we conclude that
$$2019N \le b_N \le \max_{0 \le k \le N} a_{M+k} \le a(M+N) \Rightarrow N \le \frac{aM}{2019-a}, \forall N, \quad (2)$$
which is a contradiction because of the choice of (fixed) $M$ and $a$. Therefore every $1 \le a < 2019$ gives a sequence with the required properties.

For $a \ge 2019$, the sequence $1, b_1, b_2, \dots$ satisfies the restrictions $a_n := b_{n-1} < 2019(n+1) \le a(n+1)$ for every $n$ and $\sigma(a_n) = 0$ for every $n \ge 1$. Therefore, only the first term of this sequence has sum of digits in base $4038$ which is not divisible by $2019$. Therefore $a \ge 2019$ could not bring new solution.