The answer is that Geoff wins when $N$ is of the form $2^n$ for $n$ odd or of the form $t2^n$ for $n$ even and $t > 1$ odd.

Common remarks. We will say that a set $S$ wins if the current player wins given $S$ as the current set of integers on the board. Otherwise, we will say that $S$ loses. We will let $J(S, \mathcal{T}) = (2S - 1) \cup (2\mathcal{T})$. Note that every subset of $\mathbb{Z}$ can be written as $J(S, \mathcal{T})$ for some unique pair $(S, \mathcal{T})$ of subsets of $\mathbb{Z}$. We will let $[n]$ denote the set $\{1, 2, \dots, n\}$.

Lemma 1. For any set $S, \mathcal{T}$ wins if and only if $J(S, \emptyset)$ wins. Similarly, $S$ wins if and only if $J(\emptyset, S)$ wins.
Proof. Let $(k, m)$ be a move on $S$, and let $\mathcal{T}$ be the result of applying the move. Then we can reduce $J(S, \emptyset)$ to $J(\mathcal{T}, \emptyset)$ by applying the move $(k + 1, 2m - 1)$.
Conversely, let $(k, m)$ be a move on $J(S, \emptyset)$. We can express the result of this move as $J(\mathcal{T}, \emptyset)$ for some $\mathcal{T}$. Then we can reduce $S$ to $\mathcal{T}$ by applying the move $(\max(k - 1, 0), (k + 1)/2)$.
This gives us a natural bijection between games starting with $S$ and games starting with $J(S, \emptyset)$ and thus proves the first part of the lemma. The second part follows by a similar argument.

Lemma 2. If $S, \mathcal{T}$ are nonempty and at least one of them loses, then $J(S, \mathcal{T})$ wins.
Proof. If $S$ is losing, then we can delete $J(\emptyset, \mathcal{T})$ using the move $(1, t)$ for some $t \in J(\emptyset, \mathcal{T})$, which leaves the losing set $J(S, \emptyset)$. Similarly, if $\mathcal{T}$ is losing, then we can delete $J(S, \emptyset)$ using the move $(1, s)$ for some $s \in J(S, \emptyset)$, leaving the losing set $J(\emptyset, \mathcal{T})$.

Lemma 3. If $S$ is nonempty and wins, then $J(S, S)$ loses.
Proof. From this position, we can convert any sequence of moves into another valid sequence of moves by replacing $(k, 2n - 1)$ with $(k, 2n)$, and vice versa. Thus we may assume that the initial move $(k, m)$ has $m$ odd. We want to show that any such move results in a winning position for the other player.
The move $(0, m)$ loses immediately. Otherwise, the move results in the set $J(\mathcal{T}, S)$ for some set $\mathcal{T}$. There are three cases. If $\mathcal{T}$ is empty then the other player gets the winning set $J(\emptyset, S)$. If $\mathcal{T}$ is losing then the other player can choose the move $(1, s)$ for some $s \in J(\emptyset, S)$, which leaves the losing set $J(\mathcal{T}, \emptyset)$. If $\mathcal{T}$ is nonempty winning then the other player can choose the move $(k, m + 1)$, which results in the position $J(\mathcal{T}, \mathcal{T})$. We can do induction on $|S|$ to show that this is a losing set.

Lemma 4. $[2n]$ wins if and only if $[n]$ loses.
Proof. Note that $[2n] = J([n], [n])$, the result follows directly from Lemma 2, 3.

Lemma 5. For any integer $n \ge 1$, $[2n + 1]$ wins.
Proof. By Lemma 4, either $[n]$ or $[2n]$ loses. If $[n]$ loses, then by Lemma 2 we have that $[2n+1] = J([n+1], [n])$ wins. Otherwise, $[2n]$ loses, and therefore $[2n+1]$ wins by choosing the move $(k, 2n+1)$ for $k$ big enough so only $2n+1$ is eliminated.

It remains to verify the original answer. We have two cases to consider:
* Suppose $N = 2^n$ for some $n$. For $N = 1$, every move is an instant loss for Geoff. Then by Lemma 4, Geoff wins for $N = 2^n$ if and only if Geoff loses for $N = 2^{n-1}$, and thus by induction we have that Geoff wins for $N = 2^n$ if and only if $n$ is odd.
* Otherwise, $N = t2^n$, for some $n$ and some $t > 1$ with $t$ odd. By Lemma 5, Geoff wins when $n = 0$. Then by Lemma 4, Geoff wins for $N = t2^n$ if and only if Geoff loses for $N = t2^{n-1}$, and thus by induction on $n$ we have that Geoff wins for $N = t2^n$ if and only if $n$ is even.