Found the problem.
Under System Settings > Keyboard > Shortcuts > Windows, you can set shortcuts to tile windows left and right. I set these to use Shift+Super+Left/Right.
What I didn't realize is that by default you can use those keybindings to shift windows between monitors. (This option is not visible in that shortcuts page.) I found it in dconf-editor under:
Essentially I had the same keybinding for two separate functions. This seems to have caused a race conditon when working with multiple monitors as sometimes it would work shifting between screens the rest of the time it would just tile it.
I've now changed it so tiling is done with Shift+Alt+Left/Right and have found that shifting windows between screens now functions as expected.
Hope this is useful for someone in the future.