Dual Monitor Help

I have my media server box's secondary monitor running XBMC fullscreen and it's fantastic. My problem is that the HDMI out from my media server secondary monitor goes through an HDMI switcher before going to the TV. I do this so I can easily toggle to it when not watching live TV.

Everything works great when I'm watching XBMC but when I switch the HDMI source, it effectively disables the monitor from windows and windows responds by moving XBMC to the primary monitor. That's not a problem at all, until I switch back to my server on the HDMI switch - the monitor comes back for windows, but XBMC is now on the primary monitor so it's not on TV anymore - instead I see the desktop of the secondary monitor on TV.

I have used Ultramon to ensure that when XBMC starts, it's on the secondary monitor, but once it opens on the secondary and gets kicked to the primary, it doesn't get kicked back when the secondary monitor is re-enabled.

So how can I: either ensure that XBMC always moves to the secondary monitor when enabled OR prevent windows from moving it to the primary monitor when it thinks the secondary monitor is disabled? Basically I want it always going out fullscreen on the secondary monitor's HDMI out whether it's connected to anything or not.