There's no selection criteria for prevention of listener movement. I suppose the listener could issue a header to exclude them but it sounds like you want to have a local relay to act as a holding poi ...
I'm bringing up this old thread to ask whether it is possible to prevent the automatic moving of specific listeners because of the fallback-override setting?