There are different kinds of spirits and only earthbound spirits linger in one place. They are the ones that don't realize they have passed on or how much time has come and gone since their passing. I also think dark entities can linger in different places and they latch onto living hosts and feed off their energy or influence them to do things they wouldn't otherwise do.

Spirits that have crossed over, however, can come or go wherever they want.