The variance in strength comes from competition mostly. More competitors, the more variance.
Honestly, I wouldn't pick the place I'm moving based on SEO. Moving should inform your SEO, not the other way around, which is what I think your original question was anyway.
It will affect it. But you can't control that. Just move everything over on your online listings as soon as possible. If you're an SAB you might not even move the listing if it's just a mile away. I know that's against Google's TOS but "doing the right things" in SEO is equal in importance to "not doing the wrong things". If it's not broke, don't fix it is almost universally my #1 rule. Who knows what could happen in the move (probably will be fine honestly) but I probably wouldn't take the chance.
Now, if you serve customers at your location obviously you will have to change it. Again, you can't do anything about that. Just correct all your listings address-wise quickly.