I'm sure there are many other factors, but to put my 2 cents in, I would consider whether you'll be sharing an address with anything like a virtual office or UPS store.
I don't think it's uncommon for Google to suspend listings even if they just happen to be in the same building as a virtual office, or very near a UPS store. This of course shouldn't happen, but mistakes like this do seem to occur.
In terms of rankings, I'm sure someone like @Phil Rozek would have more to say, but I'd look at the strength of nearby competitors in rank trackers -- if there are a lot of them packed into the same area, getting visibility is going to be tough against those aged listings.
@FatSniper, what @zach.todd is right on the money, as is that @Rich Owings said. You'll want to find out who else is sitting up in the cheap seats before you buy your ticket there.
How a move may affect your rankings is trickier. If you're in an industry where most searchers really care about proximity - e.g. dentistry, auto repair, private school - then a move will probably shift the part(s) of town in which you rank for search terms without geomodifiers and for "near me" terms. You'll probably rank better in some parts of town, and worse in other parts, if you're competing an industry where proximity matters.
Your rankings are more likely to get shuffled if you've got a high density of local competitors, too, especially if they're dotted throughout a larger city.
You can probably expect somewhat less fluctuation if you've got a service-area business.
I'd snoop around in the Anonymous Ad Preview Tool and see which competitors rank in various ZIPs in or near the city. If it's pretty much the same characters, your rankings probably won't swing too much in one direction or another if and when you move.
The variance in strength comes from competition mostly. More competitors, the more variance.
Honestly, I wouldn't pick the place I'm moving based on SEO. Moving should inform your SEO, not the other way around, which is what I think your original question was anyway.
It will affect it. But you can't control that. Just move everything over on your online listings as soon as possible. If you're an SAB you might not even move the listing if it's just a mile away. I know that's against Google's TOS but "doing the right things" in SEO is equal in importance to "not doing the wrong things". If it's not broke, don't fix it is almost universally my #1 rule. Who knows what could happen in the move (probably will be fine honestly) but I probably wouldn't take the chance.
Now, if you serve customers at your location obviously you will have to change it. Again, you can't do anything about that. Just correct all your listings address-wise quickly.
New advertising option:
A review of your product or service posted by a Sterling Sky employee. This will also be shared on the Sterling Sky & LSF Twitter accounts, our Facebook group, LinkedIn, and both newsletters.