This is most certainly the water pump bearing going south... It depends on how long has it been like this but from my experience it's not a critical condition, not yet. The pumps can function in this condition for quite some time, without leakages or noise. I don't want to promote irresponsibility and say you should not fix it, just sharing my experience
Unfortunately I'm in a similar condition. I've
replaced my water pump a few months ago, it has just a few hundred miles now. When it was brand new I tried to wiggle the shaft on it right when I pulled it out of the box. It moved a liiiiitle bit, which I thought it shouldn't but I was like whatever, maybe it's an acceptable condition after all (it was quiet, as opposed to the one I pulled out of my car)... Currently my fan can be moved the same way you say. And I can even do this a bit with the belt on, which means that it would probably move a mile back and forth with the belt off
No noise, no leakages. My previous pump also did this, also without leakage. But noises came, so I changed it.
Any coming-and-going vibrations in the gas pedal btw (due to eccentric movement of the fan shaft)? I feel like my vibrations are coming back a bit which makes me think that it all comes down to a defective aftermarket pump right out of factory... Or worse: I may have bent the fan when I was changing the pump and now it's out of balance and eating pumps