Rust on the outside of a bolt will not hurt anyting.

 

Take the belt off and turn the pulley by hand. It should be smooth and free of any rough spots and there should be no play up or down left or right. If there is anything other than a nice smooth feeling when you turn it then yes replace it.

Massey