I will echo the endorsement of the Sabitoru rust eraser (I have a slight preference for the coarse over the fine version), and it might be that other rust erasers (abrasive-filled rubber) would work as well as the Sabitoru. The Sabitoru is a favourite among flat stone sharpeners for cleaning stones between flatenings; as Darrell noted with his SJ, it cleans stones up like new.
Gord
Gord