I can second the Bar Keepers Friend.
Remove the clip if you can, and gently scrub with BKF and rinse well. May take a few applications.
Optional next step is to seal it to keep oxygen away from the areas where the plating has chipped off. Start by getting it very dry by holding it under a blow dryer and then (after it cools) I hit it with a layer of clear-coat matte spray varnish. This is to seal it so it doesn’t start to rust again. Then I put the clip back on the pencil. This may freak some people out, but I was concerned that exposed to the air rust may start forming again.
