There are several different arrow lubes available such as Woody's or Scorpion Venom that will help prevent this. There are a lot of other things such as soap or tire black that some use for arrow lube, but I haven't found anything that words as well.
However, if you already have it on your arrows use a copper scrubbing pad. A scotch bright pad might also work.
The copper scrubbing pad is also good for cleaning arrows for re-fletching.