ChrisBelgium Why not use deglazer ? The paper lint and the calcium that leaches off the envelopes is creating glaze, or as you say "smooth". The rubber has lost its natural tackiness.
Remove the roller and clean it or use a shop cloth if removing is not an option. Though there are superior options, even using vinegar and water may prove beneficial. Marring the surface of the roller, even marginally, just doesn't seem to be an effective solution. The pores of the rubber are clogged, have become smooth and have lost their grip/transfer ability. Not to minimize previous suggestions, just bringing attention to another option.