Doesn't sound like you used one of these...
[ame="http://www.amazon.com/3M-Stripe-Wheel-Mandrel-Diameter/dp/B00063VT0G/ref=sr_1_1?ie=UTF8&qid=1458517230&sr=8-1&keywords=3m+rubber+wheel#customerReviews"]3M Stripe Off Wheel 07498, 3/8" Mandrel, 4" Diameter x 5/8" Width, Beige (Pack of 1): Power Rotary Tool Accessories: Amazon.com: Industrial & Scientific[/ame]