Normally I give my leathers a quick wipe over with baby wet wipes once a week and if there really bad I give them a clean with car leather seat cleaner (£4.99 a bottle). I'm not saying that's the best way and no doubt it's not overly good for the leathers but I like to change them every couple of years anyway.
Never buy white bike gear if you like looking clean.