There are good reasons for not wanting to go out in the wet
1) Bike gets a lot dirtier and needs cleaning.
2) Fine if you have all the gear, but not everyone can afford to have the right jacket, trousers, gloves, boots etc. and it's miserable if you get wet and cold. Gear also has to be kept in good condition as water will get into any broken seams, so again more costs for repairs/replacement.
3) Obviously the road surface is a lot more slippery on the white paint, man hole covers, studs but also diesel spills, gravel etc. I tell all my associates to ride the same in the dry as they would in the wet e.g. go through the black on zebra crossings, then it's second nature rather than having to change your riding style. Of course there is a trade-off because if you never go out in the wet you never get any practice. Doing some advanced training or being a tutor/observer is good from that point of view because we don't avoid the wet (only ice & snow).
4) Can be difficult to see if you don't have the right visor/insert. You also can't ride with your visor up if you wear glasses.
I had one visor that was scratched and the water would run into the scratches. I've since changed that.
Also another one (Schuberth C2) that was dreadful for steaming up. I've now got an insert for that, but it's all money, so it's extra costs which not everyone can afford all the time.
I think the biggest thing I hate about riding in the wet is not being able to see properly.
