A week is sufficient, I did a trip back to Italy a few years ago and and I got to Switzerland in just over a day.
Left Oxford right after work on a Friday, stopped in Dover for the night as I have friends there, left in the morning not too early and was in Switzerland around 8pm, mind that I lost a huge amount of time as it poured down all the way through and when it stopped my radiator fan jammed and blew the radiator around Colmar, wasn't fun... but without that problem I would have probably made it there by mid afternoon.
Considering a day and one back you've got 5 days to enjoy there.
Stelvio is lush, it obviously isn't a fast road, take it nice and easy and enjoy the scenery. Then it depends what you're after... With a week I would go through Belgium, stop in Bruges, go south through Saarbrucken and the Black Forest, get into Switzerland, Stelvio, lakes (Como, Lugano and Maggiore), then off to the Matterhorn and back through the Mount Blanc, which is a bit of a rip-off, I think they charged like £20 to go through, might be more.