I drove an auto from 1991 until last year (a Carlton, a Cavalier and my Volvo S40), and loved them. I would never have considered a CVT auto, probably an unreasonable assumption, but to me a CVT is a DAF 55/66, and older motorists know what they were like!
When I drove for Stagecoach it was all automatics (my PCV licence was only for automatics) but I have also driven vans regularly, so I never had a problem swapping.
The only problem I ever had swapping was after a three week trip to California in 1986. After driving an auto (a new experience for me) for the duration, a work colleague brought his 7.2 Litre, 1971 pre-federal spec Corvette Stingray in, on my last day. I was allowed to drive it, and at the first set of lights it struggled to stop. It was not until it stalled (and it is not easy to stall a big block V8), that I realised I had forgotten about the clutch. And I'd been driving manuals for 20 years by then!