I think you identified the issue. Don't start and run any modern car for just a few seconds.
There are often various adaptive or learning functions within many ECUs which will store or adapt the last values for all sorts of parameters, and running it for a very short time can catch them out. If they don't get enough time and/or cycles it can be possible for the adaption process to not be fulfilled and it can for example revert to a default value, which is often not appropriate for the next starting conditions.
I don't know if the Jazz behaves this way, but certainly other cars I've had do. My old Yaris would be very reluctant to start next morning if I moved it on the drive when partly warm and only ran it for a few seconds. Running it for a bit longer solved the problem.
Always run the engine for at least 30sec, preferably more like 1min and ideally hold the revs up a bit, maybe 2000rpm or so. This does 2 things, it gives it enough run time to be happy and it also helps keep the plugs clean of wet carbon (plug temp is very sensitive to rpm, 2000rpm will help heat them up fairly quickly).