When buying a used car, especially one that is 12+ years old, what remains of the original bits and bobs that came with the car can vary quite a bit. That said, Honda drivers tend to be older, and more careful than average. I was lucky with ours, it was a cheap high miles car but came with a full history, all the books, and a box file stuffed with receipts - lovely.
A 2006 car should come with two keys with integrated remote locking, a spare wheel, probably a spacesaver, jack, simple toolkit and towing eye in a vinyl pack, a folder with the manuals and service book, a key codecard, and a locking wheelnut key, if it has locking wheelnuts for the alloys.
Ask to see the logbook, they should have it in their possession, but the seller will normally send it off for you, so you won't get to keep it. A private seller may pass the logbook to you, but dealers and many private sellers will insist on sending it off themselves.
If anything is missing, the minor bits can usually be found easily at a scrapyard or on ebay. A missing key can be replaced, but they aren't cheap for a genuine one, so insist on two keys - you will need the key codecard that has all the details the dealer needs to get you a new key, so make sure you have that. There is no master key with these.
I think the main extras with the SE are alloys, have a good look at those as they scuff easily, a better music system and electric windows in the back. There's probably a few other bits, but don't overlook a good S model - alloys of that age are nothing but trouble, and you might be thankful of a set of steel wheels in the end.
The Jazz is generally a well made and reliable car, but it does have it's share of niggly problems which can be fixed relatively cheaply. The main deal breakers I would look out for are gearbox and steering problems.
Listen for whining in lower gears, and a dry bearing noise in neutral when you let the clutch up. The engine should be very quiet indeed, so these noises do stand out more than they would on other cars. The electric power steering is not heavily assisted, so can feel heavy compared to other small cars, but it should be easy enough to turn when stood, with the engine running. Look out for any notchiness or stickiness in the steering, especially at speed. Try to take the car for an extended drive if you can, as this problem can manifest itself when it's warmed up. This can be a difficult and expensive problem to rectify, and not uncommon. Many mk.1 Jazz owners just put up with it, but I found it very irritating indeed.
If you are spending any sort of money privately, make sure there is no finance or logbook loans outstanding on the car. Logbook loans aren't enforceable legally, they have no legal charge over the car, but it doesn't stop these companies sending heavies round to the current registered keeper - it's just hassle you don't need. However, if there is finance outstanding, the car is not the keeper's to sell.
Finally, I'm not sure how old you are, but as a new driver you need to get insurance quotes for a Jazz. They are not the cheapest small car to insure, when I was looking at cars for my daughter, the cost of insuring a Jazz made it a non-starter.