Jacob seems to quite like his Pentax Optio W20. The big advantage is that it has no moving external parts, so it's quite durable (and waterproof, though that doesn't seem like much of an issue for you, except maybe around the high chair). I believe they have them down at CompUSA on Market if you'd like to play with it before recommending a purchase.
It does take movies; it takes about 1.2 seconds to start up; and all of the current crop of point-n-shoot digitals are about the same ease of use. Some people really like the Canon Powershot user interface, some people have other favorites.