Occasionally my car won't start the first time, I wait 1-2 minutes and then it starts fine, why?

I have a 2004 Impala with approximately 81k on it, base with 3.4 v6. It has been doing this for at least a year and a half. Most of the time when I go the start the car its starts right up the first time with no hesitation. But every once in a while, maybe 1 out of 10 starts or 1 out of 20 starts it won't start the first time but if I wait a minute or two it starts fine. I am not doing anything wrong with when starting the car (I put the key in and turn it like everybody else does). When it does this I can hear the engine trying to start and then it dies. I never changed the fuel filter or spark plugs and my mechanic and other mechanics I have been going to have not told me my start is bad or weak or anything. Just to let you know this car has no special key and back in September of 2009 a few of my keys started acting funny in the ignition (i had to hold the key while turning and even then they got stuck sometimes, so I got rid of those and I have been using the keys that turn easily instead of risking damaging the ignition and the keys I use now that turn easily are more worn than the ones I got rid of, it makes no sense to me. Thanks for any advice.