Of course my hyper-intelligent children are the exception to this rule...
By being even brainier than me.
Well, far be it from me to extrapolate to what that says about your wife...
I've only read some of the first page but in answer to the initial question could it be that the lower level humans (pretty crude I know) produce more offspring because of there lower chance of survival. Just look to nature and any number of eco-systems, you will find an organism that will over produce naturally because only a small number of offspring will survive. Just like in rougher areas of communities, having more children may increase the chance of successfully passing on your genes from one generation to the next.
We may see wealth/intelligent/etc. as the highest level humans but when it comes to natural selection survival of the fittest is all that matters. Fitness could just come down to ability to procreate better.
I think what's happened to human society is basically entirely as Arete has said - survival rates (in first world countries) are high enough, with a large enough margin that we are able to pool our resources to help those who would otherwise perish, which in turn gives them the time needed to procreate. Conscious thought and the existence of contraceptives leads many people to NOT want many kids - a trend which seems to, on average, show an inverse correlation between intelligence/education and number of offspring. In this way, traditional genetic indicators of reproductive fitness (body shape, fitness, confidence as well as a host of social traits) are being countered by what some might say is actually a "hive mind" mentality. By this I mean that some people don't want any/many kids because they consider the world overpopulated, or they believe competition for resources is or will become too severe, and as a result, they sacrifice their innate desire to pass their genes on for what they see as the improved survival of the human race.
As you say, "fitness" is not determined prior to procreation, it's only determined as a statistical concept after the sample population has lived and died. What constitutes fitness to survive (and reproduce) changes with the surrounding environment.
Personally I find this whole field incredibly interesting, as I feel that conscious thought and the existence of cooperative societies (ie civilisation in general) is the reason why humans are expected to restrain many instinctive desires. The "hive mind" concept has built up from the very basic idea of cooperation for improved chances of survival, possibly something like this:
1. Cavemen - ok mate how about this, we don't kill each other to get to food, and we hunt together to bring down or trap prey we couldn't catch alone.
2. Primitive nomadic tribes - righto boys, nobody's allowed to kill anyone else in our tribe, or steal stuff from other members of the tribe. We hunt together and the women can do other stuff like prepare food and shelter and look after the toddlers.
3. Early settled societies - as per nomadic tribes, but now we've worked out how to farm. Respect the owners of land, blah blah. Invade other people who aren't part of our society and take their stuff cos that's way easier than finding it or building it ourselves. Integrate them into our social hierarchy so that we all end up working together anyway - even if they're just slaves.
4. Modern society - righto, I don't wanna get up the duff cos I don't want kids, so you better use a condom. If we have kids they'll have to deal with oil/food/water shortage blah blah and I think that's bad for the environment because I own an Apple laptop and vote for the Greens.
As a result of all my ramblings, I think it's interesting that innate emotional reactions such as sexual desire, anger etc are considered bad for society and need to be tightly restrained most of the time, when these instincts are all genetic leftovers from eras where they were necessary simply to survive and reproduce. Various moral codes (religion, laws, etc etc) have been brought in over the past millennia by the various controlling bodies of society in order to try to explain/legitimise the apparent emotional contradiction of feeling angry/horny/whatever and not being able to unleash the physical reaction.
But ultimately I don't believe in evolution, because to quote one member of a clearly well-educated creationist group, "if we evolved from monkeys, howcome we can't speak monkey?". You simply can't argue with that...
edit: I should mention I have little to no formal education on this topic. Pretty well everything I have just said is a wild extrapolation of stuff I've dreamed up when I was drunk.
