Question: Why are my RHEL5 NICs going to sleep? First ping fails, then it wakes up.
Without some more detailed info, here are two guesses on what could be occurring.
This sort of sounds like a common ICMP / ARP issue. When a source device that does not have an entry in it’s ARP table for the destination device, it will always drop the first ICMP request while it sends out an ARP request, resulting in the behavior you describe – the first ping fails, and the rest arrive successfully. Check out this article for an example.
If this RHEL5 instance is a clone of another server, make sure the identifying information has been removed after the clone. Specifically the MAC address in /etc/sysconfig/network-scripts/ifcfg-ethX which can result in a duplicate MAC address, and potentially packet loss on the network.
If you are using managed switches on the network, it’s fairly easy to test for situation #2. You’ll potentially see MAC address flapping between interfaces.