It's Always Cables
There’s a system admin saying: It’s always DNS
But weird problems aren’t always DNS problems. Better than 50% of the time, they’re cabling problems.
Once upon a time, circa 1993, I worked for a hardware reseller. Somehow, against all odds, this company got a contract to sell to Harris Corporation, notoriously a cut-out for intelligence agencies. We believed that we were selling to the NSA. It could be that was true.
I got the assignment of writing scripts/programs that would prove to a Harris Corp. rep that the system we were selling them met a list of requirements. One of the requirements was that the system could keep its system time synced using NTP, which sparked a life-long interest in me in my systems having the correct time.
The requirement that caused me to mistrust cables was that the system should recover in 15 or 20 minutes from a power failure. Naturally, I had tried this, and it worked, but it made me, a programmer, nervous. When the time came for that particular test, I claimed that I didn’t want to actually pull the plug myself. The Harris Corp. rep pulled the 120V power cord out of the wall socket, and reinserted the plug. Everything booted, but the system could no longer perform its task, part of which was to pluck data-grade VHS tapes from a large, multi-tape carousel. I left the debugging to the hardware people, which revealed that pulling the power cord had destroyed the RS-232 serial cable that connected computer to multi-tape carousel. There should have been multiple transformers, fuses and circuits between wall socket and RS-232 serial cable, so it was probably coincidence, but…
The less said or remembered about flat ribbon, IDE cables of yesteryear, the better. IDE “busses” could have 2 disk drives. The cable determined the disk drive address on the bus. The standard connection was a 40-pin (two rows of 20) rectangular thing. At first, they just had something printed on them to indicate orientation, but later manufacturers came to their sense and put little keys on plugs, and slots on sockets so that assemblers could mostly get them installed correctly.
Ribbon cables had more faults than merely causing humans trouble. They tolerated a low number of plug-unplug cycles before breaking, so it was a crap shoot to install a new disk.
Shielding can make a difference. At one point, I had a SPARCStation 1 clone on my desk. I often had it opened up so I could fiddle with the cabling. During the winter of that year, my office mate could remove his nylon shelled down jacket, and the static electricity that generated would reboot the machine. Guess what? The internal cabling I fiddled with was unshielded ribbon cables.
I’ve had a difficult to see problem with a USB-C to RJ-45 adapter network. The RJ-45 jack had wiggled out by no more than 1/16th of an inch, which caused the network connection to disappear. Luckily, due to knowing that It’s always cables, I fixed that right away.
Speaking of network cables, CAT-5 cables mysteriously quit working sometimes. You don’t even need to unplug them and move them around, they simply quit working. CAT-5 cables with weak/missing clips gradually slide out of sockets on servers that don’t move or get re-cabled. It’s always worthwhile to re-seat CAT-5 cables, and to check the little LEDs that prove electricity is pulsing in the wires. There’s a reason why ethernet sockets universally have indicator LEDs.
It’s cabling at least 50% of the time.