There I was doing the usual ‘yum update’ then the next thing ’segmentation fault’ – agghhh! Anything I tried with yum resulted in the same problem, so turned to the underlying rpm commands instead. Same problem, anything to do with rpm resulted in a segfault. Oh dear, or similar sentiments thinks I, best check what’s going on elsewhere on the system – all seemed fine as usual, still got web server, mail etc.

I tried digging around in yum, rpm and message logs – nothing of any interest, so I remembered that you can do a rpm rebuilddb as detailed on rpm’s site however, that also failed with a segfault so it was obviously something a little more serious, but at least now I’d backed up the /var/lib/rpm directory which I’d need later.

This stumped me for a while, so I started looking at a lower level using ’strace -f rpm’ to see where things failed – always at the same point just after loading /lib64/tls/libc.so.6 – so I uploaded, did I mention this machine is remote, a replacement copy of the underlying libc-2.3.4.so and rsync’d it into place – no change rpm still segfaulting. After pondering, googling, trawling and no enlightenment later, I headed over to the nice folks at #rpm on irc.freenode.net, who were initially equally puzzled. Then, jbj (I think), mentioned about using rpm2cpio.sh – a shell script to unpack an rpm file when rpm is not installed / working etc. – which I’d obviously not spotted before.

So here’s how it was fixed – so I don’t forget, and because it may be of some use to someone else :