User's banner
Avatar

Christopher Wood

cwood@awful.systems
Joined
7 posts • 52 comments

Reading, Shadowrun, walking. Living and working in Toronto. Sysadmin (or whatever it’s called this month). He/him.

Direct message

I can’t tell which of these profits are in actual currency and which are in fake money.

permalink
report
reply

As usual, the business fundamentals thing happens after the compensation has been paid out.

permalink
report
reply

I’m getting the picture that governance is a great thing until you find out that other people want to govern you back.

permalink
report
reply

The encouragement of a situation where you disconnect with those outside, the sleep deprivation, the drip of hints that you’re not meeting the standard, the trust in the great leader.

It also sounds corporate, yes.

permalink
report
reply

You know how sometimes you use a grocery app and it’s fairly obvious that the people writing them don’t spend time in grocery stores? I’m getting that same impression here.

permalink
report
reply

With so many parts of tech operating like a mixture of religion and fandom this would be the atheistic answer. (This is my diametric opposite of a sneer.)

permalink
report
reply

I think we’ve all walked by a giant important point.

These nearly-all-male network state fans have such compelling ideas that women outside their immediate circles would rather Xerox “bits of their bodies” than engage with those ideas. Their outreach “embassy” attracts even fewer women every day. Possibly even an average number rounding to zero.

Right now it seems like their polities will be remembered in the same religious studies lessons that teach about the Shakers.

https://en.wikipedia.org/wiki/Shakers

permalink
report
reply

It’s probably good to situate in time when thinking about these things. The twin towers were how a lot of companies became examples of what location redundancy really means. These days people are keeping that lesson well in mind, but back then, not so much.

permalink
report
parent
reply

Once upon a time I was, for employment reasons, part of a team providing customer support for police forces’ booking equipment including RHL, Solaris, HP-UX, and Tru64 servers. If you were arrested in some specific parts of the USA in the early 2000s it’s likely your PII travelled across a server I had logged into on its way to the FBI.

One specific police force called us about a red light. It turned out that half of their two-disk RAID1 array had failed. Then it transpired that they had not been rotating the backup tapes. Or even putting in the tape for backup. After some discussion it turned out that their server was in a grimy, dusty janitor’s closet instead of, say, under a desk or in a spare office. Which is why it had been out of sight and mind and getting clogged with filth.

I was asked to do a checkup on this server and see how it was. Of course this was after 3 PM on a Friday. The server seemed on the face of it fine, the RAID array was working on one disk, there were no errors on the box, and so on. Apart from the dead disk everything was fine.

(While I was being finicky with this host it got late and somebody turned off the lights and I yelled “I’m still here turn the goddamn lights back on!” or words to that effect and it turned out I had unintentionally cursed at the CEO with whom I had less than ideal relations. So it goes. He seemed more copacetic than usual. He left and I got on with things.)

Eventually I was finishing my little audit and my very junior self (job title: Technician) was wondering how little work I could get away with in my correctly lazy sysadmin style. For the first time I thought the thought that has guided my actions with systems ever since: “If I stopped here, would it be okay if a problem happened later?”

I called the police force and said given circumstances I needed to take a cold backup of their Oracle database and their booking equipment would be down for a bit. The response was that this was fine given arrest volume only picked up later on a Friday anyway. I merrily took down a whole police force’s ability to book suspects to cold-backup their Oracle database onto the third disk in the host (secondary backup mechanism or something, purchasing is a weird art). Then I turned them back on and had them do a smoke test and grabbed the bus home.

I had myself a happy little weekend in the era before cell phones and when I arrived a bit late on Monday morning my workplace was in a rather unprecedented uproar. Readers, the second disk in that police force’s RAID array had failed and taken with it their ability to book prisoners and their built up years of criminal intelligence data.

(In this situation the civil rights clock ticks and judges do not accept “well our computer systems were down” for slow-rolling delivery to bail hearings as much as the public thinks they do. So if this isn’t fixed a whole bunch of innocent and/or gormless and/or unpleasant people run wild and free.)

I was called over by the Executive Vice President of Operations and asked about the database. I said in my then-typical very guileless way “oh, I did a cold backup onto the third disk”. It was like everybody had just exhaled around me.

If I recall correctly the job of restoring the Oracle database was delegated to J. who was a great Oracle DBA among other talents. Well, as soon as a disk arrived. In the meantime the police force dug out their inkpads and paper from somewhere.

This was a superior lesson in the fragility of computer systems and why the extra mile is actually no more or less than all the required miles.

permalink
report
reply