Saturday, August 09, 2008

If I Don't Do It, It Don't Get Done, And Even If I Do Do It, It Don't Get Done

The pool is green again.

Looks like the pump quit maybe during the thunderstorms on Wednesday, but it may have been out since the weekend because I didn't actually check the pump on Monday and didn't have a chance to do anything pool-wise during the week on account of getting home very late each night (work problems exacerbated by LIRR's inability to get me home in the time they allot for the job).

Yesterday afternoon, late, The Stevieling finally remembered to check the chemicals like I had asked her to, and reported that the water was "teal".

The new contact lenses she's wearing these days are obviously no use whatsoever. The water was a bilious green reminiscent of the colour at the end of July last year (and we all remember that that ended the season with two months of fruitless industrial chemistry, 24 x 7 electric pumping and no swimming followed by a pool closing) and that of the pool opening that resulted in me throwing up my hands and draining the whole damn thing and starting again. I am so mad I am ready to hit everyone with something expensive and heavy.

Why is it that the other people in Chateau Stevie cannot get a clue? The whole family sees me washing filters, checking the water, has actually heard my lecture on what happens when the Azathoth-damned pump does not run correctly (delivered at the conclusion of The Stevieling's screwing up the vacuuming last year to the point the pump failed for several days) so why am I the only one that checks whether or not the bloody GCFI protection has thrown - as it always does in a thunderstorm if the pump is running - and resets it so the water will filter and stay clean and we can swim and cool off? Is it really so hard for the women of this house to get a grip around the idea that if the pump doesn't run, we end up watching pea soup cook in the heat of August instead of soaking in cool water, that latter being the sole purpose for having the Azathoth-damned pool in the first place? I mean, they know I don't do all that filter nonsense for fun because I've told them I don't in as many words.

My current projects at work are not going well.

One of them, given me three months ago, involves moving some data from four disk cabinets and a production computer to four different disk cabinets once owned by a test computer, then reconnecting the cabinets so that the old production disks are on test and the old test disks (actually new disks put into test to test the cabinets worked) become production disks. This job would normally be done using three sets of disks but we just don't have 'em, so I needed to disconnect the test computer from it's disks, effectively taking it down, for the duration of the move.

The reason I'm doing this and not the guy traditionally our Admin contact with the users and the software vendor? Because he has become a tad disillusioned with the others and become surly and uncommunicative. The users don't like him or the vendor. He doesn't like the users or the vendor. The vendor doesn't like either of them.

A dream project.

I haven't the faintest idea how all this equipment is connected, how it is configured or even where it is located so I can actually have a look. The original admin guy did all the cabling and is the only one who knows how it all goes together. Nothing is written down.

Obviously I need to speak to him, but it takes a month before he will loosen up enough to show me what I need to know. I get so desperate in the end I write down a bunch of dumb questions and send them to him. He just returned my list on Friday, two months after it was sent.

I explain to the user that I need their test computer, cannot proceed without it in any way, shape or form, and that in three months I'm going to Canada and that they shouldn't dilly-dally. I do this every two days for two months, finally writing an e-mail to the effect that unless they release the equipment we cannot go forward. We have by now begun experiencing an interface problem on the production computer that means when a disk goes bad, neither the disk cabinet display panel not any of the server's software routines can tell us which disk it is. One tool tells us a disk has gone bad and the contents relocated to another, but we can't find the bad one to replace it. Then it happens again. Then it happens twice on the test computer too.

At this point, my boss's boss demands to know how I plan to relocate the data. I tell him that the original admin used replication software to do it and that's what I was planning to use. He refuses to countenance the use of a license for the purpose, and asks why I can't use a technique called "mirroring", in which we tell a different piece of software to replicate the data so that each file is copied simultaneously to two different disks1. I explain that I could have done that, and had been planning to all along, but had lately discovered that the lack of space had forced the original admin to swap some of the mirrored files for RAID arrays, a technique that uses clever sums to patch missing data2. It runs slower than mirroring, but uses much less space. RAID arrays cannot be mirrored. I am sent away having been told that I should use "traditional Unix copying methods then", and plan on doing that for a quiet life.

Two months after requesting the test computer go off-line for this "essential" upgrade (to no effect), I get an e-mail from the users asking why disks are failing.

I engage the original admin and he tells me he feels the interface problem is a show-stopper. I would, under other circumstances, agree with him but I have my marching orders. He suggests we recable the cabinets to another computer with a different interface, one that doesn't suffer the silent treatment issue, and run some diagnostics. I feel this is a great idea and we do it.

He didn't mention that these diagnostics would still be running eight days later.

Meanwhile, my boss's boss decides the project is foundering for want of manegerial input (again). He calls me in and rants that he doesn't understand why I'm not going to use his idea of "walking" the RAID data over to the new disks (a technique that files used for certain spooling3 purposes can use, configured from within the user's software). I explain that it is because I am not possessed of the ability to guess what he wants without his telling me.

He then demands to know whether the original admin (with whom he apparently discussed this with two months earlier in a meeting about the project I was inexplicably not invited to) had discussed this with me. I answer that no, in fact, he hadn't, and that this was the first I was hearing of it. He bitched and whined but couldn't refute my essential premise that if he wanted me to act on suggestions he would actually have to suggest them to me rather than relying on some sort of ESP-driven idea-osmosis.

I went from his office to the original admin's desk. He said he hadn't bothered to tell me about it because it was "beyond stupid". He had found no way to do it. It could not be done.

I storm off to the user and ask if this walking spool file of inconvenience plan is possible. She doesn't know, but doesn't think so.

I install the user's software on my desktop and take a look. Then I write to the vendor. Between us it takes two minutes to confirm it can be done that way. I see my boss's boss and tell him I'm on the point of confirming that his method will be workable. One in the "win" column? Not necessarily with my management.

The original admin suddenly yells that he's found the way to do it, because he's finally gotten off his Unix-admin hobby-horse and lowered himself to using the GUI tool rather than trying to do everything with Perl scripts4. I hiss "Rule Britannia!" under my breath, and begin reading the e-mail which my boss's boss has just sent me insisting that I use his walking method, explaining the reason why he wants to do it that way (which I understood when I left his office an hour before and which I told him I understood) and citing the original original admin as having done it that way.

The original original admin was a consultant5 who was universally lothed during his tenure and left his ultra-remunerative position of 5-plus years rather than undergo an investigation into certain behaviour of his by a watchdog department we all answer to here. I pointed out that although he might have indeed walked on water, the original original admin had never in his contracted duties with our agency written a single thing down in any way, and had deployed web-visible apps on obsolete computers running unpatched software several releases behind the vendors' "known safe" levels, voiding service agreements left and right.

I on the other hand had no problem telling people what I was doing, which was mostly nothing right now since no-one who had knowledge would share it with me and no-one who had equipment would release it to me so I could do what they wanted me to with it. I received an e-mail demanding I appear at 9:30 am on Friday to explain my "plan", then meet with him and the user's boss at 10:30 am to defend it to the hilt.

Friday, I arrived with a new plan in task-list form (although I couldn't do it in Microsoft Project, supposedly the site standard, since no-one had thought to get it installed on my desktop and that software I don't have permission to put up myself) and went to the boss's boss's office to wait for him.

At 9:40 he had still not appeared to interrogate me so I left to see if I could actually do some work. At 10:25 I returned to my desk so that I could answer the call to the second meeting. This meeting I had the time for, but not the location, so I expected an irritated phone call.

At 10:40 I began a systematic sweep of the likely places such a meeting could be held in that encompassed several floors, offices and meeting rooms, all to no avail.

I sent out an explanatory e-mail to the man and set about doing what I could. I got to Task 1, subtask 3 before I was up against "Production computer is recabled", at which point I knew someone higher up would have to give the go-ahead. I sent out an e-mail with the status, and asking who would give permission, who would ask for it (A recent fiasco involving Database Administration staff and an Oracle Install6 made me vary anal about this as you can see).

That took me up to the end of Friday and I went home feeling quite badly shafted by everyone involved.

I had told everyone that I was going to Canada in three months and that we had to hustle so we didn't crowd the end date. I warned them that I couldn't delay my departure. I told them that on no account would starting the job in the last week before I went be a good idea. When I started phrasing it as "take as long as you like. I'm going to Canada in a week and if it needs doing after that it will end up on someone else's desk. Probably your desk now I come to think on it" I started seeing some sluggish movement over and above the normal snail-on-crutches pace that has been standard so far.

My other project has been going really badly.

A pilot for a technology no-one wants, I was teamed with a Unix expert who promptly decided to have nothing to do with the thing leaving me on my tod. I sought the help of another, but in order to gain his help I had to do the job his way and his way is way more involved than needs be for this project.

It has gone nowhere, but is now suddenly super-high profile because some very high-up people suddenly became convinced this would enable us to cross-connect all sorts of unconnected stuff 7. Now there is a consultant from another agency who has somehow elected himself the project lead and put my name on an org chart leading to him, though he has no say in what I do.

I hear that on Monday there will be windows servers running virtual machines all built from disks provided by this chap, and the server I'm building (slowly) won't be needed. I don't care.

I'm going to Canada.

Now take all this, add in a soupcon of the LIRR losing it on a fairly regular basis and taking extra long to get me home, just in time for bed on one night8.

So I think you see that after all this I rather needed a soak this weekend in the swimming pool I spend so much time cleaning.

If only someone else had thought to pick up the slack and, I dunno, check it was working or something.

  1. Although there are two copies of everything, the computer hides the copy from you. If one disk fails, it uses the other one and you don't notice anything wrong
  2. You break the data up into "stripes". You combine the bits in each stripe using an XOR logic function and make an extra stripe of so-called "parity" data from the answer. The thing about the parity data is that if you XOR it back against all but one stripe, it will give you the missing one. Put stripes on different discs and Hey Pasta! Instant no-break data
  3. Computer term for putting something on a digital shelf for a few days until we need it
  4. Unix admins frequently refuse to use GUIs on principle. This isn't the first, or even the thirty-first Unix admin I've seen hoisted on the petard of insisting on doing things the bottom-up hard way instead of the top-down easy one just because that's 'the right way". I, being at heart bone-idle, suffer no such prejudices
  5. The same one I had trouble with in the old OLTP job
  6. I was told to "get a copy of Oracle from DA and install it". This I did, and DA were a bit snippish. Turns out, we usually ask DA staff to do all our installs and there were now noses out of joint. Now I want the wording of any request made from my own management nailed down twice as firmly as I would were the request to be from another department
  7. It won't, at least not as easily as it's been sold to them upstairs, and no-one has thought about what the load on our servers will build to when we are spending machine cycles we don't have to send info to others for work we don't do. This is, of course, a budget war issue. Someone wants the load on our budget and the credit on theirs
  8. Parked in Hicksville, 3/4 of an hour late because of "switch problems", we were bemused to hear an announcement that the Ronkonkoma train behind us was running on time. We all screamed and clutched each other, expecting to be rear-ended by the hurtling train that needed the line we had been stopped on for lo! these 45 minutes. By a stroke of luck it turned out that the station announcer was a machine and, like all LIRR machines, totally effing naff


Anonymous said...

For these reminders of wage slavery and managerial incompetence, I thank you.


Stevie said...

Yeah, I know these "work" posts lack a vital something, but what the hell as the great Spike Milligan once said (in A Dustbin of Milligan, at the end of the story about Madame Legerts).

Sorry it took so long to clear your comment Gil. I was in Canada and my dad only has a spiffing 21 kbps dial-up link, making any internet use an exercise in frustration.

I swore to avoid such while I was there.