Change Displayed Text SizeGrow Displayed Text SizeShrink Displayed Text Size

Sunday, November 02, 2003

Edge cases

At "V" we were always looking for new technical talent. Because we were so pressed for time on everything, that meant we dealt with headhunters - "placement agencies" - quite a bit. We got all sorts of people in for interviews. Some had not even been told in advance that we were an adult entertainment company. Needless to say, that lead to a lot of interesting first days. They almost never backed out during the interview - we'd tell them we were a porn company and they wouldn't bat an eyelash. Instead, they either wouldn't show up the first day, or they would have a really interesting first day. Day to day, we didn't actually see much in the way of nudity or anything like that. We dealt with servers, software, and routers - not content (we had a non-engineering team for that). Nonetheless, wether because of the adult nature of the work, or because of the tempo and skill level we operated at, a lot of our candidates quit after the first day. One memorable geek left for lunch and never came back - after a few hours we found the note he put under his mousepad.

We did manage to hang on to a few people. I myself came from a headhunter, and one guy we hired stuck around a while. I can't for the life of me recall his name, which should tell you how much of a team player he was. He had an impressive resume of DoD jobs - one item that stands out in my mind was that we worked on a version of the STU-III encrypted communication system. Since we occasionally called him Lou Diamond Phillips - inside joke there - we'll just refer to him as LDP from here on.

Now LDP was given a project to work on. It was a mass-mailing system, a double opt-in mailing list system. It's fairly straightforward Perl coding, nothing all that special. A web front-end for the opt-in stuff so that people could sign up twice, and of course the mailing components, which ran on their own server. Every few days it would send out a few hundred thousand emails (note: never use sendmail for this kind of thing. use postfix).
LDP worked on this largely on his own. He never talked much. He had long-ass ET-looking fingers, and you could see him typing whenever he was at his desk. Looked like something from the Sci Fi channel. Now, when LDP took his long, long lunch breaks, we pretty much never noticed. Sometimes I would go out to my car for something, or take a walk just to get out of the office, and I'd see his van parked around the corner with him in it, doing who knows what. Often these little lunch breaks of his would go 4 hours or so. But he got his project done, so we didn't hassle him about it. After all, as long as you get the work done, we don't care how you manage your time.
The night we launched his mail application, we did it in phases. Since we had a cluster of load balanced servers, we turned it on on 1 or 2 servers, made sure it worked, then phased it in on the others as it took in more traffic.
When we got to server number 3, everything stopped.
It turned out that LDP's code wasn't all that solid. Sure, it had passed our very informal testing process, but that doesn't mean much. LDP's perl code was weak as a kitten and dumb as a sack of hammers. While it would have passed a CS professor's scrutiny, anyone who has programmed in the real world would have laughed at it. In a couple of places, he wasn't handling known error conditions - like if the user selected nothing on a web form, or hit submit twice. The rate that the error logs on the servers filled up with the evidence of his inexperience was high enough that inside of 5 minutes, all 3 servers had full drives.
Needless to say, we made a few changes to his code at that point and redeployed it. We didn't lose anything major, and it was a big wake up call to him. When the shit hit the fan, he paniced and didn't even think to look at the error logs to see what was going wrong.

A month or two later we let him go. He was working on the database and made a mistake that blew everything away - and then tried to cover it up. He never told anyone else, we found out anyway, and let his sorry ass go.
After he was gone, we cleaned out his desk and found a huge bottle of Maalox, candy bars, and some other realllllly questionable stuff. Months later he was trying to file for some kind of unemployment and we got asked all kinds of questions about his termination.

LDP was just..... super weird.

I told you it would be about porn Krissy!

11/02/2003 10:43:00 PM ] [  0 comments  ]
A good quick laugh