Dirty IT jobs don’t always look so dirty at first glance.
Dressing up like Tom Cruise in “Mission: Impossible” and breaking into a secured facility sounds like a blast — until you’re trapped for two hours in the freezing rain waiting to be rescued. Think writing sexy games would be fun? Imagine poring over endless photo sets of explicit anatomical closeups.
Whether you’re trying to squeeze big data into tiny spaces, moderate arguments between angry geeks, or hack code that’s so old it qualifies for Social Security benefits, you’re doing a dirty but necessary job.
This installment in our Dirty Jobs series features tech jobs that can be physically challenging, mentally debilitating, or just plain irritating. Be thankful that these people are doing them — otherwise, you might have to.
Dirty IT job No. 1: B&E artist
Dressed in black camo, hiding in the woods in the dead of night on the edge of a Pennsylvania mountain; it’s not your typical IT job.
But that’s where Matt Neely found himself more than a year ago. As vice president of consulting for SecureState, an information security management consulting firm, Neely’s job is to test the physical security of his firm’s clients, which include large federal agencies, major retailers, energy plants, and even entire countries. Trained in the art of lockpicking by his previous employer (a bank), Neely uses his breaking-and-entering skills so that organizations can find holes in their perimeter and fill them.
On this cold December night, Neely and a colleague were asked to break into a mining facility just past midnight and steal “trophy data,” while two other SecureState penetration testerssocial-engineered their way in via the front gate. The coal mine was concerned about environmental activists breaking in and tampering with its SCADA systems, causing the mine to shut down. They had good reason to worry. According to Neely, the mine’s external security was so porous that he and his partner were in and out in 10 minutes, or about two hours and 20 minutes less than he’d bargained for. The area around the mine was so remote there was no cellphone coverage, so he had no way to reach the other SecureState team. He and his partner had to hunker down for two hours in a freezing rain before they got picked up.
Roughly 75 percent of the time, Neely says he’s able to break in to a facility without getting caught. On the other hand, he says his social-engineering comrades succeed about 90 percent of the time — and when they fail it’s usually because somebody got tipped off a test was coming.
Neely always carries a “get out of jail free” card on his jobs, listing the names and numbers of company personnel who’ve authorized the security test. So far, he’s never had to use it — even when caught red-handed breaking into a power plant by the cops.
Fortunately for Neely, SecureState’s policies prohibit performing tests on facilities where the guards carry weapons.
The dirtiest part of his job? The ensuing paperwork, he says.
“The biggest difference between a criminal and a penetration tester is that a criminal doesn’t have to spend half his time writing reports,” he jokes. “You get a job breaking into facilities to steal data and you think you’ll be doing all sorts of cool crazy stuff all the time. Instead you wind up spending hours looking for a security hole, there’s a couple of minutes of excitement when you get to break in, and then you spend the rest of your time writing about it.”
Dirty IT job No. 2: Big data backup slave
Yes, we know, big data will transform life as we know it. But there’s a dark underbelly to big data that few people talk about: Somebody has to back up, catalog, and restore all that data.
Stephen Roubelet has seen the ugly side of big data firsthand. As IT/IS manager for engineering design firm RDK Engineers, part of his job is to manage 8TB of data over five locations. In fact, RDK had so much data there wasn’t time to do a full backup every weekend.
“We had built a small server network so that we could spool our data to disc and then to tape,” he says. “We’d start on a Friday night and not finish until Tuesday or Wednesday. With the backups running so long, there would invariably be problems, so I and my staff were constantly checking in over the weekends to make sure the jobs were still running.”
Eventually Roubelet began doing one full backup a month and nightly incrementals, hoping none of the tapes in the series got damaged or the entire set would be lost. But about 15 months ago RDK adopted Actifio, a protection and availability storage platform that allowed the firm to virtualize its data management and simplify the backup process.
“Being the backup guy is like the hazing program for new IT workers,” says Ash Ashutosh, CEO of Actifio. “No one gives you credit for doing a good job, but screw up once and you’re fired. And when you bring that process to a big data environment people usually just give up. The data slave puts on a brave face and claims he’s keeping the data secure, but he’s hoping and praying no one ever asks him to recover something.”
The dirtiest part of being a big data backup slave? Searching for old tapes when you need to do a restore, says Roubelet.
“The first question you have to ask: Is it on tape?” he says. “If so, what tape is it on? Did it spool over two or three tapes? If so, you have to find the whole sequence. Is it stored offsite? You’ll have to contact the storage facility. Did the backup fail on the particular weekend when the file was saved? Our backup software may have crashed and restarted four times that weekend, and maybe one server didn’t fully restart. Inevitably that’s the server where the files you need were stored.”
Dirty IT job No. 3: Sexy games producer
You spend all day staring at pictures of naked women and talking dirty in three languages. Sounds like dream job, right?
Not exactly, says Patryk Bukowiecki, a game producer and manager who worked for a maker of “sexy” J2ME mobile games in the late 2000s. There he produced games like Lessons of Passion Blackjack, in which game players engaged in simulated sex talk with scantily clad models. Bukowiecki’s job was to comb through 300 explicit photos of each model, pick eight of them to use in the game, write “sexy” chat for each girl, then translate it from English into Polish and German.
“I had to make sure the girls talked dirty enough so that guys would play as long as possible and then buy other editions of the same game,” he says. “The texting part was actually pretty fun.”
But those hot models? More sad than sexy, says Bukowiecki, especially after you’ve seen their passport photos, which each model had to provide to prove she wasn’t underage and was doing this of her own free will.
“I had to approach my job kind of like I was an ob-gyn,” he says. “You have to get some distance on these things or you will go mad with all the ideas running through your head. You might be tempted to try and find these girls and save them. Don’t do it.”
The dirtiest part of this job? Telling people what you do for a living.
“You sit in an open space in the office looking at these pictures on your computer surrounded by 20 people,” he says. “They’re all passing by your desk, sneaking peeks at what you’re doing. Then you get home and your wife or girlfriend asks, ‘What did you do at work today, dear?'”
Dirty IT job No. 4: Tech forum flame warden
Tugging on Superman’s cape. Spitting into the wind. Pulling the mask off the Lone Ranger. None of those tempt fate like becoming the moderator of an online tech forum.
Bill Horne discovered this when he naively volunteered to take on the job of moderating The Telecom Digest, the oldest continuously published mailing list and Usenet forum on the Internet, back in 2007.
Horne is no stranger to dirty jobs, having done stints as an IT mortician and network sherpa. But this post tested even his mettle.
“I discovered that the Digest was running on a 1980s-era computer and outdated software,” he says. “I thought upgrading the hardware and software would be the most difficult part of the process. But those issues paled in comparison to the people problems. I often feel like I’ve taken on a job that calls for Henry Kissinger’s negotiation skills, Stalin’s ruthlessness, and Roosevelt’s charisma.”
Horne’s job is to evaluate and edit dozens of reader contributions each day, then choose which ones to publish. He quickly discovered that major contributors were spending most of their time hawking products, while others were using multiple identities in order to argue with themselves. For two hours a day, Horne attempts to douse flame wars fueled by a combustible mix of arrogance and ignorance, snuff out personal attacks, and deal with threatening emails complaining about his own performance — all for free.
The dirtiest part of the job? Dealing with oversize egos seven days a week.
“The hardest part isn’t the mechanics of editing, it’s the politics of dealing with people’s opinions,” he says. “I can’t escape the feeling that some Internauts hang around Usenet just because they’ve found out that it’s a safe place to act like a jerk. If they said some of these things at home they’d be divorced in a month, and if they held forth at work with these poorly thought-out, knee-jerk reactions they’d be fired in a minute.”
Dirty IT job No. 5: Orphaned code debugger
Programming is challenging work. Working on another developer’s code adds yet one more level of complexity. But cleaning up after someone else’s programming mess with nothing other than their badly written code as your guide? That’s when things get really dirty.
Joe Emison is vice president of research and development for BuildFax, which maintains contruction records for more than 70 million U.S. buildings. But seven years ago he did some freelance programmer work for a pair of Web developers who insisted on using ClickCartPro, a shopping cart app written in Perl.
ClickCartPro was a wet hot mess, says Emison. Among its flaws, the code had no indentation, no comments, and no documentation, which means he had to hunt for subroutines and guess at what the original coders had in mind. It used only global variables, so changing the values in one place changed it everywhere else that variable appeared. It made extensive use of eval(), which hides error messages from end-users but also from developers, making it nearly impossible to locate the source of a failure. And the code was more than 100,000 lines long.
“I had to go line by line in a text editor trying to figure out where things failed and why,” he says. “I kept saying to the developers, ‘This program is crap; stop using it.’ But they had invested a lot of time in customizing it, and they had a lot of legacy customers who were unwilling to pay for a new cart.”
Emison says this kind of dirty job is something many programmers encounter, especially when dealing with Web code written at the turn of the last century. Still, it could be worse.
“You couldn’t pay me enough money to work on vBulletin,” he says. “Whoever created that code is some crazy whacked-out psycho.”
The dirtiest part of this job? Unless you’re being paid big bucks, there’s little incentive to do things the right way — which means leaving a mess for someone else.
“If you’re dealing with awful code and you’re just being paid to fix one thing or add one feature, you don’t have much reason to use any kind of good practices in your own edits or additions,” he says. “It’s a bit like stopping at a gas station with a filthy restroom. You’re not going to spend too much time cleaning the toilet. You just want to get in, do your business, and get out as quickly as possible.”
Dirty IT job No. 6: Content hussy
Warning: The following dirty job may prove disturbing to readers with more delicate sensibilities. Please proceed with appropriate caution.
There are now more than 550 million websites on topics ranging from apiphobia (fear of bees) to zygomycosis (fungal infections), with everything in between. Someone has to generate copy for all of those sites, no matter how gross it gets.
One of those people is Kari DePhillips, owner of The Content Factory, an online PR and social media marketing firm that offers ghostwriting services for a variety of sites. It’s not a job for the squeamish.
“We frequently get weird writing projects, but the weirdest had to be when we were asked to create an entire website about vaginal discharge monitors,” she says. “Pregnant women wear them to tell if they’re leaking amniotic fluid. On the surface, it may seem as though we were just writing about different types of hoo-ha leakage, but what we were really doing was saving lives. I mean, somebody has to do it.”
That dirty job fell to one of her male staffers, a 26-year-old copywriter named Ben DeMeter. But DePhillips’ copy-producing career never seems to stray too far from that, umm, area. For example, DePhillips says her first writing gig after college was blogging about adult videos for a well-known men’s site while pretending to be a man.
Even after she started her own company, she was hired by a mom-and-pop shop that made large-format stick-on decals. That shop had an order from a fetish site that specialized in female body builders who were extremely well developed in a region of the anatomy not usually on display to the general public. She found herself writing about wall-size images of, well, you get the picture.
“It was like they were injecting steriods directly into their lady bits,” she said. “I was happy when that project was over.”
The dirtiest part of her job? Avoiding lawsuits.
“Recently a writer we hired posted this photo of downtown Omaha to a blog he was ghostwriting,” she says. “It turns out that image was copyrighted. We ended up getting sued $9,000 for this crappy photo of Nebraska. That was a big wake-up call for us. Since then we’ve strengthened our contracts with writers and double-check every image. And we carry cyber liability insurance in the unlikely case that happens again.”