There are some IT tropes that have been around for so long one takes them for granted. There is a belief that Unix machines, including Linux, BSD and Mac OS X, are better than anything. Not only is Unix better than Windows, a fact taken as a given in IT, but those who manage them are also better, smarter and more logical than Windows admins. Not to mention more “pure” than other admins. (Richard Stallman is perhaps the leading avatar for this belief, whether he believes it or not.)
Now, Paul Venezia is telling us about “Nine Traits Of The Veteran Unix Admin”. I thought immediately of that Dilbert strip with the bearded Unix guy (and he is almost always male) who gives him a nickel and tells him to “buy a better computer!”
His points can only make sense if he is tongue-in-cheek, or else he has never done any IT for anyone. Going down some of them:
Veteran Unix admin trait No. 2: We use vi, not emacs, and definitely not pico or nano
While we know that emacs is near and dear to the hearts of many Unix admins, it really is the Unix equivalent of Microsoft Word. Vi — and explicitly vim — is the true tool for veteran Unix geeks who need to get things done and not muck about with the extraneous nonsense that comes with emacs. Emacs has a built-in game of Tetris, for crying out loud.
I’ll grudgingly admit that the bells and whistles in vim such as code folding and syntax highlighting might be considered fluff, but at the end of the day, real Unix work blends extremely well with vi’s modal editing concepts. In addition, its svelte size and universal portability make it the One True Editor. Thanks Bill, thanks Bram.
I’ve used a lot of different editors in nearly 30 years of IT, on many different systems.
I’ve used some very primitive line editors; many of you may remember EDLIN in MS-DOS, but editors like it were also common on mainframes such as the CDC Cyber 170 I used in college. I remember using TECO, Emacs’ predecessor, on the PDP-11 machines I had access to, and was delighted to use the full-screen version that was available for the VT-52.
When I got my first Net access via a Unix shell, I learned Emacs and that’s all there was to it; no arguing, no fuss. If I had to learn Unix to be on the net, then there it was. On our Amigas at SATV, Memacs, the Amiga port of this editor, was the one to use. Other Linux distros I have used have had vi, vim, nano, and pico, to say nothing of whatever GUI editors came with Gnome, XFCE or KDE, which are all a blur to me.
I didn’t care what editor I needed to use just as long as I could get it to edit whatever config file I needed edited at the moment.
You care that much about vi to make that strong a point? Get a life.
Veteran Unix admin trait No. 3: We wield regular expressions like weapons
To the uninitiated, even the most innocuous regex looks like the result of nauseous keyboard. To us, however, it’s pure poetry. The power represented in the complexity of pcre (Perl Compatible Regular Expressions) cannot be matched by any other known tool. If you need to replace every third character in a 100,000-line file, except when it’s followed by the numeral 4, regular expressions aren’t just a tool for the job — they’re the only tool for the job. Those that shrink from learning regex do themselves and their colleagues a disservice on a daily basis. In just about every Unix shop of reasonable size, you’ll find one or two guys regex savants. These poor folks constantly get string snippets in their email accompanied by plaintive requests for a regex to parse them, usually followed by a promise of a round of drinks that never materializes.
Regexs have been a part of Windows from at least VBScript, and are fully included in .Net and PowerShell. Just to say, although I know it is not supposed to count. It’s worth noting that I studied compiler design in college in 1987 and regexs are one of the first topics covered in the Dragon Book.
I can’t imagine what specific problem is so simple that you can change every third character in a 100,000 line file except when followed by four and not run into problems with that scale, regardless of what the solution is programmed in. Regex or not, how would you verify the output?
But, derrrp, I stupid Windows guy. I no know regexs!
Veteran Unix admin trait No. 4: We’re inherently lazy
When given a problem that appears to involve lots of manual, repetitive work, we old-school Unix types will always opt to write code to take care of it. This usually takes less time than the manual option, but not always. Regardless, we’d rather spend those minutes and hours constructing an effort that can be referenced or used later, rather than simply fixing the immediate problem. Usually, this comes back to us in spades when a few years later we encounter a similar problem and can yank a few hundred lines of Perl from a file in our home directory, solve the problem in a matter of minutes, and go back to analyzing other code for possible streamlining. Or playing Angry Birds.
I consider myself lazy in that way too. Funny though, that it’s a virtue. You’d never know it reading Slashdot. To hear some tell it, when Unix guys are lazy they’re brilliant—even when they’re bad lazy with planning—when Windows guys are good lazy (time-saving strategic laziness), they’re stupid.
Veteran Unix admin trait No. 6: We generally assume the problem is with whomever is asking the question
To reach a certain level of Unix enlightenment is to be extremely confident in your foundational knowledge. It also means we never think that a problem exists until we can see it for ourselves. Telling a veteran Unix admin that a file “vanished” will get you a snort of derision. Prove to him that it really happened and he’ll dive into the problem tirelessly until a suitable, sensible cause and solution are found. Many think that this is a sign of hubris or arrogance. It definitely is — but we’ve earned it.
Snort! I think that is a trademark in the Linux community, WorksForMe! Now, when a Unix admin (like Mr. Venezia?) finds something genuinely wrong, does he go to the user who raised the original question and apologize? Naw, he’s always right; it’s everybody else who’s antisocial!
Veteran Unix admin trait No. 7: We have more in common with medical examiners than doctors
When dealing with a massive problem, we’ll spend far more time in the postmortem than the actual problem resolution. Unless the workload allows us absolutely no time to investigate, we need to know the absolute cause of the problem. There is no magic in the work of a hard-core Unix admin; every situation must stem from a logical point and be traceable along the proper lines. In short, there’s a reason for everything, and we’ll leave no stone unturned until we find it.
To us, it’s easy to stop the bleeding by HUPping a process or changing permissions on a file or directory to 777, but that’s not the half of it. Why did the process need to be restarted? That shouldn’t have been necessary, and we need to know why.
Naw, that Windows I use is made by pixies, and is even run by pixies. As I’m writing this, some pixie is messing around with one of my GPO’s at SATV and I have only spent a few hours trying to get down to said cause which is driving me nuts right now. Hur hur I derrrrrrrr!
Veteran Unix admin trait No. 8: We know more about Windows than we’ll ever let on
Though we may not run Windows on our personal machines or appear to care a whit about Windows servers, we’re generally quite capable at diagnosing and fixing Windows problems. This is because we’ve had to deal with these problems when they bleed over into our territory. However, we do not like to acknowledge this fact, because most times Windows doesn’t subscribe to the same deeply logical foundations as Unix, and that bothers us. See traits No. 5 and 6 above.
The Unix-Haters Handbook is very old, but sadly, it is not much less relevant than it was in the day. It’s pointless to go over it in length, but it does have something important to say still: Systems like Unix, like Windows, like everything, are designed with assumptions and use-cases. Unix has escaped its research origins and has far surpassed its original reach, yet its practitioners are still stuck in the 1970’s, when they thought it was God’s gift to computing.
As if there weren’t other systems out there long before Bill Gates used his first computer. As if I’m going to spend hours in awk picking apart some CSV file or process list when I have built-in objects in Powershell that can give me way more discoverable information than bash could even provide on a good day. I once used a very nice source-level debugger on the Cyber that could handle multiple languages including assembly. 30 years ago! It was something I have never seen on PC’s until very recently. It wasn’t on Unix at the time, either.
From what I’ve seen in passing in Slashdot, most Slashdotters, who seem also to overlap the Unix admins that Mr. Venezia talks about, are rather incurious about Microsoft. Except when the next security exploit comes out, in which case they know everything there is to know about Microsoft and security (supposed to be an oxymoron, right?)
Otherwise, the Unix administrator who really does know Microsoft, and knows the flaws of his own application software too well, might not be so arrogant.
But Unix admins wouldn’t be real admins if they weren’t arrogant alpha-hotels who hated the people they worked for, would they?
I say that tongue-in-cheek, Paul. Really.
Image of my system’s Task Manager, during a typical session with Outlook, several IE tabs and Windows Media Center (watching NASCAR via my Hauppauge card.)
Since my last post, the whirlpool that is Devil Mountain Software/XPSNet/Randall Kennedy has become more turbulent still. The other shoes have dropped furiously in true Iraqi fashion; President Bush was never so beleaguered.
Devil Mountain Software is a business Kennedy established that specializes in the analysis of Windows performance data. There is no Craig Barth, and Kennedy has stated that this fabrication was a misguided effort to separate himself (or more accurately, his InfoWorld blogger persona) from his Devil Mountain Software business.
Integrity and honesty are core to InfoWorld’s mission of service to IT professionals, and we view Kennedy’s actions as a serious breach of trust. As a result, he will no longer be a contributor to InfoWorld, and we have removed his blog from this site.
It’s the right thing to do. InfoWorld is polite enough to say that they appreciate his performance insights. I don’t. I can never trust him again. That leaves Bob Lewis as the only guy I can trust from that site.
ZDNet just pushed up an article originally scheduled for Monday publication, “Why We Don’t Trust Devil Mountain Software (And Neither Should You)”, due to Randall’s self-revelation. This article confirms a number of my suspicions.
On the software:
As for the software itself, the installer is not digitally signed. It installs two Windows services: Cfwtracker.exe and Cfwupload.exe. The tracker program adds information at regular intervals to a database (in Microsoft Access format) stored in the user profile of the currently logged-on user. The upload module periodically sends that data to a remote server.
I’d started analyzing the MSI downloadable, in which 7-Zip tells me there are two binaries. I had been about to run Orca, Microsoft’s install tool on the file when this story broke.
XPNet claims to use an SSL connection to send its data from clients. Not true:
We found this claim to be untrue. In our tests, using machines in widely separated geographic locations, the DMS software made simple (non-secure) HTTP connections on port 80, transmitting data to a server at IP address 22.214.171.124. The IP block at 66.115.28.* has DNS A records that point to devilmount.com, xpnet.com, and csaresearch.com. All of those companies are registered to Devil Mountain Software and include the name Randall C. Kennedy in the registration information.
When we attempted to use a browser to make a secure connection to https://xpnet.com, we received two certificate errors. The certificate associated with the site, originally issued by Equifax Secure Global eBusiness, had been issued to a different domain, csaresearch.com. In addition, the certificate had expired on September 7, 2009.
Performance data is uploaded to the exo.performance.network, where your most recent one week of data is stored for viewing and analysis. Performance data will be shared in aggregate only and never identified as linked to your individual account.
We saw how well that was honored.
Dignan’s article goes on to talk about Randall’s claims that his software was used at several large Wall Street firms, including Morgan Stanley and Credit Suisse. Why would they use a small third-party company to provide this service?
I should note that the only way for a user to get performance figures with ClarityNet is through the XPNet web site. There are no local gauges for the user to query. Firms like Morgan Stanley have large IT operations. They won’t rely on an outside website when Microsoft makes their performance data readily available to internal analysts. Morgan Stanley’s IT (or as it happens, IBM Global Services) is truly responsible for their client machines and they know it. They won’t knowingly enter into an agreement with XPNet. (With all of IBM’s resources, they will hire a two bit operation for analysis?)
If they have, Mr. Kennedy has many more problems than just being fired from InfoWorld. This isn’t over for him.
But hopefully, it’s over for me. I don’t want to make another pundit post again today.
UPDATE: It’s not over for Paul Thurrott. No doubt more people will weigh in on Monday, like Ed Bott and others.
The amazing aspect is, on Randall’s exo.blog, he’s been backpedaling in the comments, kissing up to Mr. Bright and claiming his data didn’t really show disk thrashing and so forth. Mr. Kennedy,
Stop digging that hole.
I hope those financial customers of his can afford counsel and seek it out. I would be delighted to see Randall get those cease-and-desist letters for once.
I haven’t made an IT Punditry post in a long time. I haven’t missed doing that; I am too cynical of the IT press to enjoy punditry; most of the reporters want to be Morley Safer or Bob Woodward, but aren’t and won’t be. Others just cynically latch on to the latest factoid that can be spun to get page views.
The latest tempest in the IT press involves a researcher who claims Windows 7 uses more memory than previous versions, “alarmingly low memory”. Ars Technica’s Peter Bright, and Computerworld have reported on this and the comments have been furious.
The claims and counterclaims center around a feature of Windows 7 called Superfetch. With this feature, Windows extends the prefetch feature in XP so that it aggressively caches programs in memory that are most commonly called up by the user, based on his or her usage patterns.
The question has been whether XPNet’s performance metrics have taken this into account.
The performance tool, DMS ClarityNet, works, as far as I know, by taking certain memory-related Windows performance counters, sending them to their server, combining them into a proprietary performance figure that is calculated and displayed on a web-site widget.
I say, “as far as I know”, for good reason. Though I do remember trying out the program a few months before this controversy—I’m still listed as a customer on their website, I don’t recall if I kept it on my machine. I’m always trying out various admin and performance software to evaluate for myself and SATV. I seem to recall I didn’t like the tool and uninstalled it, for specific reasons I can’t remember. I try and discard many programs if they don’t work or they don’t do what I thought they’d do or they’re unsuitable in some other way.
Unfortunately for XPNet, I’ve found a good reason never to try the program again: Mr. Bright’s personal machine’s performance data was divulged!
But Mr. Bright didn’t stop at simply attacking our intelligence. He took the additional step of actually contributing data from his own test PC, the one he claims shows the lie in our data. And it was at this point that the story got interesting.
You see, by connecting his PC to our network, he made his raw system metrics data available to us. And after reviewing this data, it became clear why our System Monitor widget flagged his system as being low on memory.
It’s because it’s true.
A look at the Committed Bytes counter values collected from his PC…
What XPNet just did here is a Game Over by anyone’s standard of ethics.
Whatever reason I have had to try their client has totally evaporated.
There are real companies like Pingdom that monitor connectivity, uptime and availability for their customers. Could one imagine what would happen if they made a blog post saying, “Oh, we think IIS sucks. Here’s a stupid customer of ours who uses it. See how bad their availability is?!”—and they named the customer!
It would be devastating, not least to Pingdom. They would be out of business.
Devil Mountain Software, according to their business model, analyzes performance for financial-based clients.
They have no reason to break out their data by the individual machines involved.
None at all.
If I were a client of theirs I would be an ex-client. It shouldn’t even be possible for that exo.net blogger to even have non-aggregated individual data!
And they have financial customers?!?
In my first draft of this blog post, I was going to explain how I thought the metrics were flawed and how I have not seen the performance problems that DMS claims either on my personal network or SATV’s.
But with this development, it is beyond technical. Any technical points I could make are now irrelevant. I have to consider the DMS ClarityNet client as malware, on the same level as those fake antivirus programs that cause so much grief. Or the Sony CD rootkit.
I have no compunctions about reverse-engineering this client to find out what exactly it sends. I’m sure someone will if I don’t.
More astonishing, and what finally made me post this, is this morning’s post from Exo.Net, calling out the comment of one person who agreed with them and seeking to end the discussion on that note.
Past posts were signed “Performance Team”. This one: “Randall C. Kennedy”.
Houston, we have a problem…
The earlier blog, The Survival of New Orleans Blog, is the quintessential account of survival in a datacenter during a disaster. It was, and is, recommended reading for every datacenter manager and IT professional.
SATV usually doesn’t have more than snowstorms and power outages to worry about, but the thought of a hurricane is never far from us. Since we haven’t had a hurricane since Bob in 1991, it’s argued that we are due. Most New England hurricanes happen in the fall, when most of us are in our back to school routines, and SATV is busy with students and preparing for Halloween festivities later on.
So far, the worst disaster we have experienced is a two-day blizzard in December 2003 that put us off the air for two days due to a power failure and a failed network switch. We have no generators, and our UPS systems are designed only for short-term power glitches.
In a lot of small organizations, people are too busy putting out fires and worrying about short and long term problems to think much of disasters. In fairness, most DR literature is oriented towards large companies that can put resources to bear on DR. There’s not a lot towards companies like SATV other than “put the backup tapes in a safe place”.
Any DR plan for SATV would have to bring together not just staff, but our board, volunteers and everybody in Salem, to get us out of an incident and back on the air.
Are we up to it? So far…
Paul Thurrott (“Forget About XP- Let’s Save Windows for Workgroups 3.11!”) and numerous others have written about Microsoft’s retirement of Windows For Workgroups, which has had quite an afterlife in embedded systems.
(This is not new: The 1800-series Red Line trains on Boston’s MBTA are reported to have an “A>” prompt on a console in the driver’s cabin; DOS 6.22 has been seen in embedded systems, and probably in those trains.)
Paul made a nice banner:
Windows 3.11 is special for me; it was the first Windows version I ever used.
It made me think of another “special” persona at Microsoft, Clippy! He had a retirement party a few years back. He made the rounds to various Microsoft events around the country. He happened to ask me for career directions and I obliged:
I hope he’s having a good life.
Microsoft has filed a very scary patent. The original purpose seems benign, according to Unwired View:
In addition to many benefits brought to us by mobile phones, there have been a few drawbacks as well. Especially, related to ethics/culture/social issues of the mobile phone use.
Don’t you just hate it, when during an engaging presentation, show or movie, a mobile phone of some as#$%^&&, sorry, forgetful person, begins to ring? What about someone taking out his high end cameraphone and doing something with it in the locker room? Can you be sure he’s not taking your nude pictures in the shower? What about someone secretly recording confidential conversation on his mobile phone?
Microsoft seems to have an idea how to solve all these problems at once. By creating device manners policy DMP [sic]), to which all mobile devices will have to comply to. And they even want to patent it
[From the Microsoft patent:]
Such policy may be used to communicate to various mobile and other devices the “manners” with which compliance is expected or required. Similar to some of the social manners honored among people, such as with “no smoking” or “employees only” zones, “no swimming” or “no flash photography” areas, and scenarios for “please wash your hands” or “no talking out loud”, devices may recognize and comply with analogous “device manners” policy.
This won’t stop there. I can anticipate lots of potential "no photography" zones:
- Schools: "Behind Every Camera is a Pedo!"
- Government buildings: I guess I can’t film public hearings anymore like I used to.
- Public places: Remember, papparazzi!
- Workplaces: Maybe you’ll be allowed to take pictures of the office party…if you’re good with the boss! You’re a whistleblower? Siberia might be safer.
- Anywhere where someone in power is scared of their own people.
That last is the scariest place of them all, for it is everywhere in America. Everywhere in our country, people take pictures of things the powers-that-be would rather not have seen.
Youtube has video on atrocities in Myanmar. Perhaps in a few years there’ll be smuggled videos and photos from America.
I live in the “tourist town” of Salem. What if the Peabody Essex Museum wanted exclusive photo rights for the whole city?
Suppose it was implemented. I wonder how many public places would eventually have the "no pictures" flag set? Meaning even if it’s legal to take photos, you couldn’t… Then how many police cars and federal agents would have DMP boxes with "no photos / no video / no audio recording" flags set. No more individuals documenting arrests or police action.
Someone, somewhere in the Department of Homeland Security, is smiling.
Nick Corcodilos, "Ask the Headhunter" is one of my favorite IT pundits; he’s an executive recruiter who loves to puncture the job-hunting myths I suffered through early in my career. You know the ones, "send resumes everywhere" (now, "live on Monster.com"), have the "right" resume, and the "right" answers.
He had a great blog post a few months ago that I’m just now getting to, Why Johnny doesn’t work. He asks:
The dominant explanation for why students aren’t graduating with technical degrees is H-1B and outsourcing. It goes like this: Because American companies send technical jobs overseas, and because they hire foreign nationals under the H-1B visa program, (both supposedly at lower cost than hiring Americans), students regard technical careers (in electronics engineering, software development, information technology) as undesirable. They believe they won’t get healthy salaries or enjoy any reasonable job security. They may be right.
But I see another trend that’s far more disturbing than the behavior of companies and students. K-12 schools seem to be de-emphasizing the fundamentals of technology. They seem to be teaching kids how to be technology consumers rather than designers. A case in point is my local school district, which recently spent over $30M to build a state-of-the-art middle school. Every classroom is wired for sound, video, and computers. Every teacher has a laptop, and big LCD displays dot the facility. The auditorium is state-of-the-art; the soundboard alone blows away what you’d find in most commercial theaters. The school is equipped with a video production facility that kids use to produce what’s described as professional-quality videos. The computer lab lets kids use sound samples to produce their own music CD’s. It’s all really great.
The trouble is, no one is teaching the kids how all this technology works, and how they can build their own.
I grew up reading. When I got my first jack-in-the-box, I took it apart. (Mom wasn’t happy…then!) I played with electronic stuff from an early age. I read the old Lafayette and Radio Shack catalogs and I was fascinated! I could tell you how a TV worked when I was in fourth grade. During college, I had a very tiny side business taking TV’s and electronics from dumpsters and fixing them for family and friends.
A lot of people like me, in my generation, went into IT through their fascination with electronics and computers. Like Nick, I’ve seen people become mere consumers of technology and I’m worried.
A story from SATV: We have three rack-mount Windows 2000 machines that handle our entire on-air operation; one machine is the master machine with the database and Channel 16 display (Government channel), and the other two handle the displays for Channel 3 (Public Access) and Channel 15 (Education).
One day, during a routine Windows update, Channel 3 went down, and very hard. The CPU fan would not spin up, even after it was swapped for a known good one. We pouted on that one for a day, waiting to hear from the vendor’s tech support (they were turnkey systems.)
Next day, I have a bright idea. A few years ago when those systems were new, we had one we thought was unstable due to hardware (it turned out to be a Windows bug, another story). We got a replacement machine but never returned the old machine for some reason.
Why not use the old machine? I took the hard drive from the dead machine and swapped it into the old, supposedly unstable machine.
It worked. It ran for many hours. Success! If I had had that idea the day before…
The fun came when we explained this to the vendor! The poor tech support guy was floored when Sal, our executive director, explained everything that transpired. He just could not conceive of someone doing what I did to get a machine going. We asked him to check with his boss–the vendor has a long, involved, but mostly pleasant history with us. His boss called us back, "Oh, it’s SATV, I know all about you!" (We’re trading in those systems towards a forklift upgrade from them including a video server so this’ll be all out in the wash soon.)
Even in the computer industry, most people don’t seem to realize computers are made up of components. Even Macs. The modern education that Nick talks about seems to make kids incurious. Why do you need to know how your iPod works? The Chinese will make more! They can handle the science and engineering, don’t worry about science, just buy stuff!
One commenter on Nick’s post made me think of an even sadder example:
Third, technology has radically changed the methods of “making stuff.” If you want a state-of-the art shop, it better have a CNC mill, laser cutter, and CAD workstations. I experienced this kind of obsolecense first-hand in high school. I was on the school newspaper, and spent two years learning all the skills of offset printing: photo screening, making full-size (11×17 inch) negatives of page folios, burning printing plates, and running the offset press. Then the district office bought an 11×17 photocopy machine. In that instant, all that equipment and skills were worthless. But our time from layout to finished product went from 3 days to 2 hours, and at lower cost.
A few years ago, I volunteered to be the technical person for the Salem Commission on Disabilities (before I joined them). There was a contest among elementary school students to design a logo for the Commission and a winning logo was picked. Now we needed to get that on our letterhead.
I had volunteered to scan the logo in on my computer, but the commissioners were adamant that it be done "professionally" and several of us went to someone we knew at North Shore Vocational Technical School; this guy was teaching the graphics arts class.
We met the guy and asked if they had a high-end scanner. He did not. It was all film and plates. This was only a few years ago, and by then, digital press was very well established; I’d attended enough Seybold trade shows to see that. Digital press should have been available even at that school’s budget, and I was very sad for him and his students.
In the end I scanned it myself and the results were good enough. If that teacher, or more likely, his department head had just been more curious, his students would not have been sent out dead on arrival in obsolete technologies.
Don’t fear. If you’re a student all you need to do is get your MBA or law degree, Make Money and Make Deals! Engineering, science and curiosity is for the Indians and the Chinese.