Re: [tlug] programmer competency matrix

[Joe Larabell <fred62@example.com>]

Had some thoughts about this in the shower this morning...

> > 1. The levels are inclusive, i.e. some one at level 3 must meet the
> > criteria for all levels below.
>
> Yup. That's one thing I like about it; it ensures that you have
> reasonably broad knowledge if you go up a level or two.

I took a closer look at the "matrix" and there are places where I find I 
have some knowledge of stuff under the Level 2 (or 3) column and complete 
ignorance of other stuff listed under Level 1 (or 2) in the same row. I 
have a hard time believing that anyone's experience at any level will be 
"inclusive" of the lower levels because it really depends on what you've 
been exposed to -- and for most people, that's a bit of a crap shoot.

Plus, after the first block of questions it seems to get into stuff that 
would only be useful on specific projects or at specific companies. It 
kindof reminds me of this one "non-interview" (long story) where I was 
given a list of acronyms and asked to rate myself from one to three on 
each one. Also pretty useless as a measure of one's value on a team.

Then I remembered a Joel Spolsky criteria for hiring: "Smart and gets 
things done". He claims that as technology, development tools, and 
programming styles change, specific knowledge of today's applications has 
no connection to whether the candidate will be a productive member of the 
team in the long term. It's more important that someone be able to think 
on the fly and quickly look up stuff when they need it than to be able to 
regurgitate empty facts and not be able to put them to use when the 
situation doesn't exactly match their past experience. I've seen lots of 
people who can spout off buzzwords in a convincing manner who collapse 
utterly when tossed into a situation they've never dealt with before.

And about this time there comes the row about APIs that claims that a... 
what... level 3, should have memorized most of the major APIs -- that just 
seems worthless to me. That's what books and the Internet are for -- so 
you don't *have* to memorize APIs that will probably disappear as soon as 
you move on to your next project. POSIX? I know enough to know what to 
look for and where, should someone, someday, ask my to write a piece of 
software that conforms to the POSIX spec. I don't think it makes me less 
productive at work if I have to look up the specifics of stuff like 
that when it comes up.

In the end... I think the programming tests are probably the best measure. 
That and having the candidate explain, in great detail, what challenges 
they faced on their last project and how they solved them. And maybe 
putting them in front of something new -- maybe an error message from the 
O/S -- and watching what they do in their quest to find a solution. I'd 
give extra credit to anyone who said: "Google" in response to more than 
one interview question. And anyone who instinctively copies the error 
message itself into the Google search field has aced the interview, IMHO.

For a sysadmin, sure... maybe having some stuff memorized means you'll be 
able to get the system back online in minimum time. But for programmers 
(which is *my* field and that of the Starling web page in question, I 
would guess), there is so much stuff out there that I'd be surprised if 
anyone existed on the entire planet who has experience with even 10% of 
the available tools, environments, and libraries out there. So what you 
really need is not someone who can regurgitate APIs at will but someone 
who is smart and gets things done.

Skip the matrix, stick with the open-book programming tasks, and toss the 
guy into something he's never seen before and watch what he does. That 
should give you a pretty good idea whether he's someone that will add 
value to your team.

---
Joseph L (Joe) Larabell            Never fight with a dragon
http://larabell.org/                    for thou art crunchy
http://thelemicleague.org/        and goest well with cheese.