Mailing List Archive


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [tlug] programmer competency matrix




I'd forgotten about the silliness of the API memorization thing (I'm in
API docs all the time, myself), and so I need some phrasing to fix that.
Any suggestions on phrasing?

You could phrase it in terms of API capabilities. For example, I know pretty much nothing about the Windoze APIs because I try my best to avoid that platform at all costs. But I know pretty much everything that can be done with the LibXML2 API, even though I still have to keep a FF tab open on the docs and look up almost every call to be sure I have the right arguments in the right order (my memory isn't what it used to be ;-).


Either that or something like "knows what various APIs can do and where to find the details of how to invoke the functions". I dunno... that also sounds too lame. Maybe you can leave it as-is and ask the person in the interview how they felt about that question. If they took it as meaning "memorizing things is more important than knowing how to find them", you may have another useful negative indicator.

For all the problems it has, I think that the matrix is far above the
level of a checklist of acronyms. Surely you aren't going to argue that
"no RDBMs experience" vs. "uses MySQL once in a while" vs. "understands
the relational algebra" is no more useful than "RDBMS: yes or no".

Yeah... you're certainly right there. The 1/2/3 rating scale on the list of acronyms gave me no clue as to which box to check for most of them. At least the matrix levels were more clearly spelled out. And it might help prevent the self-overrating problem, since the candidate would be able to actually see the various things he might be called upon to discuss in the live interview.


... working
with someone for three months is better. So why do we bother with
anything else? Cost.

Understood. I've never worked anywhere where we took in anyone who came along with the intention of weeding folks out later. I'm not even sure that's fair to the candidate, even if he was willing to take the 3-month contract.


If you look at where the matrix is in our process, you'll notice that we
ask for it with the resume. We consider resumes to be moderately useful
to disqualify candidates, and basically useless as a positive indicator.

That's interesting. My last three employers asked for a resume after the fact -- that is, after I'd already been hired -- just because they needed it as part of their HR process.


The thing is... you'd probably be better off hiring a really smart guy who has absolutely no experience in your field of business than hiring a mediocre duff who can talk around all the buzzwords. I've seen plenty of the latter and they *do* interview well. However, when the rubber meets the road, all you get is a lot of squealing and smoke.

It didn't escape me that you could (and possibly were) using the matrix as a way of figuring out how self-aware the candidate might be. I believe someone once said: "he who knows little thinks he knows a lot but he who knows a lot has learned his own ignorance". Had I been desperate for a job when I saw that acronym list, I may well have checked off half the stuff there, figuring I could wing it long enough to gain some real knowledge of the technologies involved -- all before anyone found out. At least I might have been tempted to try.

So, up to a certain level, you might want to use the inverse function on the matrix to get a good idea where someone is *really*. If, by chance, you have a real superstar in front of you, you'll probably set the matrix aside early in the interview anyway.

I'm thinking that maybe each "cell" in the matrix could have a checklist. The prose already lists a number of "for instance" items in most cases. Maybe turn those into checkboxes. It's a lot easier to check boxes than to write out a list, it might solve the "higher includes lower" problem, and it would give you more data points for averaging.

... By comparing experience on a resume and the answers to the
matrix, I can tell if someone's rating of himself seems completely out
of whack.

Have you done that? I would think a person with an over-inflated idea of their own accomplishments would embellish both thei resume and the matrix. I've interviewed candidates who mentioned things like "leading projects" and "designing systems" that, when asked how the thing worked, answered with something like: "Oh... that was so-and-so's job -- I just did what he told me needed to be done". Oh... so much for "leading" the project.


I don't know how you can easily weed out those guys without at least a screening interview. Their martix will scream "superstar" just as loudly as their resume. At least screening interviews aren't as costly but you still have a limit as to how many you can screen in a given amount of time and I agree with your suggestion that being able to weed out the duffs early saves a lot of time/money for everyone involved.

Maybe subjective essay questions. Maybe get them to explain why they think FP is better/worse than OO. Or what makes Ruby such an attractive language to ex-Perl fans. Or their own views on why there may never be a "silver bullet" in the area of software design. Or maybe give them a list of a dozen such subjective questions and ask them to make one up on their own
and answer it. I've been wanting to write at least one "advocacy" paper to send around to my colleagues but I never seem to have the time.


These kinds of things you can't just look-up on Google. But it might take quite a bit of original research to come up with a coherent argument. And if the questions are flexible, the potential candidate can draw on their own experience, which gives you yet more insight into whether they are likely to be a problem solver or a problem maker.

After all, that's all you need to know. Will this person, when faced with a problem at Starling, be able to draw on his own experience and the vast pool of data available on the Internet, and come up with an elegant solution even if the problem is not like any he's already solved before.

Given that the matrix takes only about five minutes to fill out
(significantly less time than writing a cover letter), it's hardly
a burden on the candidate, and I think it provides enough useful
information to justify the cost.

Yeah... I guess I have to agree. And having done a reasonably thorough job of filling out the matrix, irrespective of its content, tells you about as much about the candidate as the cover letter does, eh? At least you know if the person is interested enough in the job to put in the time. (BTW, I use a similar tactic with headhunters. If the initial contact doesn't show that the recruiter at least *looked* at my web site (which is linked from my online resume), the mail goes in the bit-bucket.)


---
Joseph L (Joe) Larabell            Never fight with a dragon
http://larabell.org/                    for thou art crunchy
http://thelemicleague.org/        and goest well with cheese.


Home | Main Index | Thread Index

Home Page Mailing List Linux and Japan TLUG Members Links