TLUG Mailing List

Hi Stephen,

sorry I couldn't reply within daylight hours today. I do have freelance work and today was one of those days, and a busy one.

What do you mean by "a spaced repetition intervals"?

Ah, SRS. Sorry, I thought people would just do a search for it. Please see: http://en.wikipedia.org/wiki/Spaced_repetition

It's used in Supermemo, Anki, and the smart.fm website. I've used the latter two for Japanese study, and for me it was effective.

This suggests that (1) users should be able to customize the
likelihood of certain kinds of questions to concentrate quizzes on
personal problem areas or work needs, (2) recording *per-question*
results for a user is a good idea (I didn't check to see whether you
did that), so that they can see *what* they have and have not learned
from the process, not merely if they've learned *anything*, and (3)
automatically increasing the likelihood of questions that were missed
in the past might be useful.

#2 is done/applied already (if the user is logged in). A spaced repetition algorithm performs #3 as one of it's first requirements. And the combo of the two has the users being exposed to the problems they get wrong more often, which is in part how I address #1.

... Many people have trouble with

forgetting semicolons in C, many others have none, for example. If

you don't have (much) trouble with semicolons, then in your personal

form you should aggregate that kind of bug with similar bugs (or into

the "misc syntax errors" bug type). If you do, you should consider

breaking down the "semicolon error" class according to the kinds of

mistake (omitting the semicolon before "then", omitting the semicolon

for an empty statement, inserting a bogus semicolon in some particular

syntax, ...). The point is that he makes this recommendation *because

in his experience as an engineering process consultant, people have

different bugs in their thinking about a language's syntax*.

In the far future I would like to take the users' combined results history and find correlation statistics to get the grouping of questions in that way.

In the SQL question about "quantification", the last two answers were
identical AFAICS.

This question: http://www.plrev.org/catalog.php?course_name=SQL&q_id=1080 ?

Yes, I see answers #1119 and #1122 were the same (bar some whitespace). I just deleted (unlinked, to be precise) the second one. Thanks for the catch.

Regarding interface, lots of suggestions, involving lots of work. :-)

Fresh eyeballs on a GUI evaluation are invaluable- I appreciate it. I know too well how time consuming it is too. I owe you.

You *may* want to consider restricting "nice touches" to logged-in
users to encourage registration.

Yes, the thing registration gives you is saving your test history so you can receive the benefit of SRS. And I haven't 'sold' that from the front page yet. I guess I feel too embarrassed to promote something that was discovered 101 years ago and been used in some software for close to two decades(?).

To make logging in and registration easier, how about supporting OpenID?

Yes, that's an appealing option. I like OpenID too.

This randomly-generated user ID without a password is an experimental idea to be truthful. My thinking is: if you make someone give you their email and password, you're probably getting the same password they share with more than a dozen other sites. So that puts on your server two pieces of data things spammers and the like would want to steal. To avoid the security burden simply don't collect it, I thought. The user's data is test history- worthless to others- so there's no good motivation for anyone to steal it. There may very well be vandalism-minded incidents though.

There are established proficiency tests for C++ at least; you might be
doing your users a service if you point them at those tests. I don't
know if any are free/net-based.

Not, no free ones out there that I'm aware of.

A separate "kuchikomi" page about the
value of such tests would be entertaining for sure and probably
useful. In fact the pointers could be kuchikomi too, then you don't
have to search for them.

I was planning on pointing out that those looking for the best *challenge* questions should try: http://www.spoj.pl/, http://projecteuler.net/. Similar recommendations by google are http://uva.onlinejudge.org/, http://www.programming-challenges.com/,http://acm.tju.edu.cn/, http://acm.timus.ru/.

The list above is probably not even 1/4 complete.

I found the terse "Wrong" off-putting. I expected an explanation of
what was right and preferably why the wrong answer(s) are wrong. You
personally needn't provide explanations, other editors can do that,
but if there isn't one, a "Sorry, no explanation yet. Would you like
to provide one (as much as you know)?" message could be automatically
generated.

Yes, you're right. I've been lazy not to put that in yet. I know it's anti-productive to just say "No, wrong", and stop. I'll definitely change the wording and extend the data model for explanations to be saved soon.

My excuse has been partly that the total number of explanations needed is a value somewhere between the count of all questions and the count of all the false candidate answers. I.e. noticeably more input is needed from an editor before a question is complete. O(n) may be a fine solution for an algorithm but an increase of n (=~ 4) for human work is high. I didn't think about taking input from non-editor users.

The explanation could provide a button linking to the Edit
Explanation form (always present; the explanation might be incorrect
or incomplete even if present). If the user isn't logged-in, the
button would link to the login/register form instead.

Displaying an explanation if available probably should be a user
option. There could be a global option with values "always", "when
mistakes are present", and "never", and a per-question hide/show
explanation toggle (available only after answering, I think).

Yep, yep.

Do you have any "fuzz" questions? Ie, taking off on the semicolon
discussion above, for C++ you could have a 5-7 line block. Then
*randomly* insert or remove 0-2 semicolons. (The UI for answering is
not going to be easy, I suspect, but it would be cool if you can come
up with a good one.)

Fancier question types (technically it's types of question-answer sets) are already on my list. They and the one above just aren't a priority at the moment, given the other issues.

The current types are: Multiple option with a single correct answer; Multiple option with 1 or more correct answers; Yes/No and True/False ('fixed format').

Other types I consider for the near future are:

* Questions that require an answer in a text input. The answer is validated through pattern matching.

* Questions that require the user to click on token. Primarily I think this is good for asking a question such as "The function below causes a compiler error. Click on the keyword/token that is the cause."

The nested if question suggested a similar unconventional fuzzable
question: reindent this block with the usual conventions. Again, good
UI is nontrivial, but I think it's possible. Eg, in C/C++ the three
main variables in style are where to place braces, how many columns
are taken by one level of indentation, and whether braces are used for
single statements. If you always provide syntactically correct
braces and simply count levels of indentation, not columns, then a
tabular form like

Reindent line | to level
--------------+---------
|
--------------+---------
[add line to reindent]

could be used to collect the answers. The question would be composed
by more or less randomly reindenting 0-2 lines of 5-7 incorrectly.

Recording per-user versions and answers to "fuzz" questions might be
non-trivial.

Yeah, it does seem non-trivial. I'd also like to avoid making a function for fuzzing C++; then having to do it for C; then for _javascript_; then for bash; etc.

revision [aussie for "review"?]

http://dictionary.reference.com/browse/revise - "3. British . to review (previously studied materials) inpreparation for an examination.". It may be the third sense of the verb but it's one I would use if I want to say I'm going to re-study something. (BTW it's typical that aussie usage matches british usage.)

"Programming language review" might be taken in the sense "an essay or article that gives a critical evaluation" or "a summary at the end that repeats the substance of a longer discussion" I thought. So I use "revision" - does that clash too much with the sense of "version" for you? There just isn't any sexy or snappy word for it, it seems.

Crap, I need a new name. And I already paid for the domain name and DNS service! Wait, no I don't- "plrev" is appropriately abbreviated. I just need to examine HTTP accept-language headers and switch some html body content :p

Sorry I gotta stop here and get some sleep. I'll be away doing Obon for the next four days, off the net.

Re: [tlug] New programming revision site - by a TLUG'er