Re: work times & accommodation @tokyo, WAS: Re: [tlug] Embedded linux dev wanting to find work in Tokyo.. Seeking advice.

[Curt Sampson <cjs@example.com>]

On 2008-07-25 06:19 +0900 (Fri), Stephen J. Turnbull wrote:

>  > C++, Java, Ruby and Haskell, since I have reasonable knowledge of all of
>  > those.
> 
> But you say below that in Google's situation, you see no reason not to
> substitute Python for Ruby, and every reason to do so.  So what you're
> saying is that Google's whole mistake is not to add a modern
> functional language to the mix.  How about a declarative language like
> Prolog?  More to the point, where's SQL?  Surely Google uses SQL out
> the wazoo.

Actually, a *more powerful* language. I think trying out Prolog would be
a fine idea, too, though my impression is that Haskell can do most of
what Prolog can do, and opposite is far from the case.

As for SQL, I'm sure Google uses a reasonable amount of that, as well,
and they certainly should be working on a better replacement for it, if
they use RDBMSes extensively. I know I am, though it tends to be a rather
difficult problem these days given the lack of available options.

> And what problem does use of Haskell solve for Google?

Programmer productivity, pure and simple.

I could spend an hour or two writing here to get into the details,
but there's already plenty of material out there, and I'm sure you're
perfectly capable of doing the research yourself.

As for the following, I'm going to address a few of your points just for
fun. But you're missing the big point here, and I'll get to that near
the end.

> Google's big problem is concurrency. So is STM a standard part of GHC 6.8?

Yes. Not to mention very good threading support. It's getting the first
non-research parallel GC in 6.10 this fall. Scaling up for massively
multicore machines is a one of the big research thrusts in GHC.

Of course, all of this addresses only part of Google's concurrency
problem. They really need systems distributed over networks to truly
scale. And that's where Haskell would shine over C++ and Java; they
could develop better ones, with fewer bugs, in less time.

> How well-debugged is it?

Seems pretty good from what I can see. People use it, and I'm not
hearing any complaints.

Keep in mind, STM is hardly a new thing; we've had it since long before
GHC even existed. It doesn't use new locking mechanisms or anything like
that. The new thing is the composibility of transactions.

> Can you refer me to the HOWTOs on using it to solve google-sized
> problems?

Yeah, right, like Google needs HOWTOs.

> How does this interface with modules written in other languages that
> don't have STM?

As far as concurrency goes, if you can cons up combinators in Haskell
that use these things in other languages, it works just fine. If not,
well, you're kinda stuck. Rewrite in Haskell. It doesn't take long.

But all these are sort of an aside; little points that miss the big
picture. Even if Haskell doesn't work out, if you're sitting there
saying, "We're planning to work at our current level in C++ for the
forseeable future," you're stuck, and someone's going to start working
with better tools and come and eat your lunch one day. In this area,
Google has stopped innovating, as far as I can tell. I'd be happy to be
proven wrong.

> We now return you to your on-going discussion.
> 
> Well, then, iterate the problem.  It's the managers' managers who need
> to deal with comfort levels, and guide their people (the first level
> managers) through change.

And when it's coming from the top? When I talk to CEOs who firmly
believe that the solution to increasing the company's programming
productivity is to add more programmers, outsourcing to India to add a
*lot* more programmers when you need a lot more productivity, what am I
supposed to do then?

The CEOs are often not technical guys. It's the responsiblity of the
tech. and research staff below them to bring them up to speed.

> Sure.  However the world is also a lot more complicated than you make
> it out to be.  Agile programming is a very local technology.  That's
> not to say it can't be applied enterprise-wide, but it has to be done
> by smallish groups on a group by group basis.

I don't think I'm making it out to be simple. I'm not saying, "dump
all of your current project management systems move immediately to
agile instead." I'm not saying "dump all of your languages and move
immediately Haskell instead." I'm just saying, "stop with the attitude
that you've got the right systems right now, and start experimenting to
find out where you can improve things."

>  > > Also, we weren't talking about general change.
>  > I think we were, back a ways in this thread.
> Again, you were, but that's hijacking the thread.
> 
> No, we were talking about disruptive innovation....

"Riiiight." Thus the title, "work times & accommodation."

>  > > This means that not only do you have to make your own people happy
>  > > with the change, you also need to get your clients and suppliers and
>  > > complementers on board.
>  > 
>  > Yup. Which is exactly why you should be starting off on this kind of
>  > thing early, rather than doing nothing about it until you've fallen
>  > behind the rest of the industry.
> 
> Everybody knows that.  Everybody also knows that jumping on the
> current trendy technology is likely to cause a catastrophic loss.

Indeed. So let me say explicitly now:

    Do Not Jump On The Current Trendy Technology.

Keep in mind, from this point on, I have never said that, and never
will say that. (Not to mention that the current "trendy" technology is
usually re-hashed twenty year old stuff that the world has a lot of
experience with. Java being the classic example, chunks of it having
re-iterated the twenty-five years of Smalltalk previous to it.)

My point is, keep trying to improve. Investigate the new technologies,
rather than blocking them. Instead of saying, "No, you may not do
anything in Haskell," find out why you have smart folks telling you
why they want to use it and, if they present some sort of reasonably
convincing argument, start looking at how you might integrate it into
your systems.

Most of this stuff isn't even new anyway, and has never been trendy.
Jane Street has managed to become a very successful company in a
relatively short period of time partly because they did try this, with
OCaml. You can ask them about it, if you don't believe me.

> So what you're saying is that you've identified the "silver bullet
> that works," and Google is being stupid to not adopt it. Reread your
> Brooks recently?

No, you have completely misinterpreted me. I firmly believe that there
is no silver bullet. But that does not mean you can't improve your
technologies. Would you recommend to a company programming in assembler
that they not even look at using C?

> It's possible to write unreadable Python, but I haven't seen much.

Yes, and it's even possible to do the kind of combinators in Python that
you can do in Haskell. They're just not as nice, don't play well with
the standard libraries, aren't type-checked, and are much slower.

> On the other hand, have you read Darcs's code?

Yup. Not good. I agreee that was probably more the person than the application.

And ghc is not exactly easy to read, either, and has some relatively
nasty stuff in it. But for a compiler that's in most ways as
sophisticated or moreso than ghc, the code is a heck of a lot cleaner.
The nastiness may well be part of the nature of production-quality
compilers that run on a lot of different platforms, but the gcc vs. ghc
comparision does show that at least some improvement is possible.

> Where is the Kernighan and Plauger, or even the Kernighan and Pike,
> of Haskell? Try going to Amazon and typing "programming style
> C/C++/Pascal/Python/Smalltalk/ Lisp" (yes, even Lisp!) and then try
> "programming style Haskell". The second group is 90% conference
> proceedings!

Yup. There would be two reasons for this:

    1. Haskell is a much less widely used langauge, and in real-world
    programming situations tends to be used by better programmers who
    already know the importance of "style." Thus, books that discuss
    when you should comment, the correct naming of variables, and
    suchlike have very little (no?) demand.

    2. Haskell doesn't have a lot of the style issues that, say,
    Java would have, due to the style of the language. If you can do
    something in a third or a quarter the amount of code, you need pay
    less attention to code organization. If your parser looks pretty
    much like a BNF description of the language, the idea of what sort
    of "coding style" you should be using for it becomes silly: you're
    using BNF style now.

I'd not actually thought about this much before, even though anybody who
knows me will tell you that I'm pretty heavily concered with programming
style. Perhaps it's something to do with this: a more compact language
and the ability to introduce embedded DSLs tends to produce communities
to whom "where to put the braces" books are irrelevant. I think that
Java and Ruby make a good comparison in this; even without IDEs, people
don't write Ruby style books. (I'd imagine that Python is in the same
situation.) I can think of, right off-hand, at least one full book
devoted to nothing but Java style; I'm sure that there are more.

> Don't get me wrong: Haskell is everything you say it is technically,
> and more, as far as I can tell in my limited experience, especially
> with Haskell.  My point is that propagating those advantages in a
> large enterprise is not going to be as easy as you think.

Right. And books and HOWTOs are definitely not going to be enough to
do it, either. You need internal expertise, which takes time to build
up. Thus: start now, especially if you're a company for whom IT is a
critical part of what makes you competitive.

> I do not think that failure to adopt Haskell "now" can kill Google.

Not the failure to adopt Haskell specifically, no. But the failure to
developing internal expertise in better languages in general, and how
they can be applied to Google's business, will unquestionably hurt them.

> Why in the world would you think Microsoft would have anything less
> than the most technically advanced labs in the world?  Cf. AT&T Bell
> Laboratories.  The question is does it show up in the practices of the
> programmers who use their systems?

It appears to be having some effect, if F# is any indication.

> And coming out of Google?  Surely you jest; if there is any really
> high-tech stuff happening at Google, the last thing they want is for
> it to get out, no?

Why not? As someone I can't recall once said, "If your idea is any good,
you won't have to worry about anybody stealing it. You'll have to shove
it down their throats." As far as I can tell, the advantages you get
from having a lot of people look at your idea, comment on it, and offer
improvements outweigh the disadvantages of having others in the world
know details about what you're working on.

> The problem is the massive infrastructure of libraries that languages
> like Java, Python, and C++ bring with them,

Actually, that whole library argument is what the Java folks use, I used
to think it was a good thing myself, and since I've become a better
programmer I've come to the conclusion that it's a load of rubbish.

Libraries are a trade-off you make: you use the library because you
believe that

    * learning the library,
    * dealing with a interface that's hard to change and perhaps
      unsuited to your application,
    * finding and working around bugs in the library, and
    * dealing with the things you need that the library doesn't do

are going to be cheaper than writing just the code you need. For
languages where it's fairly expensive to write code (Java), this is more
often true than for languages where coding is cheap (say, Ruby).

But you yourself have seen a situation recently where starting off with
a library for parsing mail headers seemed like a good idea, and this is
a library that's less than a thousand lines of code and comments.

> ...and the well-known idioms for doing things in those languages.

Right. That's something that has to be learned. Again: start now, and
figure out what works and develop external expertise. Don't just ignore
the issue and say you're going to stick with Java for the rest of your
company's existence.

> But it's typically not true; every programming language community
> seems to reinvent things like web frameworks multiple times.

And for good reason. Continuation-based web frameworks work very
differently from REST-based ones, and are suited for different kinds
of applications. Starling built its own web framework for very good
reasons: ours probably works better for internationalized keitai
applications than anything else out there. But I can think of plenty of
situations where I'd even recommend Rails to someone, much as I despise
it myself.

cjs
-- 
Curt Sampson       <cjs@example.com>        +81 90 7737 2974   
Mobile sites and software consulting: http://www.starling-software.com