Re: [tlug] TLUG Site with Hakyll Update

[Curt Sampson <cjs@example.com>]

On 2019-03-18 16:18 +0900 (Mon), Stephen J. Turnbull wrote:

> Curt Sampson writes:
>  > I'd love to hear a talk about this, formal or just over a beer.
>
> Probably over a beer.  setup.py is like debhelper or rpm scripts.
> They're all whole DSLs, right?  Not much more to say than that.

I think there's a lot more to say for people who have never had to
package things with tools like that, and using setup.py as an example
would also let people who have learn about the quirks of that
particular system and how it compares. The only people who would find
it boring would be people who already know setup.py well and probably
know about another system or two, and I'm thinking there's only one of
those in TLUG. So presentation, please!

On to the CI thing. First thing to note is that my long quote from the
_Extreme Programming Explained_ 2nd ed. is just an expanded version of
what was in the first edition, with no substantive changes. (O'Reilly
kindly kept the first edition in Safari, even though they also have
the second.)

> Beck (or someone of that generation) may be the source of the name,
> but the description looks like an updated version of Fred Brooks's
> discussion of continuous integration (which he recommends against,
> subject to your caveat that it's not *real* CI, for the simple reason
> that a system build took about a week in OS/360. ;-)

I don't recall him ever recommending against it; do you have a
reference? A quick look at the 20th anniversary edition of _The
Mythical Man-Month_ produces a couple of references in Chapter 19,
"The _Mythical Man-Month_ after 20 Years" that seem to say the
opposite:

] AN INCREMENTAL-BUILD MODEL IS BETTER—PROGRESSIVE REFINEMENT
]
] Building an end-to-end skeleton system
]
] Harlan Mills, working in a real-time system environment, early
] advocated that we should build the basic polling loop of a realtime
] system, with subroutine calls (stubs) for all the functions... but
] only null subroutines. Compile it; test it. It goes round and round,
] doing literally nothing, but doing it correctly.
]
] Next, we flesh out a (perhaps primitive) input module and an output
] module. Voilá! A running system that does something, however dull.
] Now, function by function, we incrementally build and add modules.
] At every stage we have a running system. If we are diligent, we have
] at every stage a debugged, tested system. (As the system grows, so
] does the burden of regression-testing each new module against all
] the previous test cases.)
]
] Since we have a working system at all times
] • we can begin user testing very early, and
] • we can adopt a build-to-budget strategy that protects absolutely
]   against schedule or budget overruns (at the cost of possible
]   functional shortfall).
]
] For some 22 years, I taught the software engineering laboratory at
] the University of North Carolina, sometimes jointly with David
] Parnas. In this course, teams of usually four students built in one
] semester some real software application system. About halfway
] through those years, I switched to teaching incremental development.
] I was stunned by the electrifying effect on team morale of that
] first picture on the screen, that first running system.

And under the next section, "Microsoft's "Build Every Night"
Approach":

] Software builders in other organizations are surprised, even
] shocked, by this process. One says, "I've made it a practice to
] build every week, but I think it would be too much work to build
] every night." And that may be true. Bell Northern Research, for
] example, rebuilds its 12-million-line system every week.

No recommendation against, as I read it: just an admission that with
really large systems you may not have the resources to build it
overnight.

> Conceded.  I would say that Brooks probably did not consider the
> possibility that week-long builds were an "accidental" difficulty.

Again, my reading differs. From TMMM20th Chapter 16, "No Silver
Bullet—Essence and Accident in Software Engineering":

] Most of the big last gains in software productivity have come from
] removing artificial barriers that have made the accidental tasks
] inordinately hard, such as severe hardware constraints, awkward
] programming languages, LACK OF MACHINE TIME. [Emphasis mine. --cjs]

>  > I'd imagine that a proper integration test would involve booting up a
>  > Lisp machine from scratch with the current code. That clearly would be
>  > an overnight thing. :-)
>
> That's entirely a joke about general Lisp machine performance, right?

Sadly, not entirely.

> If not, it's a misrepresentation of the "no builds" concept behind the
> Lisp machines (load an image of the Lisp data, part of which is a Lisp
> program, into memory and call the program).

And when you can't load it up on to another machine? Well, I guess you
just splice a UPS into the power cord of your developer's machine, and
ship that to the customer!

> No better than grad students doing research and expecting to write a
> dissertation in a week, eh?  But devops goes farther than continuous
> integration: it's continuous deployment.

Going back to Kent Beck's quote in my previous message, I read him as
basically talking about continuous deployment as well:

] Integrate and build a complete product. If the goal is to burn a CD,
] burn a CD. If the goal is to deploy a web site, deploy a web site,
] even if it is to a test environment. Continuous integration should
] be complete enough that the eventual first deployment of the system
] is no big deal.

I think you can safely say he wasn't thinking that you wouldn't try to
run the software on the CD, or visit the web site and poke a few
things in to see that it was working.

> I certainly agree that we want our integration tests to go all the way
> to deployment, though.  AIUI, the devops admin specialty is all about
> developing the personnel to support that discipline in the in-house
> "continuous deployment" scenario....

Yup. I have another slant on that, which is that DevOps is often about
turning the two separate teams, developers and operations, into one
single team that does the whole job, to avoid the "throw it over the
wall and let them deal with the problems" thing.

>From <https://theagileadmin.com/what-is-devops/>:

] DevOps is a new term emerging from the collision of two major
] related trends. The first was also called “agile infrastructure” or
] “agile operations”; it sprang from applying Agile and Lean
] approaches to operations work.  The second is a much expanded
] understanding of the value of collaboration between development and
] operations staff throughout all stages of the development lifecycle
] when creating and operating a service, and how important operations
] has become in our increasingly service-oriented world.

He later breaks this into a definition:

] DevOps is the practice of operations and development engineers
] participating together in the entire service lifecycle, from design
] through the development process to production support.

and a corollary:

] DevOps is also characterized by operations staff making use many of
] the same techniques as developers for their systems work.

> I thought the motto of XP was "move fast and break things"?

Well, really more like "expect things to break, and ensure you can fix
them darn quickly," IMHO.

> More seriously (and a rather different topic), how many large
> organizations really implement XP or any of these other famous
> disciplines?  And does it matter which one you use, given that
> you've got a team that's disciplined enough to stick with any of
> them for more than a quarter?

It doesn't matter much if you're actually doing "Agile," because one
of the four values of Agile is "Individuals and interactions over
processes and tools." (<https://agilemanifesto.org/>)

A truly agile project isn't doing exactly XP, or exactly Scrum, or
exactly anything else: they've started with something like that and
changed it as necessary for their situation.

And so, yes, it's really the _discipline_ that counts here, and
developing that is by far the hardest thing. But having a good,
well-tested place to start helps a lot with getting there.

cjs
-- 
Curt J. Sampson      <cjs@example.com>      +81 90 7737 2974

To iterate is human, to recurse divine.
    - L Peter Deutsch