Water -- Waterlanguage.org

Language for Simplified Web Services and XML Programming

Articles

Water's Fine for Extreme Programming

Extreme Programming (XP) as explained in Don Wells' site (www.extremeprogramming.org/) and several books is a light-weight methodology for software development. It is people-based, a collaborative management process for coherent coding. As such, XP is language independent and agnostic.

The Water programming language and Steam IDE (integrated development environment) are ideally suited to XP's goals and provide built-in support for particular XP strategies, such as refactoring, pair programming, testing before coding, spike solutions, and near-continuous integration.

Let's focus upon Water's support for those strategies, all the while understanding that adoption of particular strategies and tools doesn't guarantee quality software any more than handing a crescent wrench to a 10-year-old ensures competent auto repair. For information on integrating XP into your organization's development process, consult works by Beck, Fowler, Jeffries, Hendrickson, Anderson, Succi, Marchesi, Martin, Newkirk, Wake, Auer and Miller.

To run the following Water examples yourself, visit www.waterlanguage.org and download the no-charge version of Clear Methods' Steam deployment platform. Steam includes an IDE (integrated development environment) that allows quick construction and evaluation of Water code.

Strategy: Unit and Acceptance Testing

XP places a premium upon rapid turns on small improvements in program function. Another cardinal principle is refactoring, or making internal coding improvements that do not affect externally observed software function. This differs radically from approaching a software project as a linear engineering exercise in which all requirements, data structures, partitionings, etc. must be documented before writing a single line of code: It is naïve to expect full understanding of these considerations at project inception.

But in that case, how does an XP team identify and remedy bit rot, the well known tendency to break an application as a side effect of extending its functionality? XP's answer is testing, testing, testing, early and often, so that molehills have no opportunity to grow into mountainous surprises.

Water's testing facility eliminates the need to build or borrow a test harness. For example, consider a water code snippet to be tested:

10.<times 15/>

This expression, which applies the 'times' method to an instance '10' of the 'integer' class with the argument '15', should evaluate to '150'.

Testing is rather popular at Clear Methods. (There are more than 2200 tests in Water's own test suite, for instance.) So the Steam IDE provides a test-creation shortcut: Select (highlight) the code under scrutiny and right-click to 'Misc Inserts | <test ... />'. This produces a test case, including the expected baseline result, and runs it:

Subsequent execution of the 'test' statement shows that the test passes, as indicated in the IDE's HTML window.

Beyond this simple case, Water provides:

· Testing with a customizable definition of equality between execution result and baseline (to tolerate near-match strings, for instance)

· A means of including setup code within the test body

· A way to dynamically test for an error in executing a test

· Association of a test with an object (a class definition, for example)

· Timing of test duration

· Embedded runtime testing via assertions

· Storing, running, and reporting upon a suite of tests

Water facilitates testing at levels important to both the programmer and the customer. Testing at a fixed level of aggregation (Java's compilation unit, for example) is not necessarily appropriate. Tool-imposed insistence on testing at some particular level may result in distortion of the application's partitioning, as programmers unnaturally break down or combine units in order to meet the testability goal. Better to test at whatever level is meaningful, as Water allows.

There is very little excuse for programmers to neglect testing Water applications, no excuse at all for XP practitioners.

Strategy: Remove Obstacles from Story Implementation

XP captures requirements via customer-supplied stories. A story represents a quantum of accountable development. It's a few lines of non-technical English describing part of the business problem to be solved. Just before implementing a story, programmers meet with the customer to secure detailed requirements. A story is too long if it cannot be used for a reliable implementation time estimate or to define a customer acceptance test. A collection of stories thus serves as a flexible, high-level specification and planning tool for the application.

XP places a premium upon implementation of the current, here-and-now story. Implementation beyond the current story is strongly discouraged because making that up-front “investment” would delay current progress and might be rendered obsolete by unanticipated twists and turns in the development process. This is perhaps XP's most counter-intuitive aspect, a near-insult to programmers who consider themselves experienced and thoughtful. But it is a centerpiece of XP philosophy.

Now why do experienced, thoughtful people want to look ahead? Perhaps they worry about coping with mission creep. Perhaps they've been burned before by some inflexibility in their tools and hope to avoid that this time. Perhaps they desire to produce more maintainable code. Relieving these worries might go a long way toward allowing programmers to relax, focus and work in the moment. In XP terms, this desirable state of mind is called courage.

XP's continuous, fine-grain communication process handles mission creep quite well. So no worries in that regard. Could programming language features mitigate the other worries?

Water's designers thought so. They provided:

• Flexible method definitions and calls --- Water lets programmers freely mix positional and keyed argument lists. For example, a method defined like this:

<body_statements/>

</defmethod>

might be called like this:

like this:

or like this:

It doesn't matter whether an argument may be passed by position or key so long as it's unambiguously identifiable.

Water's 'rest' parameter allows a method to accommodate a variable number of arguments. For example, Water's built-in 'times' method accepts a variable number of arguments. This expression:

10.<times 15 20 2 1/>

evaluates to 6000, the product of the call's subject, 10, and the other arguments that happen to have been supplied.

Water method parameters may be declared 'optional' or 'required' as appropriate. Thus methods calls can be as simple as naming the method and accepting its default behavior or detailed enough to micro-manage their behavior.

Upon reading this, may hearts leap up among XP practitioners familiar with Java and several other object-oriented languages. No one proudly delivers methods whose argument lists, evolved over several cyles of story implementation & refactoring, are ordered in an ugly, ad hoc, non-intuitive way. No one likes writing seven entry points into the same method that differ in their parameter lists or require construction and passing of elaborate data structures. Yet (in Java) that's how it may work out unless complementary organizational and aesthetic (but non-functional) changes are applied throughout the application in the wake of a refactoring.

One might wait until the last moment before the final release, and then apply these non-functional changes just once. That would be reasonably economical. But according to XP doctrine, one never knows which release will be the final one because the customer controls final acceptance. Non-functional changes might be applied again and again as a result, a squandering of time and talent and an opportunity to break the application.

An experienced, thoughtful programmer attempts to avoid the problem by overreaching, by looking beyond the current story to define interfaces in well intentioned attempt to “get ahead of the power curve.” And he gets away with it from time to time. But XP posits that any project big enough to be worth driving with XP discipline takes unpredictable turns. So Mr. Experienced and Thoughful may become frustrated when his careful planning washes down the drain. Worse, he may resist refactoring an obsolete design in which he has “invested” so much time.

Better to relieve the stress at the source, as Water does.

• User-controlled, extensible typing --- Every field or variable can have an associated type. A type is a description that characterizes possible values. As such, a type may be primitive (e.g., 'number or 'boolean') or possibly quite elaborate and particular, such as:

type.<vector_of <one_of “red” “green” “blue” /> max=5/>

type.<range_of min=0.2 max=11.3367/>

Water types can fully comprehend data semantics. This lets the programmer define an object once, then use it fearlessly everywhere. “Fearless” is illustrated by “fearful” counterexample. Fearful programmers scatter argument-checking code throughout the application “just to make sure” that some object's value lies inside the range that some finicky methods require. What a mess to maintain.

On the other hand, making a strictly typed application work can consume a lot of time, as subsequent story implementations and refactorings force typing work to be done, undone, redone, and so on. So typing is optional in Water. Its stricture can be utterly relaxed to facilitate rapid story implementation, yet cranked up selectively as the team approaches (mythical) final customer acceptance.

Strategy: Pair Programming

Two programming heads are better than one according to XP adherents: One works the keyboard and thinks tactically about the method under creation. The other, looking over his colleague's shoulder, thinks more strategically about the methods' places in their classes. The idea is to create complementary views of story implementation. The purported benefit, better than two-to-one progress rate with superior code quality, earns pair programming its methodological rule status.

In terms of a development environment, how best to support both heads? Batch-mode environments (those oriented to editors, compilation units, makefiles, ifdefs, and machinery of that ilk) serve the tactically minded keyboardist well: Think locally, make one change at a time, push the “build” button, and debug. But they are useless to the strategic side of the partnership. Indeed, they can be maddening, since the keyboardist may flip from one compilation unit to another more quickly than the strategic partner can follow. So the strategic thinker might instead refer to a higher-level presentation on another computer (and in so doing break the pair-programming rule), chase parameter list documentation for the keyboardist's benefit (and so not really think strategically), or just day dream. Batch-mode tools require focus and discipline that the strategic thinker may find difficult to maintain.

The Steam IDE approaches the problem more simply: There is no compilation unit. The keyboardist can highlight a code fragment for immediate execution or run the entire application at the push of a button. The IDE's object Inspector lets the user drill down into an object's definition or upward through its lineage. The Inspector also provides access to documentation of built-in Water methods.

Meanwhile, the IDE lets the strategic thinker see a large swath of code, perhaps several entire class definitions.

This is not to say that a Water method can be defined only within the brackets of its class definition. A method might be defined elsewhere in the same file or in another file altogether. So long as the class definition has been processed, the class object exists and so subsequent method definitions can refer to it. This allows developers to partition development along lines of responsibility rather than along class lines.

Of course this level of freedom begs for responsible use. It's entirely possible to organize a Water project so as to make it as confusing to follow as a batch-mode alternative. But XP's penchant for simplicity, directness and coding standards naturally dampens this sort of abuse.

Strategy: Recording CRC Cards and Stories

XP's CRC (Class, Responsibilities and Collaboration) cards focus the team's design thinking on object orientation. The team develops its design at strategic level by drawing up only as many cards as necessary, each of them representative of a class, moving them around in order to explore relationships, and filling them with detail as appropriate.

This exercise in dynamic design is well served with good old pencil and paper. However, Water provides a facility that can be used to capture CRC cards and relate them to code at low cost.

For example, the following code associates a 'doc' object with a class named 'philemon':

philemon.<doc>
    Jack to define types and defaults;
     Don to define 'vino' and 'veritas' methods;
    Jill to define 'pail' interface to collaborating 'hill' class
</doc>

The 'doc' object content might be simple text or arbitrarily complex and descriptive HTML, for example a table laid out just the way the team prefers.

A 'doc' object can be applied to any Water object, not just the class. This allows association of stories with methods.

But why the formalism? Why not simply include comments in the code? There are two reasons. First, the 'doc' object can be used to separate information about implementation from directions regarding the program's use. The audiences differ: programmers and customers. Second, comments have a way of becoming buried in code and forgotten. To prevent this, gather 'doc' objects together for presentation. For example, the following code creates a vector of objects and displays the documentation for all of them:

thing.<set presentation=
<vector
    number.integer
    number.plus
    uri
/>
/>
presentation.<get_doc/>

Water's 'doc' definitions can follow the code around or, like 'test' definitions, can be consolidated in files. Tradeoffs between documentation relevance, ease of access, and crowding out the strategic-level view of the code are matters of the team's style and the customer's requirements.

Strategy: Spike Solutions

In XP's world, a build takes too long if it allows the programming pair to lose focus and productivity. So don't build. Use an incremental, interactive language, such as Water.

This not only eliminates distraction but also enables easy construction of “spike” solutions, simple standalone programs used to explore potential approaches to thorny problems and so control technical risk. Regardless of spike solutions' value, programmers may be loath to invest in them because they're eventually discarded. Well intentioned programmers may instead experiment on the mainline application code, sometimes with disastrous results. It's important to head this off by lowering the barrier to creation of spike solutions.

Strategy: Near-Continuous Integration

A monolithic application written in a dynamic language such as Water is implicitly integrated every time it's run. It then becomes a matter of policy to wrap around it synchronization of documentation, testing, customer buyoff, etc. XP methodology says a lot about controlling these issues effectively.

If monolithic applications weren't tough enough, the complexity of today's distributed applications increases each integration cycle's cost. This is bad news, for higher cost discourages high-frequency integration for the very applications that would most benefit.

Water helps in two ways. First, Water's built-in web services and client-server capabilities abstract and remove considerable complexity, thus allowing business logic to better shine through the application. Second, Water makes it easy to create and host all application tiers on one computer, across a LAN, or over a wide geography. Integration cycles can thus be designed to trade off between local cost-effectiveness and end-to-end relevance: One wide-geography integration cycle for every five LAN cycles for every 25 local-computer cycles may be reasonable, for example.

Strategy: Deliver Functionality First, Optimize Last

This XP strategy makes sense because only the customer can say whether an application works properly with acceptable performance. There is no point investing in optimizing a design that may be ripped up and replaced later. And it's wasteful to devote energy to portions of the application that already run quickly enough.

Nonetheless, it's good to have a few performance techniques on tap. The better techniques require only localized code changes, minimal refactoring, and little change in documentation.

Water provides a powerful one: the ability to cache both data and code among various tiers of a distributed application. This reduces network latency and can reduce datacomm costs, too. In most business applications, more than 95% of the data and logic can be cached for some period of time -- whether 10 seconds, 10 minutes, or 10 hours. The ability to asynchronously update code and data can improve performance by orders of magnitude.

As a nice side-effect, building all application tiers upon a common language and runtime system makes it relatively easy to move (i.e. refactor) the work in a distributed application so as to avoid bottlenecks and improve scalability.

Strategy: Cross-Train the Development Team

XP favors purposely rotating team members through various assignments in order to cross-train them. Developers of conventional, monolithic applications may grouse about doing certain chores. (Let's face it: No one enjoys taking out the garbage.) However, team members generally understand what's required in one area or another of the project, and they're experienced with a common tool set. With a little goodwill and peer assistance from time to time, programmers proficient in one area can contribute in another without too much trouble. It's XP doctrine that the gain in redundant capability, tolerance of staff turnover, and improved design insight is well worth cross-training's pain.

But web services applications upset the economics of this trade. A single application may involve five or six programming languages, some with their own development environments. The “central” programming language (e.g., Java) may have scores of APIs (application programming interfaces) specialized for various distributed computing standards. Add one or more documentation languages to the mix. How can each XP team member be expected to master so much material?

Well, perhaps they cannot be expected to do that. Large-scale system building companies back up their hundred-programmer groups with experts specialized in particular pieces of the technical puzzle. The experts circulate among the developers, dispensing assistance and implementing the tougher nuts. This “islands of knowledge” approach suits the major players well. In fact, its expense erects a profitable barrier to entry.

According to Kent Beck (Extreme Programming Explained: Embrace Change, Addison-Wesley, 1999), XP is designed for teams of two to 10 who work with a “mentality of sufficiency.” Indeed, Beck refers to the big team as an XP “show stopper.”

So, is there a place for XP in distributed application development? Yes. Would I raise the question if Water & Steam weren't part of the solution? Of course not. It's a matter of cutting the problem down to a size that an XP team can digest.

Water & Steam radically reduce complexity in development of distributed applications. The Steam platform allows developers to write in a single language, Water, for deployment at every tier: It's Water on the database back end, Water on the mid-tier business logic servers, Water on the user's desktop, Water's syntax to define data, logic and presentation. Water replaces multiple quirky APIs with a few well considered methods. It's Clear Methods' job to track interoperability standards and abstract and subsume their functions so that programmers need not master them.

This allows developers to learn Water once, then use it everywhere. The enormous distributed application development problem then collapses to the size of a monolithic application of similar scope with some additional test and documentation costs.

Clear Methods submits that this technology puts web services within the grasp of XP and so within the economic reach of small and mid-sized development teams. Cross-training becomes practical once again.

A simple example may clarify. But caution: It seems simple to the point of triviality, so consider how many computing technologies must be mastered to accomplish the same thing by conventional means.

<factorial 3/>


<server factorial/>


<open_browser_window "http://localhost/?n=7"/>


<web "http://localhost/?n=3"/>.result


<set remote_factorial = <web "http://localhost/"/> />
<remote_factorial n=3/>

Conclusion: Water is Ideal for Extreme Programming

The wide acceptance of other languages lacking Water's features demonstrates that they're able to support development in the real world. Programmers cope. Many got into the business because they enjoy solving problems. But there are the problems whose solutions make money, and there are the problems of coping with tools. In the long run, programmers get paid for solving one customer's problem and moving on to the next as quickly as possible. Water and Steam can help. And they may provide the only path to extending Extreme Programming's benefits to creation of distributed applications.