Functional reactive programming for the web, or: where’s my Lunascript?!

Update: there’s some good discussion on Reddit.

Even when using the latest and greatest frameworks and disciplines, writing fast, highly interacting web applications involves a lot of accidental complexity:

First you need server code to figure out what data the browser needs. Hopefully you have an ORM layer, but you still need to carefully structure your code to minimize your backend dispatches, and you need to carefully keep that in sync with your front-end code lest you don’t fetch enough data or hurt performance by fetching too much. If it’s a Web 2.0-style app, you re-implement a ton of that server-side code in JavaScript, once for creating the page and then again as controller code for keeping it up to date and consistent. And when the user changes something, you bottle that up — typically in a custom over-the-wire format — and send it as an XHR to the server. The server has to de-serialize it into different data structures in a different language, notify the persistence store, figure out what other clients care about the change, and send them a notification over your Comet pipe, which is handled by yet more JavaScript controller code. Offline support? More code.

Strike a chord? That’s the motivation setting the stage for Lunascript, a reactive programming framework for web apps. What?

Reactive programming is a style of programming where you can express state in terms of other state (data flows), and the runtime dynamically keeps everything updated and consistent whenever any state changes. This type of event propagation can be found in plenty of places, from formulas in Excel to simple data binding to signals-and-slots to the observer pattern. The most popular application area is in UI development—we have things like JavaFX and OpenLaszlo, plus tons of academic research projects like Flapjax, Coherence, a dozen Haskell libraries…the list goes on (yet the community is niche).

Anyway, back to Lunascript. It’s a framework from Asana, and it caught my eye last year when Asana posted about it on their blog (unreleased – possibly for recruiting?):

Lunascript has a syntax and ease of use reminiscent of JavaScript, but a powerful pure-functional lazily-evaluated semantics historically confined to academic languages.

A Lunascript application specifes a data model and a function from the model to the view or user interface, annotated with handler functions from user inputs to model mutations. From this the Lunascript compiler produces a functioning Web 2.0 application — the client-side JavaScript, the server-side SQL, and everything in between — complete with real-time bidirectional data synchronization. There’s no need to write separate code to help the server figure out which values need to be sent to the client…Because a Lunascript application only specifies how the UI should look given the current data (rather than how the UI should be updated as changes happen) it’s impossible to write a UI that loads correctly but does not stay correct as changes are made.

I guess the Asana guys got over the optimism phase, since they’ve scaled back Lunascript to a more modest Javascript-based framework. The vision remains tantalizing, though. At the time I had already heard of several other similar-sounding projects, and since then at least one more has appeared, so I recently took a few hours to dive deeper into them, just to see how they compared and—yes—how close they were to being usable (my own optimism phase). The main feature I’m interested in is the reactive programming/incremental evaluation/whatever you want to call it, but one which straddles the browser, app server, and database, generating code for each of these platforms from a single high-level program.

The contenders

The four most similar projects I know of, along with self-descriptions:

Links: from Philip Wadler’s group

Links eases the impedance mismatch problem by providing a single language for all three tiers. The system generates code for each tier; for instance, translating some code into Javascript for the browser, some into a bytecode for the server, and some into SQL for the database.
Swift:

In the Swift approach, application code is written in the Jif language, a Java-based language that includes information security policies. The source code is automatically partitioned into JavaScript code running in the browser, and Java code running on the server. For interactive performance, code and data are placed on the client side where possible. Security-critical code is placed on the server and user interface code is placed on the client. Code placement is constrained by high-level, declarative information flow policies that strongly enforce the confidentiality and integrity of server-side information. The Swift compiler may also choose to replicate code and data across the client and server, with benefits for both security and performance.
Ur/Web

Most client-side JavaScript programs modify page contents imperatively, but Ur/Web is based on functional-reactive programming instead. Programs allocate data sources and then describe the page as a pure function of those data sources. When the sources change, the page changes automatically.
Fun, which is directly inspired by Lunascript. Learned about this from this Quora question. From the (somewhat old) introduction:

What if you could build realtime web apps with the same ease as you build static web pages in PHP today? Without long polling, event handling and state synchronization, the engineering complexity of realtime web applications would drop by an order of magnitude. There would be a fundamental shift in the way we build the realtime web. This is the future of Fun.

…

First, credit where it’s due! This project is conceptually inspired from conversations with Justin Rosenstein about the language LunaScript that he’s building together with Dustin Moskovitz & team at Asana, and from conversations with Misko Hevery about the HTML compiler that he’s building over at Google.

My take-aways

Fun

Fun is the language that’s closest in spirit to Lunascript – probably not surprising, given the direct inspiration. Key to this is how the reactive expressions transcend both the client-server and server-database divides, allowing you to create UI elements expressed directly in terms of persistent database state, and have them dynamically updated as the database state changes. From the chat example below, notice how the list of messages generated by the loop is dynamically updated as messages are appended to the global set of messages—from any user (via the send button click handler). Making this easy makes sense for Asana since they’re building an app with a lot of collaborative interaction.

<div class="chat">
  <input data=userInput class="messageInput"/>
  <button>"Send"</button onClick=handler() {
    let newMessage = new { text: userInput }
    userInput.set("")
    global.messages.unshift(newMessage)
  }>
  <div class="messages">
    for (message in global.messages) {
      <div class="message"> message.text </div>
    }
  </div>
</div>

That’s pretty much a “complete” AJAX-y chat app! (Modulo some declaration lines.)

However, the project is early and incomplete, with the language in flux. And I still have questions regarding how easy it is to control explicitly things like what’s dynamically updated vs. static, what runs on the server, how much communication occurs, etc. (Asana says " And, of course, how this approach pans out in practice on real projects. The author seems to be hacking on this in pockets of spare time.

Ur/Web

I’m impressed with Ur (an ML-style programming language) and Ur/Web (the web framework for Ur). Ur has a type system featuring dependent types but does more traditional type-checking, avoiding asking the user for proof assistance. Instead, the dependent types provide a framework for metaprogramming that allows you to, say, write a CRUD app that generates views generically for any compound data type (database schema). However, while Ur/Web features client-side reactive programming, client-server/AJAX communication is explicit. For instance, in the incrementing counter example, the RPC call that updates the counter is also needed to get the updated counter value (though once the client-side src variable has been set, the view is dynamically updated):

sequence seq

fun increment () = nextval seq

fun main () =
    src <- source 0;
    return <xml><body>
      <dyn signal={n <- signal src; return <xml>{[n]}</xml>}/>
      <button value="Update" onclick={n <- rpc (increment ()); set src n}/>
    </body></xml>

Notice how this only updates the counter when you press the button, and not when others update the counter—a type of reaction shown off by the Fun chat app.

The author believes Ur/Web is ready for production use and offers consulting/support services. There are a few Ur/web users in the wild, though seemingly countable by hand.

Links

Links is a project from Philip Wadler’s group. I was surprised to find that it doesn’t offer functional reactive programming of any kind, and things like DOM manipulation are imperative. For client-server communication, again we have explicit RPCs. I think of Links as somewhat similar to GWT in that it’s just a single language (functional in flavor) that can generate server and client code, in particular shielding you from JavaScript. As an example of this shielding, Links offers client-side concurrent processes with explicit message-passing communication (specifically, actors). These come in handy for writing blocking code (e.g. RPCs) without causing the entire client app to block.

The following code is from their updatable autocompletion demo (source), which is an app that lists DB-backed completions for the word you’re typing and can insert new completions. Starting with the UI, we have a text box, the completion list, and a form for new definitions:

<form l:onkeyup="{handler!Suggest(s)}">
 Search: <input type="text" l:name="s" autocomplete="off"/>
</form>

<div id="suggestions"/>

<h3>New definition</h3>
<div id="add">
 {addForm(handler)}
</div>

handler is an actor running client-side that makes blocking RPCs on behalf of the UI:

var handler = spawn {
  fun receiver(s) {
    receive {
      case Suggest(s) -> suggest(s); receiver(s)
      case NewDef(def) ->
        newDef(def);
        replaceChildren(addForm(self()), getNodeById("add"));
        if (s <> "") suggest(s) else (); receiver(s)
    }
  }
  receiver("")
};

So handler!Suggest(s) sends a Suggest message to handler. Notice how the NewDef handler needs to remember to update the completion list as well (by calling suggest, just like the Suggest handler) to keep the completions in sync with what’s in the database.

The suggest function asks the server to query the database, then imperatively updates the DOM:

fun suggest(s) client {
  replaceChildren(format(completions(s)), getNodeById("suggestions"))
} 

fun completions(s) server {
  if (s == "") []
  else {
    query [10] {
      for (def <-- defsTable)
       where (def.word =~ /^{s}.*/)
       orderby (def.word)
        [def]
    }
  }
}

Notice how the functions are annotated with where they should be running.

This isn’t under active development.

Swift

Swift bears a lot of resemblance to GWT, but with automatic code partitioning. The idea is that you can “anchor” certain data to the server side (the motivation is for security, and the paper formulates everything as an information flow control problem), and then the partitioner constructs a control-flow graph out of the code and uses min-cut driven by a ton of static heuristics about the edge costs—cute, but sounds scarier than wrestling with database query optimizers. It isn’t really focused on the language/expressiveness, so programming in it is apparently just writing GWT where the system decides for each line of code whether it’s running on the client or the server. This isn’t under active development.

Wrap-up

So it looks like although neither you nor Asana are going to be using Lunascript any time soon, you have a couple other options if you’re feeling adventurous. Of the projects covered here, Fun is the closest to Lunascript and is ripe for big-impact contribution and shaping, though Ur can get you pretty close while being the most mature. If there are any other similar projects out there that I should be aware of, let me know.

Follow me on Twitter for stuff far more interesting than what I blog.