Building interactive web pages with Guile Hoot
Dave Thompson —
A question we frequently hear in discussions about WebAssembly (Wasm) is:
"Can Wasm call the DOM (Document Object Model) API?"
The answer is: Yes, thanks to Wasm Garbage Collection!
In this post, we will use Guile Hoot (our Scheme to Wasm compiler) to demonstrate how a language that compiles to Wasm GC can be used to implement the kind of interactivity we're used to implementing with JavaScript. We'll start very simple and build up to something that resembles a React-like application.
In our previous post about running Scheme in the browser, we had to use quite a lot of JavaScript code to call Scheme procedures (functions), render the user interface, and handle user input. However, today we're pleased to announce that those days are behind us; Hoot 0.2.0, released today, now includes a foreign function interface (FFI) for calling JavaScript from Scheme. In other words, the vast majority of code in a Hoot application can now be written directly in Scheme!
Hello, world!
Let's start with a "Hello, world" application that simply adds a text node to the document body.
(define-foreign document-body
"document" "body"
;; Parameters: none
;; Result: an external reference which may be null
-> (ref null extern))
(define-foreign make-text-node
"document" "createTextNode"
;; Parameters: a string
;; Result: an external reference which may be null
(ref string) -> (ref null extern))
(define-foreign append-child!
"element" "appendChild"
;; Parameters: two external references which may be null
;; Result: an external reference which may be null
(ref null extern) (ref null extern) -> (ref null extern))
(append-child! (document-body) (make-text-node "Hello, world!"))
The define-foreign syntax declares a Wasm import with a given
signature that is bound to a Scheme variable. In this example, we'd
like access to document.body, document.createTextNode, and
element.appendChild.
Each import has a two-part name, which correspond to the strings in
the define-foreign expressions above. Imported functions have a
signature specifying the parameter and result types.
WebAssembly follows the capability security model (and if you know us, you know we're big fans). Wasm modules act as guests within a host environment — in our case, the host is the browser. By default, a Wasm module has no access to functions that interact with its host. The Wasm guest must be granted explicit permission by its host to be able to, for example, add DOM elements to a web page.
The way capabilities work in Wasm is as follows:
- The Wasm module declares a set of imports.
- When the Wasm module is instantiated, the host maps the import names to concrete implementations.
So, define-foreign merely declares the module's need for a
particular function import; it does not bind to a particular
function on the host. The Wasm guest cannot grant capabilities unto
itself.
The host environment in the browser is the JavaScript engine, so we must use JavaScript to bootstrap our Wasm program. To do so, we instantiate the Wasm module and pass along the implementations for the declared imports. The import names in JavaScript match up to the import names in the Wasm module. For our "Hello, world" example, that code looks like this:
window.addEventListener("load", function() {
Scheme.load_main("hello.wasm", {}, {
document: {
body() { return document.body; },
createTextNode: Document.prototype.createTextNode.bind(document)
},
element: {
appendChild(parent, child) { return parent.appendChild(child); }
}
});
});
The Scheme class is provided by our JavaScript integration library,
reflect.js.
This library provides a Scheme abstraction layer on top of the
WebAssembly
API
and is used to load Hoot binaries and manipulate Scheme values from
JavaScript.
The result:
HTML as Scheme data
Now that we've explained the basics, let's move on to something more
interesting. How about rendering an entire tree of DOM elements? For
that, we'll need to declare imports for document.createElement and
element.setAttribute:
(define-foreign make-element
"document" "createElement"
(ref string) -> (ref null extern))
(define-foreign set-attribute!
"element" "setAttribute"
(ref null extern) (ref string) (ref string) -> none)
And here's the additional JavaScript wrapper code:
// These are added to the existing bindings from the previous section.
{
document: {
createElement: Document.prototype.createElement.bind(document),
// ... the previously defined document functions
},
element: {
setAttribute(elem, name, value) { elem.setAttribute(name, value); },
// ... the previously defined element functions
}
}
Now we need some markup to render. Thanks to the symbolic
manipulation powers of Scheme, we have no need for an additional
language like React's
JSX to cleanly mix
markup with code. Scheme has a lovely little thing called
quote
which we can use to represent arbitrary data as Scheme expressions.
We'll use this to embed HTML within our Scheme code.
We use quote by prepending a single quote (') to an expression.
For example, the expression (+ 1 2 3) calls + with the numbers
1, 2, and 3 and returns 6. However, the expression '(+ 1 2 3) (note the ') does not call + but instead returns a list of 4
elements: the symbol +, followed by the numbers 1, 2, and 3.
In other words, quote turns code into data.
Now, to write HTML from within Scheme, we can leverage a format called SXML:
(define sxml
'(section
(h1 "Scheme rocks!")
(p "With Scheme, data is code and code is data.")
(small "Made with "
(a (@ (href "https://spritely.institute/hoot"))
"Guile Hoot"))))
section, h1, etc. are not procedure calls, they are literal
symbolic data.
To transform this Scheme data into a rendered document, we need to walk the SXML tree and make the necessary DOM API calls, like so:
(define (sxml->dom exp)
(match exp
;; The simple case: a string representing a text node.
((? string? str)
(make-text-node str))
;; An element tree. The first item is the HTML tag.
(((? symbol? tag) . body)
;; Create a new element with the given tag.
(let ((elem (make-element (symbol->string tag))))
(define (add-children children)
;; Recursively call sxml->dom for each child node and
;; append it to elem.
(for-each (lambda (child)
(append-child! elem (sxml->dom child)))
children))
(match body
;; '@' denotes an attribute list. Child nodes follow.
((('@ . attrs) . children)
;; Set attributes.
(for-each (lambda (attr)
(match attr
;; Attributes are (symbol string) tuples.
(((? symbol? name) (? string? val))
(set-attribute! elem
(symbol->string name)
val))))
attrs)
(add-children children))
;; No attributes, just a list of child nodes.
(children (add-children children)))
elem))))
Then we can use our newly defined sxml->dom procedure to generate
the element tree and add it to the document body:
(append-child! (document-body) (sxml->dom sxml))
The result:
HTML templating with Scheme
That was pretty neat, but we don't need JavaScript, let alone WebAssembly, to render a simple static document! In other words, we're far from done here. Let's introduce some interactivity by adding a button and a counter that displays how many times the button was clicked. We could take an imperative approach and modify the element by mutating the counter value every time the button is clicked. Indeed, for something this simple that would be fine. But just for fun, let's take a look at a more functional approach that uses a template to create the entire document body.
Scheme provides support for structured templating via
quasiquote,
which uses backticks (`) instead of single-quotes (').
Arbitrary code can be evaluated in the template by using unquote,
represented by a comma (,). The expression `(+ 1 2 ,(+ 1 2))
produces the list (+ 1 2 3). The first (+ ...) expression is
quoted, but the second is not, and thus (+ 1 2) is evaluated as code
and the value 3 is placed into the final list element.
Scheme's quasiquote stands in stark contrast to the limited string
templating available in most other languages. In JavaScript, we could
do `1 + 2 + ${1 + 2}`, but this form of templating lacks
structure. The result is just a flat string, which makes it clumsy to
work with and vulnerable to injection
bugs.
Below is an SXML template. It is a procedure which generates a
document based on the current value of *clicks*, a global mutable
variable that stores how many times the button was clicked:
;; It is a Scheme/Lisp naming convention to use asterisks, AKA
;; earmuffs, to mark global mutable variables.
(define *clicks* 0)
(define (template)
`(div (@ (id "container"))
(p ,(number->string *clicks*) " clicks")
;; Increment click counter when button is clicked.
(button (@ (click ,(lambda (event)
(set! *clicks* (+ *clicks* 1))
(render))))
"Click me")))
To handle interactive elements, we've added a new feature on top of
SXML. Notice that the button has a click property with a procedure
as its value. To get some interactivity, we need event handlers, and
we've chosen to encode the click handler into the template much like
how you could use the onclick attribute in plain HTML.
To make this work, we need imports for document.getElementById,
element.addEventListener and element.remove:
(define-foreign get-element-by-id
"document" "getElementById"
(ref string) -> (ref null extern))
(define-foreign add-event-listener!
"element" "addEventListener"
(ref null extern) (ref string) (ref null extern) -> none)
(define-foreign remove!
"element" "remove"
(ref null extern) -> none)
And the JavaScript bindings:
{
document: {
getElementById: Document.prototype.getElementById.bind(document),
// ...
},
element: {
remove(elem) { elem.remove(); },
addEventListener(elem, name, f) { elem.addEventListener(name, f); },
// ...
}
}
Here is what the attribute handling code in the sxml->dom procedure
looks like now:
(match body
((('@ . attrs) . children)
(for-each (lambda (attr)
(match attr
(((? symbol? name) (? string? val))
(set-attribute! elem
(symbol->string name)
val))
;; THIS IS THE NEW BIT!
;;
;; If the attribute's value is a procedure, add an
;; event listener.
(((? symbol? name) (? procedure? proc))
(add-event-listener! elem
(symbol->string name)
(procedure->external proc)))))
attrs)
(add-children children))
(children (add-children children)))
To keep things simple (for now), every time the button is clicked we'll delete what's in the document and re-render the template:
(define (render)
(let ((old (get-element-by-id "container")))
(unless (external-null? old) (remove! old)))
(append-child! (document-body) (sxml->dom (template))))
The result:
Building a virtual DOM
Wouldn't it be even cooler if we could apply some kind of React-like diffing algorithm that only updates the parts of the document that have changed when we need to re-render? Why yes, that would be cool! Let's do that now.
We'll add bindings for the
TreeWalker
interface to traverse the document, as well as some additional element
methods to get/set the value and checked properties, remove event
listeners, replace elements, remove attributes, and get event targets:
;; TreeWalker constructor:
(define-foreign make-tree-walker
"document" "createTreeWalker"
(ref null extern) -> (ref null extern))
;; TreeWalker API:
(define-foreign current-node
"treeWalker" "currentNode"
(ref null extern) -> (ref null extern))
(define-foreign set-current-node!
"treeWalker" "setCurrentNode"
(ref null extern) (ref null extern) -> (ref null extern))
(define-foreign next-node!
"treeWalker" "nextNode"
(ref null extern) -> (ref null extern))
(define-foreign first-child!
"treeWalker" "firstChild"
(ref null extern) -> (ref null extern))
(define-foreign next-sibling!
"treeWalker" "nextSibling"
(ref null extern) -> (ref null extern))
;; More element API:
(define-foreign element-value
"element" "value"
(ref null extern) -> (ref string))
(define-foreign set-element-value!
"element" "setValue"
(ref null extern) (ref string) -> none)
(define-foreign remove-event-listener!
"element" "removeEventListener"
(ref null extern) (ref string) (ref null extern) -> none)
(define-foreign replace-with!
"element" "replaceWith"
(ref null extern) (ref null extern) -> none)
(define-foreign remove-attribute!
"element" "removeAttribute"
(ref null extern) (ref string) -> none)
;; Event API:
(define-foreign event-target
"event" "target"
(ref null extern) -> (ref null extern))
And the JavaScript bindings:
{
document: {
createTreeWalker: Document.prototype.createTreeWalker.bind(document),
// ...
},
element: {
value(elem) { return elem.value; },
setValue(elem, value) { elem.value = value; },
checked(elem) { return elem.checked; },
setChecked(elem, checked) { elem.checked = (checked == 1); },
removeAttribute(elem, name) { elem.removeAttribute(name); },
removeEventListener(elem, name, f) { elem.removeEventListener(name, f); },
// ...
},
treeWalker: {
currentNode(walker) { return walker.currentNode; },
setCurrentNode(walker, node) { walker.currentNode = node; },
nextNode(walker) { return walker.nextNode(); },
firstChild(walker) { return walker.firstChild(); },
nextSibling(walker) { return walker.nextSibling(); }
},
event: {
target(event) { return event.target; }
}
}
Now we can implement the diffing algorithm. Below is an abridged version, but you can see the beautiful, fully-fledged source code on GitLab:
(define (virtual-dom-render root old new)
;; <nested helper procedures have been omitted>
(let ((walker (make-tree-walker root)))
(first-child! walker)
(let loop ((parent root)
(old old)
(new new))
(match old
(#f
;; It's the first render, so clear out whatever might be
;; in the actual DOM and render the entire tree. No
;; diffing necessary.
(let loop ((node (current-node walker)))
(unless (external-null? node)
(let ((next (next-sibling! walker)))
(remove! node)
(loop next))))
(append-child! parent (sxml->dom new)))
((? string?)
;; Replace text node with either a new text node if the
;; text has changed, or an element subtree if the text
;; has been replaced by an element.
(unless (and (string? new) (string-=? old new))
(let ((new-node (sxml->dom new)))
(replace-with! (current-node walker) new-node)
(set-current-node! walker new-node))))
(((? symbol? old-tag) . old-rest)
(let-values (((old-attrs old-children)
(attrs+children old-rest)))
(match new
((? string?)
;; Old node was an element, but the new node is a
;; string, so replace the element subtree with a
;; text node.
(let ((new-text (make-text-node new)))
(replace-with! (current-node walker) new-text)
(set-current-node! walker new-text)))
(((? symbol? new-tag) . new-rest)
(let-values (((new-attrs new-children)
(attrs+children new-rest)))
(cond
;; The element tag is the same, so modify the
;; inner contents of the element if necessary.
((eq? old-tag new-tag)
(let ((parent (current-node walker)))
(update-attrs parent old-attrs new-attrs)
(first-child! walker)
(let child-loop ((old old-children)
(new new-children))
(match old
(()
;; The old child list is empty, so
;; diffing stops here. All remaining
;; children in the new list are fresh
;; elements that need to be added.
(for-each
(lambda (new)
(append-child! parent (sxml->dom new)))
new))
((old-child . old-rest)
(match new
;; The new child list is empty, so any
;; remaining children in the old child
;; list need to be removed, including
;; the current one.
(()
(let rem-loop ((node (current-node walker)))
(unless (external-null? node)
(let ((next (next-sibling! walker)))
(remove! node)
(rem-loop next)))))
;; Recursively diff old and new child
;; elements.
((new-child . new-rest)
(loop parent old-child new-child)
(next-sibling! walker)
(child-loop old-rest new-rest))))))
(set-current-node! walker parent)))
;; New element tag is different than the old
;; one, so replace the entire element subtree.
(else
(replace-with! (current-node walker)
(sxml->dom new)))))))))))))
Now, instead of deleting and recreating every single element on the
DOM tree to update the text for the counter, we can call the
virtual-dom-render procedure and it will replace just one text node
instead.
;; We'll diff the new vdom against the current one.
(define *current-vdom* #f)
(define (render)
(let ((new-vdom (template)))
(virtual-dom-render (document-body) *current-vdom* new-vdom)
(set! *current-vdom* new-vdom)))
To-do app
Let's wrap things up with our take on a classic: the to-do list.
The humble to-do list is often used as a "Hello, world" of sorts for client-side UI libraries (there's even an entire website dedicated to them).
A to-do list has many tasks. Tasks have a name and a flag indicating
if the task has been completed. We'll use Scheme's
define-record-type to encapsulate a task:
(define-record-type <task>
(make-task name done?)
task?
(name task-name)
(done? task-done? set-task-done!))
Tasks can be created and deleted. We'll use a bit of global state for managing the task list as a substitute for actual persistent storage:
(define *tasks* '())
(define (add-task! task)
(set! *tasks* (cons task *tasks*)))
(define (remove-task! task)
(set! *tasks* (delq task *tasks*)))
Now we can define a template for rendering the UI:
(define (template)
(define (task-template task)
`(li (input (@ (type "checkbox")
;; Toggle done? flag on click.
(change ,(lambda (event)
(let* ((checkbox (event-target event))
(checked? (element-checked? checkbox)))
(set-task-done! task checked?)
(render))))
;; Check the box if task is done.
(checked ,(task-done? task))))
(span (@ (style "padding: 0 1em 0 1em;"))
;; Strikethrough if task is done.
,(if (task-done? task)
`(s ,(task-name task))
(task-name task)))
(a (@ (href "#")
;; Remove task on click.
(click ,(lambda (event)
(remove-task! task)
(render))))
"remove")))
`(div
(h2 "Tasks")
;; Tasks are stored in reverse order.
(ul ,@(map task-template (reverse *tasks*)))
(input (@ (id "new-task")
(placeholder "Write more Scheme")))
;; Add new task on click
(button (@ (click ,(lambda (event)
(let* ((input (get-element-by-id "new-task"))
(name (element-value input)))
(unless (string-=? name "")
(add-task! (make-task name #f))
(set-element-value! input "")
(render))))))
"Add task")))
For this final example, we've embedded the result in an iframe below. This requires a browser capable of Wasm GC and tail calls, such as Chrome 119 or Firefox 121:
It should be said that a React-like virtual DOM isn't necessarily the best way to implement a UI layer. We like how the abstraction turns the state of the UI into a function mapping application state to HTML elements, as it avoids an entire class of synchronization bugs that impact more imperative approaches. That said, the overhead introduced by a virtual DOM is not always acceptable. Simple web pages are better off with little to no client-side code so that they are more usable on low power devices. Still, a to-do application with a virtual DOM diffing algorithm is a neat yet familiar way to show off the expressive power of Hoot and its FFI.
Looking forward
With the introduction of an FFI, you can now implement nearly your entire web frontend in Scheme; the examples we've looked at today are but a glimpse of what's now possible!
In the future, we hope the Guile community will join us in developing a colorful variety of wrapper libraries for commonly-used web APIs, so that building with Hoot becomes increasingly fun and easy.
If you'd like to start playing around with the demos today, you can find the complete source code on GitLab.
To see everything that's new in Hoot 0.2.0, be sure to check out our release announcement post!