Spritely Networked Communities Institute

LWN on Spritely

2024-02-29T12:00:00Z

Daroc Alden over at LWN.net has penned a piece on the Institute's long-term goal of developing a new paradigm for distributed computing, our progress so far, and the role OCapN will play in supporting our work.

Full writeup

[Contract] Goblins (Scheme/Wasm) Integrations Engineer

2024-02-26T09:00:00Z

We're recruiting for a remote, 20hrs/wk contract as a Goblins (Scheme/Wasm) Integration Engineer. Come help us bring secure decentralized infrastructure to everyone!

Developer Community Forum open for everyone!

2024-02-02T10:00:00Z

The Spritely Institute developer community forum is now public and free for all to join and paricipate!

Our forum is the place to talk about all Spritely Institute projects and technolgies - the OCapn protocol, the Spritely Goblins secure distributed programming environment, and Guile HOOT bringing Scheme to WebAssembly (and browsers!).

How to get help with technical issues for Spritely Institute technologies

If you are running into an issue with the codebase or would like to report a bug, please check the issues on the relevant repo and/or the docs, and file an issue on the appropriate repo as necessary: Goblins Hoot OCapN
Use the forum search function to check if anyone has asked a similar question before. The responses may point you in a helpful direction! If there are no related threads, please sign up, review the community guidelines and post a new topic with your question.

If you do choose to join this community, please introduce yourself and tell us about your project!

Guile Hoot v0.3.0 released!

2024-01-30T11:30:00Z

We are excited to announce the release of Guile Hoot v0.3.0! Hoot is a Scheme to WebAssembly compiler backend for Guile, as well as a general purpose WebAssembly toolchain. In other words, Scheme in the browser!

Highlights

Partial implementation of the Scheme module system! In past releases, programs were compiled by combining user code with a large Scheme prelude that contained the entire standard library. In this release, we've extracted all of the R7RS interface to their appropriate (scheme ...) modules and some Hoot-specific interfaces to (hoot ...) modules. Users can now use the import syntax to import these modules at the beginning of their programs. If no import form appears, a default set of modules is imported. User-defined modules are not yet supported. We plan to address this in the next release.
Supporting the Scheme module system in a whole-program compiler like Hoot is no easy task due to procedural macros (syntax-case). If you're interested in the details, check out Hoot lead engineer Andy Wingo's blog post on the subject.
More of R7RS-small has been implemented and Hoot can now run over half of the Scheme benchmarks.
The toolchain can now emit and parse binaries with a name custom section. This optional feature makes binaries larger and is intended for debugging. By default, name sections are not emitted.
New hash table and weak key hashtable interfaces. Hoot's hashtable interface is inspired by, but does not conform to, R6RS. Hash keys are currently limited to identity (eq) based hashing. eqv and equal hashing will come in a future release.
The (hoot reflect) module now automatically loads reflect.wasm. In past releases, users would have to locate and load this binary on their own. It quickly became the most common source of confusion for new users. We hope it is easier than ever to get started with Hoot!

Read on for the full change log.

R7RS-small

Added import syntax.
Added truncate and exact-integer-sqrt procedures.
Added guard exception handling syntax.
Added call-with-port procedure.
Added complex number procedures make-polar, make-rectangular, magnitude, and angle.
Added string-map and substring procedures.
Added support for two list arguments in map and for-each (3+ lists still not implemented.)
Enhanced string->number to parse all types of numbers.
Fixed names of string comparison procedures. string-=? is now string=?, etc.
Fixed eta expansion of - and /.

Non-standard interfaces

Added hash tables and weak key hash tables.
Added external-non-null? procedure.
Added support for (ref extern) params/results to define-foreign.
Added support for (ref eq) results to define-foreign.
Fixed port seeking.

Compiler

Added library-group form which encapsulates the entire compilation unit based on Enabling cross-library optimization and compile-time error checking in the presence of procedural macros.
Added #:emit-names? and #:imports compiler flags.
Removed #:env compiler flag.
read-and-compile now looks for an import form and includes only those modules in the compilation unit when specified.
Added support for f64 named temporaries to %inline-wasm.
Fixed fast path check for quo primcall.
Fixed fast path code for rsh/immediate primcall.

Runtime

Uncaught exception details are now printed. True Scheme backtraces are not yet possible, however.
Fixed $argv table initialization to fit the maximum number of procedure arguments that can be known at compile time.
Fixed $apply to dynamically grow the $argv table when the number of arguments exceeds the current size.
Fixed $add-fracnum-fracnum when GCD is a bignum.
Fixed $quo when most negative fixnum is divided by -1.
Fixed stack pointer arithmetic in $fluid-ref*.

Toolchain

Added support for generating name custom section to resolve-wasm in (wasm-resolve).
Added support for name custom section to assemble-wasm in (wasm assemble).
Added support for name custom section to parse-wasm in (wasm parse).
Added id field to <wasm> type along with wasm-id accessor.
Added <names> type for name custom section.
Changed hoot-instantiate in (hoot reflect) to automatically load reflect.wasm from Hoot's installation directory or $HOOT_DATADIR.
reflect.js is now partially documented, though we hope to automatically generate all necessary JavaScript in the future so users need not concern themselves with this.

Browser compatibility

At last, Firefox 121, released in December, enables Wasm tail calls by default! This is the minimum Firefox version needed to run Hoot binaries without messing around in about:config.
Compatible with Google Chrome 119 or later.
Safari/WebKit is still unsupported.

Get Hoot 0.3.0!

Hoot is already available in GNU Guix:

$ guix pull
$ guix install guile-next guile-hoot

(Hoot currently requires a bleeding-edge version of Guile, hence guile-next above.)

Otherwise, Hoot can be built from source via our release tarball. See the Hoot homepage for a download link and GPG signature.

Documentation for Hoot 0.3.0, including build instructions, can be found here.

FOSDEM 2024

Several of us at Spritely will be attending both FOSDEM and the Guix days which precede them. Robin Templeton will be presenting Scheme in the Browser with Guile Hoot and WebAssembly in the Declarative and Minimalistic Computing devroom on Sunday, February 4th.

If you're planning to attend, please come say hi!

Get in touch!

For bug reports, pull requests, or just to follow along with development, check out the Hoot project on GitLab.

If you build something cool with Hoot, let us know on our community forum!

The code in this release was brought to you by Andy Wingo, Robin Templeton, David Thompson. The lovely Hoot art is by tessa. Special thanks to the MetaMask folks for funding this work!

Until next time, happy hooting! 🦉

Meet us at GuixDays & FOSDEM!

2024-01-23T09:00:00Z

We will be presenting and attending FOSDEM 2024 and the Guix Days unconference in Belgium the first week of February.

"Scheme in the Browser with Guile Hoot and WebAssembly" by Robin Templeton (Feb 4, 11:00 CET). Guile Hoot is a free and open source WebAssembly toolchain and Scheme compiler that is opening doors for all programming languages to run within Wasm environments.
"Spritely, Guile, Guix: a unified vision for user security" by Christine Lemmer-Webber (Feb 4, 12:00 CET). Spritely's goal is to create networked communities that puts people in control of their own identity and security. This talk will present a unified vision of how Spritely, Guile, and Guix can work together to bring user freedom and security to everyone!

We'll also be attending as many related dinner events as we can.

Let's talk about partnering

Spritely Institute is non-profit, and we are excited about partnering to unlock the power of secure decentralized applications. Those partnerships could take many forms, including possible joint-applications for project funding development of interoperable applications and infrastructure. Talk to us about how we might work together!

Request a f2f meeting in Brussels

Meet in-person with CTO Christine Lemmer-Webber, Goblins Engineer Jessica Talon, or Hoot Compiler Engineer Robin Templeton.

To request a meeting, join our developer community message board and see this topic to let us know in advance about the cool project you're interested in collaborating on. We'd love to schedule some time to meet with you.

Hope to see you there!

Spritely Goblins v0.12.0 Released: Two new netlayers join the OCapN family and more!

2024-01-22T10:30:00Z

We're excited to announce the v0.12.0 release of Spritely Goblins!

The highlights of this release:

New Prelay netlayer
New TCP-TLS netlayer
Various fixes and enhancements to CapTP support
A new Queue (FIFO) actor in our actor-lib library

Goblins is Spritely's distributed object programming environment with implementations for both Guile and Racket Scheme. Goblins provides an intuitive security model, automatic local transactions for locally synchronous operations, and an easy to use and efficient asynchronous programming interface for encapsulated objects which can live anywhere on the network.

(While both versions of Spritely Goblins are fully network compatible over the OCapN protocol, most new features are aimed at Guile Goblins.)

Prelay Netlayer

The Prelay netlayer (the "preview of relay" netlayer) is a new netlayer that sits on top of our existing netlayers and uses a client/server model. This works especially well with our new TCP-TLS netlayer, as users will often be behind firewalls and routers which require special configuration to allow for peer-to-peer connections.

This new netlayer hopefully will help connect more nodes who otherwise couldn't have connected to each other. It's currently a technology preview with some limitations, those being that traffic is unencrypted and unauthenticated, something akin to IRC or vanilla ActivityPub. It currently requires trusting the relay server, however in the near future we will be looking to address those limitations.

TCP-TLS Netlayer

This is a new netlayer which supports connections over TCP sockets with encryption built in with TLS. Eager readers of the our news blog might already familiar with this netlayer, as we built it last year and used it with the Fantasary Game Jam. This netlayer has now made its way into this release of Goblins.

Unlike the Tor netlayer, which requires extra software (the Tor daemon) and configuration, this allows for both simpler and quicker communication between two nodes. Encryption occurs with self-signed certificates. This netlayer works particularly well with the new Prelay netlayer as making peer-to-peer connections is often difficult due to firewall or router configurations.

A new Queue actor

This new queue actor exists (goblins actor-lib queue) and represents a queue (i.e. FIFO) where new items can be queued or removed. Please refer to the new documentation to see how to use this.

Improved CapTP support

CapTP is part of OCapN, the network protocol that Goblins uses to communicate securely over the network. Specifically CapTP is the bigger of the three specifications which make up OCapN, being that which supports object communication, promises, Third Party Handoffs and more. Since CapTP working the same across both of the two Goblins implementations, these changes come to both Racket & Guile Goblins.

The improvements we've made to CapTP are:

Added crossed hellos mitigation (when two nodes try to connect to each other at the same time)
Removal of the now deprecated op:bootstrap operation (this is now done by always exporting the bootstrap object at position `0``).
More robust verification on third party handoff certificates.

In addition to ensure both our implementations and other OCapN implementations work according to the specifications we're helping to develop in the OCapN group, we added the test suite code to each repository to make it easier to test them against the OCapN test suite.

Getting the releases

You can test the latest Guile Goblins release using, Guix:

$ guix pull
$ guix install guile-goblins

If you'd prefer to try out the Racket version:

raco pkg install goblins

You can also read the documentation for this release here:

FOSDEM 2024

Finally, while not directly related to the Spritely Goblins release, several of us at Spritely will be attending both FOSDEM and the Guix days which precede them. So, if you're planning to attend and would like to speak to us or tell us about the cool things you're building with Goblins, let us know by joining our community, we'd love to chat with you!

[EFF] Spritely and Veilid: Exciting Projects Building the Peer-to-Peer Web

2023-12-15T12:30:00Z

The EFF (Electric Frontier Foundation) blog gives a great overview of our work in their latest post Spritely and Veilid: Exciting Projects Building the Peer-to-Peer Web.

"Spritely is worth keeping an eye on. Being developed by the Institute of the same name, Spritely is a framework for building distributed apps that don’t even have to know that they’re distributed..."

[Read more @ EFF.org]

Scheme in Scheme on Wasm in the browser

2023-12-15T12:00:00Z

Hey, folks! Today we want to talk about the wonderful read-eval-print-loop (REPL). Thanks to WebAssembly (Wasm), it's becoming increasingly common for programming language websites to embed a REPL in which passersby can easily evaluate code and get a feel for the language without having to install anything on their computer. We'd like to do the same thing for our language of choice, Guile Scheme, using Guile Hoot.

Guile Hoot is a Scheme-to-Wasm compiler that leverages the Wasm garbage collection (GC) extension that has been rolling out to major browsers recently. Wasm GC has finally made it possible to use dynamic languages besides JavaScript on the client-side web, but it will still take some additional effort to bring our favorite tools along for the ride. In this post, we'll walk through building a tiny Scheme interpreter with a simple user interface and explain what's in store for the future.

To learn more about Hoot, check out the 0.2.0 release announcement from a couple of weeks ago!

The case of the missing REPL

For Scheme programmers (and Lisp programmers in general), the REPL is at the core of our development workflow. We modify some code in our text editor, evaluate it in the REPL, inspect the output or enter a debugger, and repeat the process until the program behaves the way we want.

We're used to having a native REPL process running directly on our operating system, but since Hoot is putting Scheme on the web, we'd really like a REPL available in the browser, too. A web REPL would be convenient for seasoned Schemers and newcomers alike. However, at this early stage in Hoot's development, a fully-featured Scheme REPL is not yet possible. We don't have access to the macro expander, interpreter, etc. from the comfort of our Wasm runtime... yet.

We'll have these things eventually, but what can we have now?

Well, we could implement an interpreter for a small subset of Scheme... in Scheme! Sounds pretty recursive, and if you know anything about Schemers you know we love recursion (and also, that we love recursion), so let's give it a shot!

Scheme in Scheme

In Chapter 4 of the classic computer science textbook Structure and Interpretation of Computer Programs (SICP), Gerald Sussman and Hal Abelson walk through building a small Scheme interpreter embedded in Scheme. They called this the "metacircular evaluator", meaning that it's an interpreter written in the language that it is interpreting.

This sounds pretty fancy — the kind of task reserved for MIT professors that write iconic textbooks — but implementing a Scheme interpreter is simpler than one might think! For one thing, since Scheme can manipulate its own syntax by design, we don't need to spend any time parsing, a notoriously complex subject in its own right. We can skip directly to the fun part instead: evaluation!

Spritely's own Scheme Primer distills SICP's interpreter into a much simpler form intended for people who are brand new to Scheme. The code examples in the rest of this post are derived from the "Scheme in Scheme" section of the Primer. If you're completely new to Scheme and want to better understand the code in this post, consider working through our Primer sometime!

The metacircular evaluator is composed of two main components: eval and apply:

eval processes an expression in the context of an environment.
apply calls a procedure (function) with a list of arguments.

The output of eval is fed to apply, which may feed more input to eval, and this mutually recursive cycle continues until the program runs out of expressions to evaluate.

In order to implement eval, we first need a way to represent environments. An environment maps variable names to their bound values. We'll use association lists for our environments.

An empty environment with no variable bindings is represented by the empty list '().

The first bit of code from the metacircular evaluator we'll cover is the extend-env procedure, which creates a new environment by extending an existing one:

(define (extend-env env names vals)
  ;; If there are no more variables to add to env, return env.
  (if (null? names)
      env
      ;; Otherwise, add the first binding to the recursive
      ;; extension of env with the rest of the bindings.
      (cons (cons (car names) (car vals))
            (extend-env env (cdr names) (cdr vals)))))

We can extend the empty environment to make a new environment that includes the variable foo:

(extend-env '() '(foo) '(1)) ; => ((foo . 1))

We can further extend that environment to make a new environment that includes the variable bar:

(extend-env '((foo . 1)) '(bar) '(2)) ; => ((bar . 2) (foo . 1))

Note that environment manipulation is implemented in a functional style. Extending an environment does not mutate the original one. This is because mutating the original environment would propagate the new bindings to other evaluation contexts, potentially breaking Scheme's lexical scoping rules.

We can look up a variable's value using the env-lookup procedure:

(define (env-lookup env name)
  ;; Lookup name in association list.
  (match (assq name env)
    ;; Success: return the bound value.
    ((_ . val) val)
    ;; Failure: throw an error.
    (#f (error "Variable unbound:" name))))

Example:

(env-lookup '((foo . 1)) 'foo) ; => 1

And now for the rest of the owl. Our toy eval supports:

booleans
numbers
strings
quoted expressions
conditionals
procedures as values
procedure calls

(define (eval expr env)
  (match expr
    ;; Booleans, numbers, strings
    ((or (? boolean?) (? number?) (? string?))
     expr)
    ;; Quoted expressions
    (('quote quoted-expr)
     quoted-expr)
    ;; Variable lookup
    ((? symbol? name)
     (env-lookup env name))
    ;; Conditionals
    (('if test consequent alternate)
     (if (eval test env)
         (eval consequent env)
         (eval alternate env)))
    ;; Procedures
    (('lambda args body)
     (lambda vals
       (eval body (extend-env env args vals))))
    ;; Procedure application
    ((proc-expr . arg-exprs)
     ;; Recursively evaluate the procedure expression and all
     ;; argument expressions, then apply the procedure with the
     ;; arguments.
     (apply (eval proc-expr env)
            ;; Recursively evaluate the arguments.
            (map (lambda (arg-expr)
                   (eval arg-expr env))
                 arg-exprs)))))

Fortunately, we don't need to implement apply ourselves. Since lambda forms in our interpreter evaluate to regular Scheme procedures, we can just use Scheme's built-in apply to pass arguments to them. Calling a procedure will recursively call eval on the procedure's body expression.

Example:

(eval '(+ 1 2 3) `((+ . ,+))) ; => 6

Incidentally, we've created a capability-secure system reminiscent of Wasm's security model! The program (+ 1 2 3) only has access to the + procedure from our Scheme runtime. Aside from creating unbounded loops, a devious developer can't do any harm if their code is evaluated in this restricted environment. Neat!

Scheme in Scheme on Wasm

Now that we have a simple evaluator, we can compile it with Hoot to run it inside a web browser. But to actually use it, we need a user interface. To make one, we can borrow the React-like rendering code we walked through in our previous tutorial on rendering web pages with Wasm, "Building interactive web pages with Guile Hoot"!

For simplicity, we'll go for a minimalist design reminiscent of a terminal. As for the REPL output log, we can store it as a list of strings. Let's pre-populate it with a friendly welcome message:

(define *log*
  '("Welcome to the Hoot metacircular evaluator demo!

This is a miniature Scheme interpreter written in Scheme that's been
compiled to WebAssembly, and is running directly in the browser!  Its
UI is also written in Scheme, and uses the Hoot FFI to render itself
to the DOM.

"))

(define (log-append! . lines)
  (set! *log* (append *log* lines)))

Here's the rendering template, which prints all the log lines, and appends the classic > prompt and a textarea to the bottom so the user can write some Scheme code:

(define prompt "> ")

(define (render)
  `(div (@ (class "container"))
        (h1 "Hoot REPL")
        (div (@ (id "repl")
                (class "repl repl-text"))
             (div (@ (class "log")) ,@*log*)
             (div (@ (class "prompt"))
                  ,prompt
                  (textarea (@ (id "expression")
                               (class "repl-text")
                               (placeholder "(+ 1 2 3)")
                               (keyup ,maybe-eval)))))))

Every time the user presses a key while focused on the textarea, we call the maybe-eval procedure shown below. When the user presses the Enter key, the REPL

evaluates their code
clears the textarea
appends any new output
scrolls to the bottom of the log to keep the terminal screen up to date

(define (maybe-eval event)
  ;; Get the event's key.
  (let ((key (keyboard-event-key event)))
    ;; Evaluate user code when Enter is pressed, but not when
    ;; Shift is being held so the user can edit across multiple
    ;; lines.
    (when (and (string-=? key "Enter")
               (not (keyboard-event-shift? event)))
      ;; Get the text within the expression textarea.
      (let* ((input (get-element-by-id "expression"))
             (exp (element-value input)))
        ;; If the textarea is empty, do nothing.
        (unless (string-=? exp "")
          ;; Clear the textarea.
          (set-element-value! input "")
          ;; Evaluate and append output to log.
          (eval! exp)
          ;; Update UI.
          (refresh!)
          ;; Scroll the log to show the next output.
          (scroll-to-bottom!))))))

At the bottom, you'll notice maybe-eval uses a new procedure we've called eval! (how exciting). eval! is eval + read — it parses the user's text using Scheme's built-in read procedure, calls eval, and prints the output to the log:

(define (eval! str)
  ;; Parse user input.
  (let ((exp (read (open-input-string str)))
        ;; Open output port.
        (output (open-output-string)))
    ;; Redirect all output to our output port.
    (parameterize ((current-output-port output))
      ;; Echo the prompt and user code.
      (display prompt)
      (display str)
      ;; Invoke the interpreter.
      (call-with-values (lambda () (eval exp init-env))
        ;; Display each returned value on its own line.
        (lambda vals
          (if (null? vals)
              (display "\n")
              (for-each (lambda (val)
                          (unless (unspecified? val)
                            (display "=> ")
                            (write val))
                          (newline))
                        vals)))))
    ;; Append output to log.
    (log-append! (get-output-string output))))

(Note how, thanks to read, we've elided the messy work of parsing.)

Expressions are evaluated within the context of the following environment that provides capabilities for basic arithmetic, lists, multiple value return, and printing:

(define init-env
  `((+ . ,+)
    (- . ,-)
    (* . ,*)
    (/ . ,/)
    (= . ,=)
    (cons . ,cons)
    (car . ,car)
    (cdr . ,cdr)
    (list . ,list)
    (pair? . ,pair?)
    (null? . ,null?)
    (values . ,values)
    (display . ,display)
    (newline . ,newline)))

The initial environment describes the primitive functionality of our interpreter. Primitives are simple things that the programmer can take for granted. This environment doesn't provide much, but it's enough to have a bit of fun.

And...

🥁

...here's the finished program!

Some expressions you could try:

Greet the world:

(display "Hello, world!")

Make a list of the veggies you want on your sandwich:

'(lettuce tomato onion pepper pickles)

Display all your favorite pets as multiple return values:

(values 'cat 'dog 'chicken)

Apply a procedure that squares numbers:

((lambda (x) (* x x)) 4)

Recursively apply a procedure to compute the nth Fibonacci number:

((lambda (f x) (f f x))
 (lambda (fib n)
   (if (= n 0)
       0
       (if (= n 1)
           1
           (+ (fib fib (- n 1))
              (fib fib (- n 2))))))
 10)

Whoa, that last one was kinda wild, huh? That's because our little interpreter lacks even the ability to bind variables outside of lambda. There is no let nor define, so recursive code gets weird fast! Talk about minimalism...

Scheming ahead

The interpreter we've shown here is just a toy, but in the coming months we expect to add support for the R7RS-small standard's eval, taking our REPL from toy to full Scheme interpreter.

Once we do so, we could embed Scheme REPLs directly into documents like the Scheme Primer, further reducing the activation energy required to get started with Scheme. And who knows what exciting integrations our community will come up with?

In the meantime, if you'd like to start immediately hacking on the REPL demo, you can find the complete source code on GitLab. If you're into homework assignments, try adding support for more Scheme syntax such as let, or adding more procedures to the environment to grant more power to the interpreter.

And be sure to show off what you build with Hoot in our community forum! (Use OCAPN2023 for the invite code when you join.)

Happy hooting! 🦉

Having a Hoot at the Lisp Game Jam: A Tale of Two Daves

2023-12-07T12:00:00Z

This October, several teams and individuals gathered online to create sixteen games over the course of ten days for the biannual Lisp Game Jam. The earliest recorded instance of the jam took place in 2010, then intermittently in the years that followed, before landing on a more semi-annual basis starting in 2016.

This year, two of the entries, Cybersol by David Wilson and Strigoform by Spritely's own Dave Thompson, leveraged Hoot, our open-source Wasm toolkit to enable Scheme in the browser. What follows is an interview with the two devs on how it felt to participate in the jam this year, as well as their experiences with Hoot.

Note: Interviews have been lightly formatted

How did you come to learn about the jam?

David Wilson

I've known about the Lisp Game Jam for years now and have tried participating a couple of other times but never submitted an entry. Since Hoot released 0.1.0 practically a week before the jam, it seemed like an excellent opportunity to participate and experiment with the toolchain.

Dave Thompson

I had become friends with one of the former jam hosts, David O'Toole (dto), via the #lispgames IRC channel. It turned out that dto and I live very close to each other and even went to the same college (though not concurrently), and through that connection I got more involved in hobby game development with Lisp, using Guile Scheme as my Lisp of choice. I submitted my first game to the jam that happened soon after: Spring 2016 Lisp Game Jam.

I try to participate in every jam, but most of the time I don't submit anything. Finishing stuff is hard!

How long have you been participating in this or other jams?

David Wilson

I've been participating in game jams like Ludum Dare, GitHub Game Off, and the Lisp Game Jam for around 10 years. They're a great way to try out new languages, libraries, and programming techniques! I usually live stream the work on my jam entries on the YouTube channel Flux Harmonic.

Dave Thompson

I started participating in 2016, when the jams started being hosted on itch.io, a hosting platform and digital store for independent games.

How did you start running the jam?

Note: Dave Thompson is a co-organizer of the game jam

Dave Thompson

In 2022, Michael Fiano (mfiano), who had been hosting the jam since late 2016, decided it was time for someone else to take over. Phil Hagelberg (technomancy), who maintains Fennel, among other things, volunteered. Phil felt it would be best to have some redundancy with a second person available to moderate, so I volunteered to co-host. You'll see my name in the "Hosted by" line starting with Lisp Game Jam 2022, but really Phil does all the work organizing the dates and setting up the jam page. The jams do not have strict rules, and there is no money involved, so there isn't much to moderate. Phil does kindly check in with me to see if his proposed dates for the next jam sound good to me. I always say "yes." :)

Can you tell us a bit about this year's entry?

David Wilson

I have a recurring idea to make keyboard-friendly versions of classic games like Solitaire, so this time I decided to do just that and worked on a cyberpunk-themed Solitaire game. Instead of clicking on the cards to move them around, you use keyboard keys that are mapped to the various card stacks so that you can be a lot more efficient with your movements.

Dave Thompson

I made a vertical scrolling space shooter with a retro arcade style called Strigoform. I wrote all of the code, Christine Lemmer-Webber came up with the name, the owl theme, and did all of the pixel art, and my wife Andrea Parthemore wrote the music.

What inspired your game?

David Wilson

The world needs more keyboard-driven games because using the mouse is boring :)

Dave Thompson

Fear! Hoot 0.1.0 had just been released, and since it was the very first release there were a lot of limitations and some bugs waiting to be discovered. I knew I'd be pushing my technical skills to get the game done in time, so I decided to play it safe otherwise and make something that was familiar to me. I have made several similar games for past jams, so a space shooter became the obvious safe choice. Also, Christine thought it would be cool if I made a space shooter and offered to make pixel art.

What tools did you use to make your game and why?

David Wilson

I used Emacs for development because it is the ultimate work environment, Inkscape for creating vector graphics, HTML5 canvas APIs for rendering the game, and Google Chrome Unstable for testing the game because it had all the bleeding-edge WebAssembly features needed by Hoot.

Dave Thompson

The game was made completely with FOSS. The code was written in Emacs because it's the best editor for Lisp out there. The art was made in LibreSprite because it is built for pixel artists. The level was made in Tiled because it's a flexible 2D map editor and I already had a parser for its TMX format written in Scheme. The music was made in MilkyTracker because it's good for retro chiptune music.

Most importantly, the source code was compiled with Guile Hoot because it builds Wasm binaries that can run in web browsers. I've done game jams long enough to know that browser games will be tried and rated by more people than games with only native builds because it is much less work for the user. Browser games are also more likely to work on other people's computers. Providing native builds for all major operating systems that work reliably is difficult as an independent developer.

What challenges did you encounter in development?

David Wilson

Because Hoot did not have a foreign function interface as of the 0.1.0 release, I had to write a decent portion of the game in JavaScript because Hoot could not call the JS functions needed for rendering the screen handling input, etc.

The biggest challenge there was designing the Scheme code to be purely focused on the game logic and to create a simple protocol for communicating the game state between the JavaScript and Scheme code so that the JS code can render the game and Scheme can process the user's actions. It was a lot of extra work to get around a temporary limitation of Hoot!

This problem has now disappeared as of Hoot 0.2.0. I've already started rewriting the game to make use of the new FFI capabilities.

Dave Thompson

Hoot's in an early stage of development, so challenges caused by bugs and missing features were to be expected. I ran into bugs with some floating point operations and hacked around them with some inline Wasm. There was a particularly gnarly compiler bug that caused the Scheme stack to get into an invalid state sometimes and throw a Wasm runtime exception. User defined record types, which Schemers know as define-record-type, wasn't implemented yet so I resorted to tagged vectors to describe all my data types for the player, enemies, etc. All of these issues and more have since been fixed!

It was also challenging to not be able to use Chickadee, the game programming library for Guile that I maintain. Chickadee uses the C FFI to interact with native libraries like libGL, libpng and libfreetype, so it needs extensive work to add a new backend that uses web APIs instead via the Hoot FFI.

Note: Check out Dave's Chickadee template here

What about successes or wins?

David Wilson

The main success was that I was able to implement basic working game logic and rendering in the time of the jam! The biggest win, though, was that, thanks to Hoot, I was able to write working Scheme code to run in a browser.

Dave Thompson

The biggest win is that the prototype FFI for Hoot that I was hacking on before the jam started worked well enough to access the HTML5 Canvas API. I wasn't sure if performance would be good enough without using WebGL, which I definitely did not have time to use, but it worked out great!

Another small success story happened a few days into the jam. When the jam started, I was writing the level data by hand, tile by tile, in the Scheme source. I thought that since it was just a jam game I could deal with how annoying it was to edit the level, but it quickly became a productivity destroyer. I decided I should use a proper map editor like Tiled. However, writing the code to fetch, parse, and transform a Tiled map in the browser was going to be too difficult to implement in the time I had. I thought I was stuck, but then I remembered: this is Scheme! Code is data! I could compile the map file to Scheme code at build time. That auto-generated Scheme code could then be composed with the Scheme code that I wrote for the rest of the game and then the level would be baked into the final binary. So, that's what I did and it worked great! No need to use an XML parser at runtime!

What do you hope to see from Hoot going forward?

David Wilson

Since Hoot is quickly reaching compatibility with R7RS-small, it will be a fully functional Scheme for basic application and game development. What I'm more excited about is new libraries written for Hoot which add APIs for graphics programming, UI development, and networking support via HTTP requests or WebSockets. I imagine that is an area where the Hoot community could help out significantly!

Dave Thompson

I'd like to see a lot of useful browser APIs get wrapped into ready-to-use Guile libraries. Hoot still lacks a module system, but that's on the way and it will enable the community that's growing around Hoot to start making cool and reusable things. Besides the DOM, the most important APIs for games are HTML5 Canvas, WebGL, WebGPU, and Web Audio. Porting Chickadee to the web will make it much easier to build games with Hoot.

Do you have any advice or favorite resources you'd like to share with future participants?

David Wilson

My advice for anyone participating in a game jam is to limit the scope of your ideas as much as possible so that you can actually complete the game! You'll have a lot more fun if you make a "simpler" game that you can finish and then spend the rest of the time polishing it to make it more fun.

Dave Thompson

This is common jam advice but it's so true that it's worth repeating: Scope your ambitions down. Way down. Whatever your brain is telling you is a reasonable goal, try to reduce it even further. This takes practice, and I've failed at it more times than I've succeeded. The Lisp Game Jam is great because of its relaxed rules and how it always takes place across 2 full weekends for a total of 10 days, which makes it accessible to busy people.

Come into the jam with your tools ready and don't wait until the last day to figure out how you're going to distribute your game. Toolchain issues are no fun, and jams are supposed to be fun!

To learn more about what the options are for making games with Lisp, check out the source code for the entries in past Lisp Game Jams. Every game is required to publish source code so there's a lot of good code to learn from! The Strigoform source code is available on my personal website. I think Hoot is great for building web games and I'm going to try using it for the next jam, too.

Are there any other questions you wish you'd been asked about the jam?

David Wilson

Maybe "What was your favorite game from the jam?" My answer would be Dave Thompson's Strigoform because it's much better than mine and a real example of what you can do with a fully-Scheme browser game using Hoot :)

Dave Thompson

I think you covered all the important stuff. Thanks! :)

Building interactive web pages with Guile Hoot

2023-11-30T13:30:00Z

A question we frequently hear in discussions about WebAssembly (Wasm) is:

"Can Wasm call the DOM (Document Object Model) API?"

The answer is: Yes, thanks to Wasm Garbage Collection!

In this post, we will use Guile Hoot (our Scheme to Wasm compiler) to demonstrate how a language that compiles to Wasm GC can be used to implement the kind of interactivity we're used to implementing with JavaScript. We'll start very simple and build up to something that resembles a React-like application.

In our previous post about running Scheme in the browser, we had to use quite a lot of JavaScript code to call Scheme procedures (functions), render the user interface, and handle user input. However, today we're pleased to announce that those days are behind us; Hoot 0.2.0, released today, now includes a foreign function interface (FFI) for calling JavaScript from Scheme. In other words, the vast majority of code in a Hoot application can now be written directly in Scheme!

Hello, world!

Let's start with a "Hello, world" application that simply adds a text node to the document body.

(define-foreign document-body
  "document" "body"
  ;; Parameters: none
  ;; Result: an external reference which may be null
  -> (ref null extern))

(define-foreign make-text-node
  "document" "createTextNode"
  ;; Parameters: a string
  ;; Result: an external reference which may be null
  (ref string) -> (ref null extern))

(define-foreign append-child!
  "element" "appendChild"
  ;; Parameters: two external references which may be null
  ;; Result: an external reference which may be null
  (ref null extern) (ref null extern) -> (ref null extern))

(append-child! (document-body) (make-text-node "Hello, world!"))

The define-foreign syntax declares a Wasm import with a given signature that is bound to a Scheme variable. In this example, we'd like access to document.body, document.createTextNode, and element.appendChild.

Each import has a two-part name, which correspond to the strings in the define-foreign expressions above. Imported functions have a signature specifying the parameter and result types.

WebAssembly follows the capability security model (and if you know us, you know we're big fans). Wasm modules act as guests within a host environment — in our case, the host is the browser. By default, a Wasm module has no access to functions that interact with its host. The Wasm guest must be granted explicit permission by its host to be able to, for example, add DOM elements to a web page.

The way capabilities work in Wasm is as follows:

The Wasm module declares a set of imports.
When the Wasm module is instantiated, the host maps the import names to concrete implementations.

So, define-foreign merely declares the module's need for a particular function import; it does not bind to a particular function on the host. The Wasm guest cannot grant capabilities unto itself.

The host environment in the browser is the JavaScript engine, so we must use JavaScript to bootstrap our Wasm program. To do so, we instantiate the Wasm module and pass along the implementations for the declared imports. The import names in JavaScript match up to the import names in the Wasm module. For our "Hello, world" example, that code looks like this:

window.addEventListener("load", function() {
  Scheme.load_main("hello.wasm", {}, {
    document: {
      body() { return document.body; },
      createTextNode: Document.prototype.createTextNode.bind(document)
    },
    element: {
      appendChild(parent, child) { return parent.appendChild(child); }
    }
  });
});

The Scheme class is provided by our JavaScript integration library, reflect.js. This library provides a Scheme abstraction layer on top of the WebAssembly API and is used to load Hoot binaries and manipulate Scheme values from JavaScript.

The result:

HTML as Scheme data

Now that we've explained the basics, let's move on to something more interesting. How about rendering an entire tree of DOM elements? For that, we'll need to declare imports for document.createElement and element.setAttribute:

(define-foreign make-element
  "document" "createElement"
  (ref string) -> (ref null extern))

(define-foreign set-attribute!
  "element" "setAttribute"
  (ref null extern) (ref string) (ref string) -> none)

And here's the additional JavaScript wrapper code:

// These are added to the existing bindings from the previous section.
{
  document: {
    createElement: Document.prototype.createElement.bind(document),
    // ... the previously defined document functions
  },
  element: {
    setAttribute(elem, name, value) { elem.setAttribute(name, value); },
    // ... the previously defined element functions
  }
}

Now we need some markup to render. Thanks to the symbolic manipulation powers of Scheme, we have no need for an additional language like React's JSX to cleanly mix markup with code. Scheme has a lovely little thing called quote which we can use to represent arbitrary data as Scheme expressions. We'll use this to embed HTML within our Scheme code.

We use quote by prepending a single quote (') to an expression. For example, the expression (+ 1 2 3) calls + with the numbers 1, 2, and 3 and returns 6. However, the expression '(+ 1 2 3) (note the ') does not call + but instead returns a list of 4 elements: the symbol +, followed by the numbers 1, 2, and 3. In other words, quote turns code into data.

Now, to write HTML from within Scheme, we can leverage a format called SXML:

(define sxml
  '(section
    (h1 "Scheme rocks!")
    (p "With Scheme, data is code and code is data.")
    (small "Made with "
           (a (@ (href "https://spritely.institute/hoot"))
              "Guile Hoot"))))

section, h1, etc. are not procedure calls, they are literal symbolic data.

To transform this Scheme data into a rendered document, we need to walk the SXML tree and make the necessary DOM API calls, like so:

(define (sxml->dom exp)
  (match exp
    ;; The simple case: a string representing a text node.
    ((? string? str)
     (make-text-node str))
    ;; An element tree.  The first item is the HTML tag.
    (((? symbol? tag) . body)
     ;; Create a new element with the given tag.
     (let ((elem (make-element (symbol->string tag))))
       (define (add-children children)
         ;; Recursively call sxml->dom for each child node and
         ;; append it to elem.
         (for-each (lambda (child)
                     (append-child! elem (sxml->dom child)))
                   children))
       (match body
         ;; '@' denotes an attribute list.  Child nodes follow.
         ((('@ . attrs) . children)
          ;; Set attributes.
          (for-each (lambda (attr)
                      (match attr
                        ;; Attributes are (symbol string) tuples.
                        (((? symbol? name) (? string? val))
                         (set-attribute! elem
                                         (symbol->string name)
                                         val))))
                    attrs)
          (add-children children))
         ;; No attributes, just a list of child nodes.
         (children (add-children children)))
       elem))))

Then we can use our newly defined sxml->dom procedure to generate the element tree and add it to the document body:

(append-child! (document-body) (sxml->dom sxml))

The result:

HTML templating with Scheme

That was pretty neat, but we don't need JavaScript, let alone WebAssembly, to render a simple static document! In other words, we're far from done here. Let's introduce some interactivity by adding a button and a counter that displays how many times the button was clicked. We could take an imperative approach and modify the element by mutating the counter value every time the button is clicked. Indeed, for something this simple that would be fine. But just for fun, let's take a look at a more functional approach that uses a template to create the entire document body.

Scheme provides support for structured templating via quasiquote, which uses backticks (`) instead of single-quotes ('). Arbitrary code can be evaluated in the template by using unquote, represented by a comma (,). The expression `(+ 1 2 ,(+ 1 2)) produces the list (+ 1 2 3). The first (+ ...) expression is quoted, but the second is not, and thus (+ 1 2) is evaluated as code and the value 3 is placed into the final list element.

Scheme's quasiquote stands in stark contrast to the limited string templating available in most other languages. In JavaScript, we could do `1 + 2 + ${1 + 2}`, but this form of templating lacks structure. The result is just a flat string, which makes it clumsy to work with and vulnerable to injection bugs.

Below is an SXML template. It is a procedure which generates a document based on the current value of *clicks*, a global mutable variable that stores how many times the button was clicked:

;; It is a Scheme/Lisp naming convention to use asterisks, AKA
;; earmuffs, to mark global mutable variables.
(define *clicks* 0)

(define (template)
  `(div (@ (id "container"))
    (p ,(number->string *clicks*) " clicks")
    ;; Increment click counter when button is clicked.
    (button (@ (click ,(lambda (event)
                         (set! *clicks* (+ *clicks* 1))
                         (render))))
            "Click me")))

To handle interactive elements, we've added a new feature on top of SXML. Notice that the button has a click property with a procedure as its value. To get some interactivity, we need event handlers, and we've chosen to encode the click handler into the template much like how you could use the onclick attribute in plain HTML.

To make this work, we need imports for document.getElementById, element.addEventListener and element.remove:

(define-foreign get-element-by-id
  "document" "getElementById"
  (ref string) -> (ref null extern))

(define-foreign add-event-listener!
  "element" "addEventListener"
  (ref null extern) (ref string) (ref null extern) -> none)

(define-foreign remove!
  "element" "remove"
  (ref null extern) -> none)

And the JavaScript bindings:

{
  document: {
    getElementById: Document.prototype.getElementById.bind(document),
    // ...
  },
  element: {
    remove(elem) { elem.remove(); },
    addEventListener(elem, name, f) { elem.addEventListener(name, f); },
    // ...
  }
}

Here is what the attribute handling code in the sxml->dom procedure looks like now:

(match body
  ((('@ . attrs) . children)
   (for-each (lambda (attr)
               (match attr
                 (((? symbol? name) (? string? val))
                  (set-attribute! elem
                                  (symbol->string name)
                                  val))
                 ;; THIS IS THE NEW BIT!
                 ;;
                 ;; If the attribute's value is a procedure, add an
                 ;; event listener.
                 (((? symbol? name) (? procedure? proc))
                  (add-event-listener! elem
                                       (symbol->string name)
                                       (procedure->external proc)))))
             attrs)
   (add-children children))
  (children (add-children children)))

To keep things simple (for now), every time the button is clicked we'll delete what's in the document and re-render the template:

(define (render)
  (let ((old (get-element-by-id "container")))
    (unless (external-null? old) (remove! old)))
  (append-child! (document-body) (sxml->dom (template))))

The result:

Building a virtual DOM

Wouldn't it be even cooler if we could apply some kind of React-like diffing algorithm that only updates the parts of the document that have changed when we need to re-render? Why yes, that would be cool! Let's do that now.

We'll add bindings for the TreeWalker interface to traverse the document, as well as some additional element methods to get/set the value and checked properties, remove event listeners, replace elements, remove attributes, and get event targets:

;; TreeWalker constructor:
(define-foreign make-tree-walker
  "document" "createTreeWalker"
  (ref null extern) -> (ref null extern))

;; TreeWalker API:
(define-foreign current-node
  "treeWalker" "currentNode"
  (ref null extern) -> (ref null extern))

(define-foreign set-current-node!
  "treeWalker" "setCurrentNode"
  (ref null extern) (ref null extern) -> (ref null extern))

(define-foreign next-node!
  "treeWalker" "nextNode"
  (ref null extern) -> (ref null extern))

(define-foreign first-child!
  "treeWalker" "firstChild"
  (ref null extern) -> (ref null extern))

(define-foreign next-sibling!
  "treeWalker" "nextSibling"
  (ref null extern) -> (ref null extern))

;; More element API:
(define-foreign element-value
  "element" "value"
  (ref null extern) -> (ref string))

(define-foreign set-element-value!
  "element" "setValue"
  (ref null extern) (ref string) -> none)

(define-foreign remove-event-listener!
  "element" "removeEventListener"
  (ref null extern) (ref string) (ref null extern) -> none)

(define-foreign replace-with!
  "element" "replaceWith"
  (ref null extern) (ref null extern) -> none)

(define-foreign remove-attribute!
  "element" "removeAttribute"
  (ref null extern) (ref string) -> none)

;; Event API:
(define-foreign event-target
  "event" "target"
  (ref null extern) -> (ref null extern))

And the JavaScript bindings:

{
  document: {
    createTreeWalker: Document.prototype.createTreeWalker.bind(document),
    // ...
  },
  element: {
    value(elem) { return elem.value; },
    setValue(elem, value) { elem.value = value; },
    checked(elem) { return elem.checked; },
    setChecked(elem, checked) { elem.checked = (checked == 1); },
    removeAttribute(elem, name) { elem.removeAttribute(name); },
    removeEventListener(elem, name, f) { elem.removeEventListener(name, f); },
    // ...
  },
  treeWalker: {
    currentNode(walker) { return walker.currentNode; },
    setCurrentNode(walker, node) { walker.currentNode = node; },
    nextNode(walker) { return walker.nextNode(); },
    firstChild(walker) { return walker.firstChild(); },
    nextSibling(walker) { return walker.nextSibling(); }
  },
  event: {
    target(event) { return event.target; }
  }
}

Now we can implement the diffing algorithm. Below is an abridged version, but you can see the beautiful, fully-fledged source code on GitLab:

(define (virtual-dom-render root old new)
  ;; <nested helper procedures have been omitted>
  (let ((walker (make-tree-walker root)))
    (first-child! walker)
    (let loop ((parent root)
               (old old)
               (new new))
      (match old
        (#f
         ;; It's the first render, so clear out whatever might be
         ;; in the actual DOM and render the entire tree.  No
         ;; diffing necessary.
         (let loop ((node (current-node walker)))
           (unless (external-null? node)
             (let ((next (next-sibling! walker)))
               (remove! node)
               (loop next))))
         (append-child! parent (sxml->dom new)))
        ((? string?)
         ;; Replace text node with either a new text node if the
         ;; text has changed, or an element subtree if the text
         ;; has been replaced by an element.
         (unless (and (string? new) (string-=? old new))
           (let ((new-node (sxml->dom new)))
             (replace-with! (current-node walker) new-node)
             (set-current-node! walker new-node))))
        (((? symbol? old-tag) . old-rest)
         (let-values (((old-attrs old-children)
                       (attrs+children old-rest)))
           (match new
             ((? string?)
              ;; Old node was an element, but the new node is a
              ;; string, so replace the element subtree with a
              ;; text node.
              (let ((new-text (make-text-node new)))
                (replace-with! (current-node walker) new-text)
                (set-current-node! walker new-text)))
             (((? symbol? new-tag) . new-rest)
              (let-values (((new-attrs new-children)
                            (attrs+children new-rest)))
                (cond
                 ;; The element tag is the same, so modify the
                 ;; inner contents of the element if necessary.
                 ((eq? old-tag new-tag)
                  (let ((parent (current-node walker)))
                    (update-attrs parent old-attrs new-attrs)
                    (first-child! walker)
                    (let child-loop ((old old-children)
                                     (new new-children))
                      (match old
                        (()
                         ;; The old child list is empty, so
                         ;; diffing stops here.  All remaining
                         ;; children in the new list are fresh
                         ;; elements that need to be added.
                         (for-each
                          (lambda (new)
                            (append-child! parent (sxml->dom new)))
                          new))
                        ((old-child . old-rest)
                         (match new
                           ;; The new child list is empty, so any
                           ;; remaining children in the old child
                           ;; list need to be removed, including
                           ;; the current one.
                           (()
                            (let rem-loop ((node (current-node walker)))
                              (unless (external-null? node)
                                (let ((next (next-sibling! walker)))
                                  (remove! node)
                                  (rem-loop next)))))
                           ;; Recursively diff old and new child
                           ;; elements.
                           ((new-child . new-rest)
                            (loop parent old-child new-child)
                            (next-sibling! walker)
                            (child-loop old-rest new-rest))))))
                    (set-current-node! walker parent)))
                 ;; New element tag is different than the old
                 ;; one, so replace the entire element subtree.
                 (else
                  (replace-with! (current-node walker)
                                 (sxml->dom new)))))))))))))

Now, instead of deleting and recreating every single element on the DOM tree to update the text for the counter, we can call the virtual-dom-render procedure and it will replace just one text node instead.

;; We'll diff the new vdom against the current one.
(define *current-vdom* #f)

(define (render)
  (let ((new-vdom (template)))
    (virtual-dom-render (document-body) *current-vdom* new-vdom)
    (set! *current-vdom* new-vdom)))

To-do app

Let's wrap things up with our take on a classic: the to-do list.

The humble to-do list is often used as a "Hello, world" of sorts for client-side UI libraries (there's even an entire website dedicated to them).

A to-do list has many tasks. Tasks have a name and a flag indicating if the task has been completed. We'll use Scheme's define-record-type to encapsulate a task:

(define-record-type <task>
  (make-task name done?)
  task?
  (name task-name)
  (done? task-done? set-task-done!))

Tasks can be created and deleted. We'll use a bit of global state for managing the task list as a substitute for actual persistent storage:

(define *tasks* '())

(define (add-task! task)
  (set! *tasks* (cons task *tasks*)))

(define (remove-task! task)
  (set! *tasks* (delq task *tasks*)))

Now we can define a template for rendering the UI:

(define (template)
  (define (task-template task)
    `(li (input (@ (type "checkbox")
                   ;; Toggle done? flag on click.
                   (change ,(lambda (event)
                              (let* ((checkbox (event-target event))
                                     (checked? (element-checked? checkbox)))
                                (set-task-done! task checked?)
                                (render))))
                   ;; Check the box if task is done.
                   (checked ,(task-done? task))))
         (span (@ (style "padding: 0 1em 0 1em;"))
               ;; Strikethrough if task is done.
               ,(if (task-done? task)
                    `(s ,(task-name task))
                    (task-name task)))
         (a (@ (href "#")
               ;; Remove task on click.
               (click ,(lambda (event)
                         (remove-task! task)
                         (render))))
            "remove")))
  `(div
    (h2 "Tasks")
    ;; Tasks are stored in reverse order.
    (ul ,@(map task-template (reverse *tasks*)))
    (input (@ (id "new-task")
              (placeholder "Write more Scheme")))
    ;; Add new task on click
    (button (@ (click ,(lambda (event)
                         (let* ((input (get-element-by-id "new-task"))
                                (name (element-value input)))
                           (unless (string-=? name "")
                             (add-task! (make-task name #f))
                             (set-element-value! input "")
                             (render))))))
            "Add task")))

For this final example, we've embedded the result in an iframe below. This requires a browser capable of Wasm GC and tail calls, such as Chrome 119 or Firefox 121:

It should be said that a React-like virtual DOM isn't necessarily the best way to implement a UI layer. We like how the abstraction turns the state of the UI into a function mapping application state to HTML elements, as it avoids an entire class of synchronization bugs that impact more imperative approaches. That said, the overhead introduced by a virtual DOM is not always acceptable. Simple web pages are better off with little to no client-side code so that they are more usable on low power devices. Still, a to-do application with a virtual DOM diffing algorithm is a neat yet familiar way to show off the expressive power of Hoot and its FFI.

Looking forward

With the introduction of an FFI, you can now implement nearly your entire web frontend in Scheme; the examples we've looked at today are but a glimpse of what's now possible!

In the future, we hope the Guile community will join us in developing a colorful variety of wrapper libraries for commonly-used web APIs, so that building with Hoot becomes increasingly fun and easy.

If you'd like to start playing around with the demos today, you can find the complete source code on GitLab.

To see everything that's new in Hoot 0.2.0, be sure to check out our release announcement post!

Guile Hoot v0.2.0 released!

2023-11-30T12:00:00Z

We are excited to announce the release of Guile Hoot v0.2.0! Hoot is a Scheme to WebAssembly compiler backend for Guile, as well as a general purpose WebAssembly toolchain. In other words, Scheme in the browser! It has been a busy two months since 0.1.0, and we've got a lot of goodies to share.

Highlights

Nearly all of R7RS-small is now implemented! Hoot 0.2.0 is now capable of running many more standard Scheme programs than 0.1.0.
A foreign function interface (FFI) has been added to make it easy to declare imported host functions and call them from Scheme.
User-defined record types, sorely missing from 0.1.0, have been added.

Read on for a detailed list of changes.

R7RS-small

Added record types and define-record-type syntax.
Added read.
Added case-lambda and case-lambda*.
Added system timer procedures current-second, current-jiffy, and jiffies-per-second.
Added promise procedures make-promise, promise, ``delay,force, anddelay-force`.
Added string->symbol.
Added partial string->number implementation.
Added character procedures char-upcase, char-downcase, char-titlecase, char-upper-case?, char-lower-case?, char-alphabetic?, char-whitespace?, char-numeric?, digit-value, char-ci<?, char-ci<=?, char-ci=?, char-ci>=?, and char-ci>?.
Added mutable string procedures string-set!, string-copy!, and string-fill!.
Added stubs for command-line, get-environment-variable, and get-environment-variables procedures that do not make much sense in a web context.
Added integer flonum support for quotient, remainder, and modulo.
Fixed string->?.
Fixed assq, assv, and assoc.
Fixed memq, memv, and member.
Fixed gcd, expt, and exact.
Fixed finite?, infinite?, and nan?.
Fixed peek-char and read-char for multibyte chars.

Non-standard interfaces

Added foreign function interface via define-foreign syntax.
Added support for unboxed parameters and results to %inline-wasm.
Added %wasm-import form for use by the FFI.
Added Guile's structured exception system. All errors raised in the Scheme prelude are now structured.

Toolchain

Added global lowering pass to more easily emit Scheme constants that are not Wasm constants.
Added support for symbols/keywords in secondary Wasm modules that import the ABI from a primary module.
Improved compiler tree shaker to reduce overall Wasm binary size.
Reduced the size of bytevector literals in Wasm binaries.
Fixed lowering of stringview_iter.slice instruction.
Fixed flonum constants sometimes being treated as fixnums.
Fixed printing of mutable strings in both JS and Scheme reflection libraries.
Fixed scm_from_char in reflect.wasm.
Fixed br_table implementation in Wasm interpreter.
Fixed mis-compilation of rational? and integer? primcalls.
Fixed switch optimization for repeated symbol comparisons.

Browser compatibility

Google Chrome 119 was released about a month ago and comes with Wasm GC and tail calls enabled by default. Hoot binaries now work in Chrome without fiddling with any experimental settings!
Firefox 120 was released with Wasm GC enabled, but tail calls will not be enabled until December 19th when Firefox 121 is released. For the time being, Firefox users can either install a Nightly build or enable javascript.options.wasm_tail_calls in about:config on Firefox 120.
Safari/WebKit is still unsupported.

Get Hoot 0.2.0!

Hoot is already available in GNU Guix:

$ guix pull
$ guix install guile-next guile-hoot

(Hoot currently requires a bleeding-edge version of Guile, hence guile-next above.)

Otherwise, Hoot can be built from source via our release tarball. See the Hoot landing page for a download link and GPG signature.

Documentation for Hoot 0.2.0, including build instructions, can be found here.

Check out our greatest hoots!

For a detailed walkthrough of using the new foreign function interface, check out our cool new blog post about building interactive web pages with Hoot!

Robin Templeton and David Thompson recently presented about Hoot at the SeaGL conference in Seattle, WA. The talk was livestreamed and you can catch the replay on YouTube!

For bug reports, pull requests, or just to follow along with development, check out the Hoot project on GitLab.

If you build something cool with Hoot, let us know on our community forum! (Use OCAPN2023 for the Invite Code when you join.)

The code in this release was brought to you by Andy Wingo, Robin Templeton, David Thompson, and Christine Lemmer-Webber. The lovely Hoot art is by tessa. Special thanks to the MetaMask folks for funding this work!

Happy hooting! 🦉

Introducing OCapN, interoperable capabilities over the network

2023-11-16T13:45:00Z

OCapN (Object Capability Network) is a new protocol to add support for secure, seamless, and distributed (often peer-to-peer) communication for networked applications.

OCapN was originally developed as part of Goblins, our distributed object programming environment. But OCapN is no longer just for Goblins! Thanks to the support of a grant from NLnet, we have documented OCapN's functionality as a set of specifications so that OCapN's technology can be incorporated into any programming language environment. We have been working with colleagues across several partner projects and organizations to shape these documents following a consensus-based review process.

Without OCapN, building networked applications such as social networks, chat rooms and online virtual worlds otherwise would have to manually solve many problems which OCapN handles automatically as part of its design. OCapN simplifies building otherwise complicated peer-to-peer systems and has many useful features:

Intuitive distributed programming: allows for networked programming such that it doesn't matter where objects live: they can be fully local or distributed over the network. Performing asynchronous operations against networked objects is the same as asynchronous programming against local ones.
Object capability security: makes complicated secure systems simpler to reason about following the principle of "if you don't have it, you can't use it."
Promise Pipelining: enables sending messages to a promise's potentially settled output. This streamlines reasoning about promises, and cuts down on round trips across the network.
Distributed Garbage Collection: powers garbage collection across multiple devices. While programming languages may sufficiently handle garbage collection within a local machine, when references to objects are scattered across a network, the network itself must also explicitly support garbage collection of obsolete objects.
Certificate-Based Third-Party Handoffs: provide a mechanism by which two machines can securely reference objects on a third. Connections between machines that potentially span across networks necessitate the free sharing of object references, even those which exist on machines outside that connection. Third-party handoffs enable such references to be intuitively and securely shared within Goblins, without needing to consider the network.

NLnet Grant

Jessica Tallon has been working with Spritely on an NLnet grant-funded initiative for the past year, with a focus on improving OCapN support in both Goblins implementations and initiating the standardization push for OCapN. Below, Jessica delivers her personal account of her efforts to develop, grow, and refine open standards for the protocol to ensure solid foundations and interoperability across the OCapN ecosystem over the course of the grant.

Milestone 1: Racket Goblins <-> Guile Goblins Interoperability

When I started on this grant work in late 2022, Racket Goblins had a mature OCapN implementation, but Guile Goblins had none. This meant the two implementations couldn't interoperate (send messages to each other) at all.

At the start of 2023, I completed this milestone, adding comprehensive OCapN support to Guile Goblins and a protocol to enable streamlined communication with its Racket counterpart.

To assess the boundaries of OCapN coverage, the team and I ported a simple and minimalist chat application used internally for testing from Racket Goblins to Guile Goblins.

The evaluation was successful, meaning that Goblins objects written in Guile scheme can send messages with those written in Racket scheme and vice versa over OCapN.

Milestone 2: Start the OCapN Pre-standardization Group

Although we've made significant additions to the OCapN standard — such as certificate-based third-party handoffs — through our work on its Goblins implementation, it's important to note that the protocol wasn't invented at Spritely; we've drawn from and built upon the contributions of many others (the E programming language, Agoric, and Cap'n Proto to name a few) and is actively being shaped by input from many developers of those projects.

The OCapN group is comprised of many organizations such as Spritely and Agoric as well as many independent experts and interested parties. The group's goal is to come to consensus on draft documents which define the protocols within OCapN.

If you're interested in joining the group and helping the standardization effort, or you just want to take a look at our work, please find more info at ocapn.org

Milestone 3: Three draft specifications

Part of the standardization effort is to have documents the OCapN group can develop as the group meets consensus. The initial draft specifications are based on the Goblins' implementations of OCapN, The OCapN protocol is split into three documents:

CapTP: covers the majority of the protocol including message sending, promises, promise pipelining, third-party handoffs.
Netlayers: describes how new netlayer types should be defined and documents the ones currently in use, such as the Tor Onion Services Netlayer. Netlayers are an abstraction for different transport protocols which CapTP uses to communicate on, this is because OCapN is designed to be agnostic to the specific protocol CapTP runs on. While Goblins so far has largely used Tor, as it provides strong anonymity, other protocols, such as libp2p, IBC or carrier pigeons with little flash drives could be used.
Locators: defines how representations of nodes (machines) on the network and specific objects are encoded. The node identity specifications include both in-band and out-of-band, string-like representations, both of which are needed in order to bootstrap new connections and to share references within established connections.

As the OCapN group progresses these standardization efforts, we'll continue to update these documents; and on Spritely's end, our Goblins OCapN implementation.

Milestone 4: Test suite

One of the main goals of the OCapN standardization effort is to develop an open standard with many implementations. To ensure all of them interoperate as expected, the OCapN test suite verifies that a given implementation has correctly implemented the protocol according to spec.

Excitingly, this test suite has already been put to use: Alexander Bondarenko's Haskell implementation of OCapN has been built against and passes the OCapN test suite!

Milestone 5: Both goblins implementations pass test suite

Perhaps it goes without saying, but Spritely's Racket and Guile Goblins' OCapN implementations should also both pass the test suite. This may seem simple because the specification documents, and therefore the test suite itself, were written based on our Goblins' OCapN implementations. However the standard drafts continued to change, and the test suite along with it, necessitating appropriate updates to Goblins in the process. In addition, we found and fixed a few bugs thanks to the new suite.

As of today, both implementations pass the test suite! 🎉

Where next?

While we've gotten a lot done during the last 12 months, we still have a ways to go on the road to having an OCapN standard. The higher level work of solidifying a shared set of definitions for OCapN will be done as a community as the OCapN group refines the documents until we reach a foundational consensus.

Once we've agreed upon the first version of the specification documents, with the test suite and Goblins implementations following in lockstep, we plan to bring these documents to a standards body such as IETF or W3C.

We also hope these documents and tools will empower more people to develop OCapN libraries, tools, and implementations, and grow a rich ecosystem so that more secure, distributed applications and games can be easily developed which put the users freedom and privacy first.

Acknowledgments

This work was funded through the NGI Assure Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. We want to say a big thanks for supporting this work and enabling us to do this important work.

Guile Hoot v0.1.0 released!

2023-10-16T16:00:00Z

We are excited to announce the first official release of Guile Hoot, v0.1.0! Hoot is a Scheme to WebAssembly compiler backend for GNU Guile, as well as a general purpose WebAssembly toolchain. In other words, Scheme in the browser! At long last, thanks to WASM GC, it’s finally possible for other dynamic languages to be first-class citizens on the web. Make room, JavaScript, you’re no longer the only game in town!

This initial v0.1.0 release features support for a subset of R7RS-small Scheme, a JavaScript library providing an interface to Scheme, and a full WASM toolchain implemented as a set of Scheme libraries.

Features

Hoot has a number of notable features that distinguish it from many other WASM tools out there:

Self-contained toolchain: No emscripten, binaryen, wabt, etc.
GC reference type usage: The host provides the garbage collector.
Small programs produce small binaries: Kilobytes, not megabytes.
Full development environment: Compile and run WASM binaries without leaving Guile.

Hoot’s WASM toolchain includes:

WAT parser: Converts WAT to WASM. Both folded and unfolded forms are supported.
Assembler: Generates WASM binaries.
Binary parser: Converts WASM binaries to an internal WASM data structure.
Disassembler: Converts WASM to WAT.
Linker: Composes WASM modules with a standard libary, adding in only what is actually used.
Interpreter: Validates and evaluates WASM without having to jump into the web browser.

Limitations

Hoot is still in an early phase of active development and there are many missing features, including:

Module system
Record types
Environments and evaluation
Foreign function interface

For our next release, we intend to provide full R7RS-small support as we work towards the larger goal of supporting all of Guile.

Get Hoot v0.1.0!

Hoot is already available in GNU Guix:

$ guix pull
$ guix install guile-next guile-hoot

(Hoot currently requires a bleeding-edge version of Guile, hence guile-next above.)

Otherwise, Hoot can be built from source via our release tarball. See the Hoot landing page for a download link and GPG signature.

Learn more!

Documentation for Hoot v0.1.0, including build instructions, can be found here.

For code samples and a live demo of Hoot, check out last week’s blog post about implementing Wireworld in Scheme and running it inside a web browser.

BONUS: Three members of the Spritely team recently appeared on the System Crafters live stream to talk with David Wilson in detail about Hoot. If this interests you, check out the recording of the stream.

For bug reports, pull requests, or just to follow along with development, check out the Hoot project on GitLab.

If you build something cool with Hoot, let us know on our community forum! (Use OCAPN2023 for the Invite Code when you join.)

This release was brought to you by Andy Wingo, Christine Lemmer-Webber, Robin Templeton, and David Thompson. Special thanks to the MetaMask folks for funding this work!

Happy hooting! 🦉

Scheme in the browser: A Hoot of a tale

2023-10-10T11:00:00Z

Hey there, it’s been awhile! We’re back to share some more exciting news about Guile Hoot, a WebAssembly toolchain and Scheme→WASM compiler. In our last post we demonstrated that the Guile Hoot toolchain can be used to assemble programs written in WebAssembly Text (WAT) format, which allowed us to develop for the WASM-4 fantasy console. That was pretty cool, of course, but our goal is to compile Scheme programs (of the R7RS-small dialect, for now) to WASM. We are getting ready for our first release, and we’re happy to report that we’ve made excellent progress! We can now demonstrate the Hoot compiler working end-to-end.

Before we get to the demo, let’s go over what makes Hoot special. Hoot differs from most WASM compiler projects in some notable ways:

Self-contained toolchain: No emscripten, binaryen, wabt, etc.
GC reference type usage: The host provides the garbage collector.
Small programs produce small binaries: Kilobytes, not megabytes.
Full development environment: Compile and run WASM binaries without leaving Guile.

Self-contained toolchain

The Hoot toolchain has been written from the ground up to be self-contained with no external dependencies required (aside from Guile, of course.) This gives us a lot of flexibility and control over the entire compilation process. The toolchain includes:

WAT parser: Converts WAT to WASM. Both folded and unfolded forms are supported.
Assembler: Generates WASM binaries.
Binary parser: Converts WASM binaries to an internal WASM data structure.
Disassembler: Converts WASM to WAT.
Linker: Composes WASM modules with a standard libary (for example), adding in only what is actually used.
Interpreter: Validates and evaluates WASM without having to jump into the web browser.

All of these tools are available as a Scheme library that could be used to automate other WASM targeted build workflows.

GC reference types

In WASM 1.0, if a language implementation needed a garbage collector then it had to bring its own built on top of linear memory. There are many drawbacks to this approach:

Large binaries: A whole GC needs to be shipped along with the program.
Stop the world: Parallelism and concurrency are limited in WASM right now, so GC performance suffers.
Collectable confusion: How do you know when references to your heap objects are collectable if they’ve been passed to JavaScript or other WASM modules? It’s a heap of trouble.

In other words, you had to ship a bad garbage collector when there is a battle tested, high performance, production garbage collector sitting right there in the browser. Check out Andy Wingo’s excellent BOB 2023 talk (in text or video form) for more detailed information about the problems with GC in WASM 1.0. (Andy is the lead engineer for the Hoot project.)

Thankfully, WASM GC has added the ability to allocate heap values that are managed by the host. Hoot is currently one of the few dynamic language implementations that makes use of GC reference types. We hope that by sharing our work and our experience we can help other dynamic languages find their way into web browsers, too.

Hoot is also taking advantage of another recent addition to WASM: tail calls. Loops are expressed as recursive function calls in Scheme, so having a tail call feature available simplifies our implementation effort. We don’t have to wait long to use these features, either. Production versions of WASM GC and tail calls are already shipping in bleeding edge versions of browsers and will become available to all users of the web soon.

Small programs produce small binaries

We made the choice when starting this project not to compile Guile’s C runtime to WASM using emscripten. While it would have been feasible, the resulting binaries would have been quite large, among other things. We did not want the end result of this project to be a multiple megabyte binary for “Hello, world.” Instead, we’ve taken the approach of writing a dedicated compiler backend for Guile with some handwritten WASM providing primitive functionality and calling out to imported host functions when necessary. Programs compiled with Hoot include only the parts of the standard library that they rely upon. As mentioned earlier, WASM GC obviates the need to link in an entire garbage collector. For these reasons, Hoot binaries tend to be much smaller than the output of most other compilers. Small programs produce binaries measured in kilobytes, not megabytes.

Full development environment

Hoot includes an embedded WASM interpreter that makes it possible to run WASM directly within the Guile process being used for development. Since the toolchain is also readily available as a set of Scheme modules, you can compile and test out programs (either Scheme programs or hand-crafted WAT) from the comfort of the Guile REPL. There is also a set of REPL tools to make inspecting and debugging WASM easier. Keep reading for a practical demonstration of the development workflow.

OK, onto Wireworld!

Wireworld part 1: Scheme

To show off the compiler, we decided to revisit our favorite cellular automaton: Wireworld. Check out our last post to learn more about what Wireworld is and how we used Hoot to build a pure WASM version of it. This time, instead of writing several hundred lines of WAT, we wrote about 70 lines of Scheme:

(let ((grid-size 40)
      (empty 0)
      (cu 1)     ; copper
      (ehead 2)  ; electron head
      (etail 3)) ; electron tail
  (define (make-grid)
    (make-bytevector (* grid-size grid-size) 0))

  (define (grid-ref grid x y)
    (bytevector-u8-ref grid (+ (* y grid-size) x)))

  (define (grid-ref/wrap grid x y)
    (bytevector-u8-ref grid (+ (* (modulo y grid-size) grid-size)
                               (modulo x grid-size))))

  (define (grid-set! grid x y t)
    (bytevector-u8-set! grid (+ (* y grid-size) x) t))

  (define (neighbors grid x y)
    (define (check x y)
      (if (= (grid-ref/wrap grid x y) ehead) 1 0))
    (+ (check (- x 1) (- y 1))
       (check x (- y 1))
       (check (+ x 1) (- y 1))
       (check (+ x 1) y)
       (check (+ x 1) (+ y 1))
       (check x (+ y 1))
       (check (- x 1) (+ y 1))
       (check (- x 1) y)))

  (define (update from to)
    (do ((y 0 (+ y 1)))
        ((= y grid-size))
      (do ((x 0 (+ x 1)))
          ((= x grid-size))
        (let* ((t (grid-ref from x y))
               (t* (cond
                    ((= t empty) empty)
                    ((= t cu)
                     (if (<= 1 (neighbors from x y) 2) ehead cu))
                    ((= t ehead) etail)
                    ((= t etail) cu))))
          (grid-set! to x y t*)))))

  (define (copy from to)
    (let ((k (bytevector-length from)))
      (do ((i 0 (+ i 1)))
          ((= i k))
        (bytevector-u8-set! to i (bytevector-u8-ref from i)))))

  (define grid-a (make-grid))
  (define grid-b (make-grid))

  (define (update-and-swap)
    (update grid-a grid-b)
    (copy grid-b grid-a))

  (define (ref x y)
    (grid-ref grid-a x y))

  (define (set x y t)
    (grid-set! grid-a x y t))

  (define (cycle x y)
    (set x y (modulo (+ (ref x y) 1) 4)))

  (values grid-size ref set cycle update-and-swap)))

After compiling and gzipping the resulting binary, we’re looking at a 28KiB download. Not too shabby! (And we’re anticipating adding a size optimization pass resembling binaryen’s wasm-opt -Oz in the future to reduce binary size even further.)

Before we get to the browser, let’s take a quick detour and talk about development workflow. It took several iterations to arrive at the code above. In true Scheme fashion, the program was developed incrementally in an interactive REPL session. By using Hoot’s WASM interpreter, we repeatedly compiled and tested the program without having to load it into a web browser or shell out to V8’s d8 tool. We think it’s pretty neat! Below is an annotated excerpt of a REPL session that shows the values of 3 cells before and after an update.

;; Import necessary Guile modules.
scheme@(guile-user)> ,use (hoot reflect) (wasm parse)
;; Load reflection WASM module.
scheme@(guile-user)> (define reflect-wasm
                       (call-with-input-file "js-runtime/reflect.wasm"
                                             parse-wasm))
;; Compile and evaluate Wireworld source.
scheme@(guile-user)> (define-values (grid-size grid-ref grid-set
                                     grid-cycle grid-update)
                       (compile-value reflect-wasm wireworld-src))
scheme@(guile-user)> (grid-ref 0 0)  ; electron tail at (0, 0)
$10 = 3
scheme@(guile-user)> (grid-ref 1 0)  ; electron head at (1, 0)
$11 = 2
scheme@(guile-user)> (grid-ref 2 0)  ; copper at (2, 0)
$12 = 1
scheme@(guile-user)> (grid-update)   ; step the simulation
scheme@(guile-user)> (grid-ref 0 0)  ; copper at (0, 0)
$13 = 1
scheme@(guile-user)> (grid-ref 1 0)  ; electron tail at (1, 0)
$14 = 3
scheme@(guile-user)> (grid-ref 2 0)  ; electron head at (2, 0)
$15 = 2

The electron has moved to the right one cell, exactly as it should. With some amount of confidence that the program is working, we proceed to the web browser.

Wireworld part 2: JavaScript

To get this version of Wireworld running in the browser, an additional ~100 lines of JavaScript were used to glue the Scheme program to various browser APIs. Since we aren’t targeting WASM-4 this time, we needed a new rendering method. We chose HTML5 canvas to keep things simple. The code below updates the simulation at some regular interval, renders to the canvas, and handles keyboard/mouse input. Hoot’s reflect.js library allows us to call Scheme procedures from JS.

async function init() {
  const canvas = document.getElementById("canvas");
  const ctx = canvas.getContext("2d");
  const [_gridSize, gridRef, gridSet, gridCycle, gridUpdate] =
        await Scheme.load_main("scheme-wireworld.wasm", {});
  const gridSize = Number(_gridSize), tileSize = 16;
  const canvasSize = gridSize * tileSize;
  const empty = 0, cu = 1, ehead = 2, etail = 3;
  let paused = true, mouseX = 0, mouseY = 0;

  function render() {
    requestAnimationFrame(() => {
      for (var y = 0; y < gridSize; y++) {
        for (var x = 0; x < gridSize; x++) {
          const t = Number(gridRef.call(BigInt(x), BigInt(y)));
          switch(t) {
          case empty:
            ctx.fillStyle = "#fff6d3";
            break;
          case cu:
            ctx.fillStyle = "#f9a875";
            break;
          case ehead:
            ctx.fillStyle = "#7c3f58";
            break;
          case etail:
            ctx.fillStyle = "#eb6b6f";
            break;
          }
          ctx.fillRect(x * tileSize, y * tileSize, tileSize, tileSize);
          if(x == mouseX && y == mouseY) {
            ctx.fillStyle = "#00000020";
            ctx.fillRect(x * tileSize, y * tileSize, tileSize, tileSize);
          }
        }
      }
    });
  }

  function update() {
    if(!paused) {
      gridUpdate.call();
      render();
    }
    setTimeout(update, 100);
  }

  function pixelToTile(p) {
    return Math.min(Math.max(Math.floor(p / tileSize), 0), gridSize - 1);
  }

  function cycleTile(x, y) {
    gridCycle.call(BigInt(x), BigInt(y));
    render();
  }

  function setTile(x, y, t) {
    gridSet.call(BigInt(x), BigInt(y), BigInt(t));
    render();
  }

  function clearTile(x, y) {
    setTile(x, y, empty);
  }

  function togglePause() {
    paused = !paused;
  }

  canvas.width = canvasSize;
  canvas.height = canvasSize;
  canvas.addEventListener("mousemove", (event) => {
    mouseX = pixelToTile(event.offsetX);
    mouseY = pixelToTile(event.offsetY);
    if(event.buttons == 1) { // left
      setTile(mouseX, mouseY, cu);
    } else if(event.buttons == 2) { //right
      clearTile(mouseX, mouseY);
    }
    render();
  });
  canvas.addEventListener("mousedown", (event) => {
    const x = pixelToTile(event.offsetX);
    const y = pixelToTile(event.offsetY);
    if(event.button == 0) { // left
      cycleTile(x, y);
    } else if(event.button == 2) { // right
      clearTile(x, y);
    }
  });
  canvas.addEventListener("contextmenu", (event) => event.preventDefault());
  document.addEventListener("keydown", (event) => {
    if(event.code == "Space") togglePause();
  });
  update();
  render();
}
window.addEventListener("load", init);

Note the use of BigInt in several places. This is a consequence of how JavaScript’s Number type works. JS numbers are floating point values, but Scheme has a much richer set of numeric types including exact integers. So, values of type Number are treated as Scheme floats, and the BigInt type is used to box all integer values that traverse the JS⇄Scheme bridge.

As Hoot matures and we implement a foreign function interface (FFI) for calling into JS (or other hosts) from Scheme, we expect to see the amount of glue code shrink considerably.

Live demo

OK, now it’s time to see Scheme Wireworld in action! But first, a disclaimer: As of publishing, major web browsers are just beginning to ship WASM GC support. Starting with Firefox 120 and Chrome 119, GC is enabled by default. Neither version is stable yet, so use either Firefox Nightly or Chrome Dev if you’d like to run the demo. If you’re reading this further into the future then congratulations! The browser you are using should have everything you need.

How to use:

Press the left mouse button and move the cursor to draw copper.
Press the right mouse button and move the cursor to erase.
Left click on a cell to cycle its type (copper becomes electron head, etc.)
Right click on a cell to clear it.
Press the space bar to pause or resume the simulation.

Now you try!

If you’d like to compile and run this yourself, check out our GitLab repo for this demo. Like most of our projects, it uses Guix to setup the development environment. The environment requires an unreleased version of Guile compiled from source, so be prepared to wait awhile for it to build.

git clone https://gitlab.com/spritely/scheme-wireworld.git
cd scheme-wireworld
guix shell
# The following commands are evaluated *within* the Guix shell
guile wireworld.scm
guile web-server.scm

Once the web server is up and running, visit http://localhost:8080 in your web browser. Feel free to modify the code and have some fun!

We look forward to sharing more when we release Hoot 0.1.0!

[Hiring] Open Source Developer Relations Manager

2023-09-29T15:00:00Z

We're hiring an Open Source Developer Relations Manager - come help us support the Goblins ecosystem!

We will be posting full-time, part-time, and contract openings shortly for Engineers, UI/UX Design/Research, and (Funding) Development as funding permits. If you would like to be notified of job openings, email jobs@spritely.institute, and we'll contact you the moment they are listed.

Spritely Networked Communities Institute is an Equal Opportunity Employer and is committed to ensuring that all employees and applicants receive equal consideration and treatment, regardless of race, color, creed, gender (including gender identity or gender expression), religion, marital or domestic partner status, age, body size, national origin or ancestry, physical, mental or medical disability, sex, sexual orientation, citizenship, military service status, veteran status, or any other characteristic protected by state or federal law or local ordinance.

Lisp Game Jam - "Wireworld" - Hoot's low level WASM tooling in action

2023-06-12T14:00:00Z

This blogpost focuses on our second Spring Lisp Game Jam 2023 entry, wasm4-wireworld, an implementation of Wireworld on top of Hoot's lower-level assembly tools which are a part of our Guile → WASM project to bring Spritely Goblins-powered distributed applications to the common web browser.

For our entry, we targeted WASM-4, a "fantasy console" which uses low-level WebAssembly constructs.

In fact, if you see the animated "wireworld" splash screen at the top of this article, that's running wasm-wireworld itself... you can even run it live in your browser!

A brief intro to Wireworld

The real point of this blogpost isn't Wireworld itself, but how Hoot enabled making this Wireworld demo. So we're going to walk into how Hoot turned out to be an incredible toolkit to build this game jam entry, but first let's talk about Wireworld to give some more context!

Wireworld is a powerful cellular automata, categorically similar to Conway's Game of Life. Unlike Life, Wireworld's paths are fixed: "electrons" flow upon copper wires. The rules of Wireworld are very simple:

Blank tiles remain blank
Electron heads always become electron tails
Electron tails always become copper
Copper stays copper, unless there are one or two electron heads in any neighboring cells, in which case it becomes an electron head

That's it! Despite that simplicity, this leads to powerful emergent behavior. Electron head and tail pairs appear to "flow" fluidly upon copper wires and can branch easily. But we can also make "generators" and diodes that only permit electrons to flow in one direction and not another:

(Try it live!)

These rules result in behavior which permits visually stunning examples, many of which even look like computational circuitry.

Below is a "binary adder" circuit running our implementation of Wireworld.

Encoded in little endian, 3 (011) + 6 (110) = 9 (1001)! (And here's the live version... press the "x" button to start the simulation when you're ready!)

If you find this kind of thing delightful, see The Wireworld Computer for a tutorial that uses Wireworld to build from the simplest elements all the way up to a computer that can calculate prime numbers.

Try Wireworld yourself!

While building a cool version of Wireworld wasn't the real goal of our participation in the game jam, it was a lot of fun to build... and we hope it's fun to use and play with, too! And since you're reading this in a web browser and we're using Webassembly anyway, why not try playing with Wireworld right here in the browser?

You can click to place wires and electron heads/tails and press X to start/stop the emulation. Enjoy!

Insights from using Hoot to build WASM-4 Wireworld

Our intent in this game jam was to show off how the "lower level layers" of Hoot (i.e. the assembler, disassembler, etc) are themselves useful and powerful and could be used to compile something interactive and compelling. WASM-4's minimalist homage to 80s video game consoles helped us deliver on this focus by encouraging "coding towards the bare metal".

As usual, game jams force you to pare down features. For instance, 4-colored sprites were planned from the beginning, and we considered some unusual additions to wireworld, such as tiles which could make noise, which we thought maybe would allow Wireworld to be used as a custom music synthesizer.

We were happy to be able to deliver a completed and working Wireworld demo by the end of the game jam, but most of these features were removed, and notably we did not yet even have time to properly get in the sprites we had designed... instead we shipped with placeholder graphics using the "hello world" smiley from the WASM-4 tutorial!

We were surprised to wake up the next morning to find that Spritely community member Vivi had taken our program and not only made a cool world with it, but had modified the program itself:

Thus it was Vivi who first put a version of the binary adder (the general design taken from the wireworld computer) inside of wasm4-wireworld. With no assistance from Spritely's internal team, Vivi changed the grid size to render at 40x40 to have sufficient space to represent the binary adder (the sprite data was not changed for this grid redesign in Vivi's initial hack, leading to an entertaining effect resembling renderings from a corrupted Gameboy cartridge). The game jam submitted entry had a 20x20 grid size, but once we saw the binary adder, we knew we had to have it, so we resized the official implementation to 40x40 and redrew the sprites accordingly.

But the biggest takeaways from the jam were that Hoot, while still in early development, is already a very promising and powerful environment for doing low-level Webassembly programming. This merits some further explanation!

How we used Hoot to build wasm4-wireworld

The last time we talked about Hoot (our Scheme->WASM project) we talked about directly compiling Scheme to WebAssembly. This is of course the higher level goal of Hoot: since Spritely's tooling is written in Guile Scheme, we want Spritely to be in the browser, and compiling Scheme programs themselves to WebAssembly is a great way to accomplish that goal.

But what if we wanted to play with WebAssembly on a lower level? It turns out Hoot is a great choice for this: since Hoot uses Guile's compiler tower and has multiple steps of transformation, the lower level assembler, disassembler, etc tools of WebAssembly are also available to the inspired Guile hacker.

Though Hoot will be a complete Scheme→WebAssembly compiler... it already contains a nascent powerful and general WebAssembly toolkit!

Getting Technical: Making a minimal WASM-4 cart using Hoot

WASM-4 is intentionally minimal and sparse. No garbage collector extension, and no room for higher level constructs. You've got 64kb of memory, 4 colors, and individual instructions count.

WASM-4 runs fully compiled WebAssembly programs called "carts". A very minimal WASM-4 cart (in fact a cut down version of WASM-4's "hello world" cart) could be defined like so:

(define $smiley$ '(i32.const #x19a0))   ; location of the smiley
(define smiley-data
  #vu8(#b11000011      ; 1bpp (1 bit per pixel) sprite!
       #b10000001      ; We have two colors represented by 0 and 1
       #b00100100      ; which means if you squint
       #b00100100      ; you can kinda see the smiley!
       #b00000000
       #b00100100
       #b10011001
       #b11000011))

(define (our-game)
  `(module
    ;; Copies pixels to the framebuffer.
    (import "env" "blit" (func $blit (param i32 i32 i32 i32 i32 i32)))

    ;; Define the smiley sprite in memory
    (data ,$smiley$ ,smiley-data)

    (func (export "update")
      (call $blit ,$smiley$
            (i32.const 76) (i32.const 76)  ; draw at 76, 76
            (i32.const 8) (i32.const 8)    ; 8x8 sprite
            (i32.const 0)))))              ; 1bpp sprite

Here we are using a special lisp/scheme feature called "quasiquote". Quasiquote is enabled with the "back-tick" character appearing before module, switching into read-only data. In this way, inside of our-game, in the places we have written ,$smiley$ we are actually substituting in the $smiley$ definition from the top of the program, and where we say ,smiley-data we are inserting the bytevector describing the smiley sprite. This turns out to be a powerful templating system for producing WebAssembly, and we will use it in greater detail later.

Now let's say we want to try this cart. Since we're in a lisp and lisp culture encourages "live development", let's make a helper utility to compile and run our game:

(use-modules (wasm assemble))

(define* (try-game #:optional (game (our-game)))
  (call-with-output-file "our-game.wasm"
    (lambda (op)
      (put-bytevector op (wat->wasm game))))
  (system* "wasm4" "our-game.wasm"))

Now giving the game a try is as simple as running try-game at the REPL:

scheme@(guile-user)> (try-game)

Here's a version of the cart we just built above running live:

Programmatically generating WebAssembly

Unfortunately, WebAssembly is very verbose. Consider how messy our "update" procedure was already looking:

  `(func (export "update")
     (call $blit ,$smiley$
           (i32.const 76) (i32.const 76)  ; draw at 76, 76
           (i32.const 8) (i32.const 8)    ; 8x8 sprite
           (i32.const 0)))                ; 1bpp sprite

This is very noisy! All of those i32.const operations are getting in the way of us placing our smiley and defining its width and height. Not to mention that the final argument defines various flags specified by individual bits which are very hard to remember. If we were writing many of these, this would get very hard to read indeed. Time for an abstraction!

(define (maybe-i32.const x)
  (if (number? x)
      `(i32.const ,x)
      x))

(define* (call-blit sprite-ptr x y width height
                    #:key 2bpp? rotate? flip-x? flip-y?)
  (define flags
    (logior #b0                            ; start with empty byte
            (if 2bpp?   #b00000001 #b0)    ; rightmost bit for 2-bits-per-pixel
            (if flip-x? #b00000010 #b0)    ; second bit for flipping x axis
            (if flip-y? #b00000100 #b0)    ; third bit for flipping y axis
            (if rotate? #b00001000 #b0)))  ; fourth bit for rotating 90 degrees
  `(call $blit ,sprite-ptr
         ,(maybe-i32.const x) ,(maybe-i32.const y)
         ,(maybe-i32.const width) ,(maybe-i32.const height)
         ,(maybe-i32.const flags)))

Now we can update our update procedure to rotate and place the smiley multiple times:

(func (export "update")
  ,(call-blit $smiley$ 76 76 8 8)
  ,(call-blit $smiley$ 42 42 8 8
              #:flip-y? #t #:rotate? #t))

Despite now blitting the smiley twice, and rotating and flipping it on the second version, this is now dramatically easier to read than the first version.

Scheme procedures to generate data and whole programs

While we're not translating Scheme to WebAssembly directly in this particular usage of Hoot, we are using Scheme to generate WebAssembly... and that means we have the full power of Scheme at our fingertips!

This turned out to be hugely useful during the game jam. For instance, while the "smiley" sprite we showed earlier was defined in 1 bit per pixel ("1BPP") and thus could be "seen while squinting" in its binary representation:

(define smiley-data
  #vu8(#b11000011      ; 1bpp (1 bit per pixel) sprite!
       #b10000001      ; We have two colors represented by 0 and 1
       #b00100100      ; which means if you squint
       #b00100100      ; you can kinda see the smiley!
       #b00000000
       #b00100100
       #b10011001
       #b11000011))

However, for Wireworld we wanted to take advantage of WASM-4's support for a 4-color palette to draw prettier sprites. These are not readable as text in pure binary data in the same way. However, for the sake of fast iteration and easy ability to "play with" sprite appearance, we wanted to get back the defined-in-ascii-art approach.

So we did just that. Here are the head and tail sprite definitions:

(define head-text
  "\
X##X
#~.#
#~~#
X##X")

(define tail-text
  "\
XXXX
X#~X
X##X
XXXX")

Here X represents a dark blue pixel, # is dark purple, ~ is light purple, and . is off-white. These sprites are compiled into their binary representation from within Scheme itself:

scheme@(guile-user)> (text->2bpp-sprite-bv head-text)
$7 = #vu8(235 146 150 235 0)

We can then directly insert the binary representation into the assembly of the program! (Understanding how text->2bpp-sprite-bv works is left as an exercise for the reader.)

When constructing this blogpost, we also wanted a way to generate multiple Wireworld WASM-4 carts which each started with an initial world state. Once again, we wanted to define these in plaintext for fast iteration and experimentation while preserving readability:

  *#                                #@
 @  ################################  *
  #                                  #
 #                                    #
  #                                  #
 #                                    #
  ######     ####### ####   ####     #
 #     #     #   #   #   # #          #
  #    #  #  #   #   ####   ###      #
 #     #  #  #   #   #  #  #          #
  #    #  #  #   #   #   # #         #
 #      ## ##  #######   ## ####      #
  #                             ######
 #                                    #
  #                                  #
 #                                    #
  #  #     #   ##   ####  #    ###   #
 #   #     #  #  #  #   # #    #  #   #
  #  #  #  # *    @ ####  #    #  #  #
 #   #  #  # @    * #  #  #    #  #   #
  #  #  #  #  #  #  #   # #    #  #  #
 #    ## ##    ##   #   # #### ###    #
  #                                  #
 #                                    #
  #                                  #
 #                                    #
  #                                  #
 *  ################################  @
  @#                                #*

Writing a text parser interpreted by a WASM program for reading textual wireworld descriptions would have been too much work (both for us and for the spirit of an old-school fantasy console). We took the same technique as with the textual representation of sprites: our Scheme program loaded and translated worlds from the textual representation to the very in-memory representation our WASM-4 game would use, then thanks to the power of quasiquote, we simply inserted the game into the generated program.

The really cool thing here is that the generation of carts is itself a procedure. Generating a custom cart is as simple as:

(wat->wasm (make-wireworld-game
            #:world (load-world-file world-filename)))

Thus our make file could simply spit out custom files with custom initial world states, "baking" the initial level descriptions into memory:

$ make
Built build/wireworld-adder.wasm
Built build/wireworld-splash.wasm
Built build/wireworld-intro.wasm
Built build/wireworld-blank.wasm

If you've ever heard lisp programmers talk about "programs that write programs", consider this a nice example!

Lessons learned and meta-observations

Our biggest successes were when we began embracing the abstraction powers provided by Hoot. Initially we simply hand-coded using WebAssembly's textual format and compiled such files. Once we moved to the scheme-as-code-generator abstractions shown in this article there was a marked uptick in productivity and correctness. Code became easier to write and understand and iteration became faster. Tools such as the textual representations of sprites and levels became easy to directly integrate. And significantly, by embracing the "programs that write programs" philosophy, generating the custom carts shown off in this article became as simple as passing in the relevant arguments to the procedure which "baked" the carts with the relevant levels. This latter part was particularly satisfying but was simply a natural outgrowth of the style of programming we took.

Hoot's primary goal is to get Spritely's tooling available in the browser by directly compiling Scheme to WebAssembly. However it turns out that Hoot's lower level layers of abstractions are powerful tooling in their own right! Game jams are a great opportunity to put your tooling to the test, and we're delighted with the outcome.

Lisp Game Jam - "Fantasary" - prototyping an object world (& TCP netlayer)

2023-06-07T16:30:00Z

Spritely participated in the Spring Lisp Game Jam 2023 last week. We submitted not one, but two, entries. This post is about one of them: Fantasary.

Fantasary is a prototype textual virtual world with interactive objects. Users can move between multiple virtual rooms and interact with the objects (other humans and bots) in those rooms. We used an ncurses-based user interface and cool-retro-term to make it look pretty. The result resembles an IRC chat application but under the hood it is built using object capabilities. Building this prototype under the time constraints of a game jam served multiple purposes:

Testing out a new, prototype TCP+TLS netlayer for OCapN.
Experimenting with petnames.
Experimenting with the unum pattern to model objects in a distributed virtual world.
Getting more experience building and releasing applications with Goblins AKA dogfooding.

A new, prototype netlayer

Test driving a new OCapN netlayer was the primary motivation for working on Fantasary. As of today, Goblins ships with a Tor Onion services netlayer. This netlayer has its uses and was relatively simple to implement, but there are significant downsides:

It requires running a Tor daemon configured in a particular way.
Connecting to a Tor onion service for the first time tends to be slow.
There's a lot of lag due to the overhead inherent in onion routing.

It's great to have Tor as an available option, but we'd also like to provide something simpler and faster. To that end, we've written a prototype netlayer, internally called the "Simple TCP" netlayer, that operates over standard TCP sockets and uses TLS to encrypt the data that passes through. The "simple" part refers to using widespread networking protocols to achieve peer-to-peer connections to the extent possible, without being part of a true peer-to-peer network such as one built with libp2p. This prototype seems to be working pretty well so far. The 3-day game jam rating period is still ongoing at the time of writing and people who have never used Goblins before are able to connect to our Fantasary server and play around, and our server hasn't crashed. That said, there are several caveats to using this netlayer at the moment, and we had to build Fantasary carefully to avoid those shortcomings.

For starters, most people who try Fantasary will be behind a NAT gateway and a firewall on their home network, which makes establishing peer-to-peer connections very difficult. We did not solve this problem before writing Fantasary, so we made sure that all network communication was between a client and our public server, never peer-to-peer. In OCapN terms, this meant no third-party handoffs of object references. Looking ahead, we are exploring using public STUN servers to do the necessary NAT hole punching so that clients can be connected directly, much like WebRTC-based chat systems (among others.)

Another major problem is bootstrapping clients onto OCapN. In order to connect to an application using OCapN from the outside world, you need a sturdyref URI. At the moment, we are doing the simplest possible thing and using an encoded form of the X.509 certificate as the host identifier, and rely upon hints to point a client towards the machine that owns that certificate. Here's an abridged example:

ocapn://<encoded-cert>.simple-tcp/s/<swiss-num>?host=example.com&port=8888

It would be unreasonable to print a real world example is because the encoded cert is a huge number of characters. The sturdyref shipped in Fantasary is 4124 characters long! This makes the URIs difficult to share and thus poses a usability problem. We are looking to fix this by using a hash of the certificate as the host identifier and introducing a step prior to the TLS handshake where the server sends over the certificate and the client confirms that it hashes to the expected value.

As an aside, the URI above may seem inverted from typical URIs you are used to because the host name appears in the query string. In OCapN, the essential identifier of a machine is its public key, not its name or location. We reserve the query string as a place for "hints" that can help clients establish a connection to the machine with the given identity. Here we are using a hint that relies upon DNS, but there could be other ways to point a client towards the machine's location.

The last major problem is the X.509 certificate system being a poor fit for the distributed world we are building. We are using TLS because of its ubiquity, not because we love it. X.509 certificates include all sorts of data that is for the centralized certificate authority system. All we care about in Goblins is that we have a public/private key pair that we can use to establish an encrypted channel. In fact, the netlayer does not rely on CA certificate chains (the stuff you'd typically find in /etc/ssl) at all. Because the OCapN URI has the certificate for the remote machine encoded in it, the associated TLS session inserts that certificate into its trust store and nothing else. Also, we didn't want to make users figure out how to generate their own certificates as that would be a major barrier to entry. Instead, we shelled out to OpenSSL to automatically generate one, filling in all text fields with hardcoded values to satisfy the generator. Moving forward, we'd like to preserve automatic certificate generation but want to generate them through library calls within the program rather than spawning a subprocess.

Petnames

A distributed world doesn't rely upon centralized naming authoritities like domain name registrars, so we pushed ourselves to keep the server code free of name registration behavior. Instead, users say what their self-proclaimed name is, and other users are free to replace that name with a "petname", a name local to their client that only they will see. It works much like how a contact list on a cell phone associates local names to phone numbers. To distinguish self-proclaimed names from petnames, we introduced a little bit of new syntax into the client interface. A self-proclaimed name appears prefixed with ? like ?Alice. A local petname appears prefixed with @ like @AuntAlice. Because there's no central registry of names, self-proclaimed name collisions are both possible and not a big deal. When clients see many objects with the same self-proclaimed name, they add an integer prefix to distinguish them. If there were three users named Goblin in the room, they would show up as ?Goblin, ?Goblin+1, and ?Goblin+2.

Fun fact: Fantasary started out with centralized names, and when we decided to expand scope to incorporate petnames it actually simplified the code for the objects on the server because they no longer needed to care about names at all, only object identity.

If you'd like to read more about petnames, check out our blog post from last year about two papers we wrote on the subject.

The unum pattern

The unum pattern, diagramed in the E literature and explained in greater detail by Chip Morningstar, is a way of representing an object in a virtual world that has a presence across many machines. The owner of an object views/interacts with that object in one way, a server may have yet another view, the users in the same room may have yet another view, etc., yet they all refer to the same conceptual "thing." For Fantasary, we tried to understand and implement a simple version of this pattern.

Our game objects, the things that can exist inside a room, are called "entities." The owner of an entity has the capability to do whatever it would like. Fantasary is very simple, and the only things to be done are setting a name and an event handler. When an entity enters a room, the room is given a "puppet" of that entity. A puppet is a proxy for the entity with limited capabilities. The room has read-only access to the entity's identity and can invoke the event handler to notify the entity of activity within the room. When an entity leaves a room, it cuts the metaphorical puppet strings (represented as dashed lines in the diagram above) to revoke the capability for the room to send messages to it. Other entities in the room see puppets of their roommates, as well, but puppets of a different sort. These puppets are constructed by the room specifically for use by a single roommate. If either entity leaves the room, the puppet strings are cut and they can no longer talk via that puppet. This proxying strategy provides both security (an entity never gets a direct reference to another entity, nor does a room) and a means to layer on context-specific behavior. This initial experiment with the unum pattern suggests to us that it's a pattern worth exploring more in future projects.

Packaging

In order to make Fantasary something that people could actually run without too much time investment, we utilized GNU Guix. We use Guix internally for development and server administration, but this was our first time using it to ship software. We used the guix pack program to produce a redistributable binary bundle that contains all of the dependencies required to run Fantasary, all the way down to glibc. There a couple of caveats:

The bundle is big. Nearly 1G uncompressed, ~250M gzipped.
The bundle works on Linux only, but that's OK for now.

The trade-off for making minimal assumptions about the Linux host running the game is that the bundle is quite large. It could be made smaller, as there are things included that are not needed at runtime, but that requires optimizing Guix package builds upstream. From what we've seen, the bundle has worked for most people that have tried Fantasary, and some of the failures are due to assumptions made in our own code (I didn't expect (getprotobyname "tcp") to fail on some systems) and are not the fault of Guix.

Guix also made it to easy to write a wrapper script that automatically launches Fantasary in cool-retro-term for that extra coolness factor. This fancy version doesn't work for all users, however, so we also included a fallback script that simply runs in the user's regular terminal.

Conclusion

Participating in the Lisp Game Jam gave us a great opportunity to dedicate a fixed amount of time to develop a prototype that will be important to us moving forward. Fantasary is not without its share of bugs and missing features (like cleaning up users that have disconnected without sending a quit message), but that's part of what a game jam is all about: Ruthlessly reducing scope to just the things that matter the most and trying to sprint to the finish line with something that mostly works. We look forward to shipping the Simple TCP netlayer in a future release of Goblins.

Directly compiling Scheme to WebAssembly: lambdas, recursion, iteration!

2023-05-30T14:00:00Z

It's been just over three months since we announced the Guile on WebAssembly project (codenamed Hoot). Since then we've brought on two fantastic hackers to develop the project and progress has been quick.

We now are at the point where we have things to show: we can now compile various Scheme procedures directly to WebAssembly. Let's clarify that: by compiling directly to WebAssembly, we mean they compile and run without any intermediate virtual machine. No C, no Rust, no Zig: Hoot outputs pure WebAssembly. It's early, but Hoot compiled programs are starting to work... and they're fast!

Hoot by familiar Scheme examples

The Hoot test suite includes some examples of fairly standard scheme procedures, compiled directly to WebAssembly and executed against V8. Let's take a look at some of those examples to contextualize what Hoot provides. (If you're not familiar with Scheme, you can focus on the text of this blogpost rather than the code. Or, you can read our Scheme Primer as a way to get started on following along!)

Here's an all time favorite intro-to-Scheme example (one of the first examples in SICP):

;; Your intro-to-CS factorial function
(test-call "120"                         ; expected value
           (lambda (n)                   ; factorial function which takes `n`
             (let fac ((n n))            ; start recursing at `n`
               (if (eq? n 0)             ; are we at zero?
                   1                     ; if so, return 1, otherwise...
                   (* n (fac (1- n)))))) ; `n` times factorial of `n` minus one
           5)                            ; run `factorial(5)`

test-call takes two or more arguments: the value we expect to see V8 spit out from running the compiled WebAssembly code, a procedure we want to test, and any more arguments are arguments to the function.

Hoot is compiling this program to WebAssembly directly. Astoundingly, at the time of writing, the size of the compiled WebAssembly program is only 970 bytes... less than a kilobyte! (Did you gasp?)

Other familiar intro-to-computer-science friends are also now available. Who doesn't love the Fibonacci function?

;; Fibonacci, everyone's favorite
(test-call "9227465"                     ; expected value
           (lambda (n)                   ; fibonacci function which takes `n`
             (let fib ((n n))            ; start recursing at `n`
               (if (<= n 1)              ; is `n` 1 or less?
                   1                     ; if so, return 1, otherwise...
                   (+ (fib (- n 1))      ; add `fib(n - 1)` to
                      (fib (- n 2))))))  ;     `fib(n - 2)`
           34)                           ; run `fibonacci(34)`

Now the astute reader may be looking at the above two functions and observing that these are recursive procedures! Yes, recursion is all fine and well in the land of Scheme, and it's all fine and well in the land of Hoot, too.

Of course, the astute reader will also recognize that neither of these is the most efficient way to implement either of these functions: both grow more memory than necessary. It's great that we can recurse, but should we recurse?

Well, it turns out that iteration can be defined in terms of recursion. If a procedure doesn't need to do any more work, there's no need to keep it on the stack anymore. And if a procedure doesn't need to do any more work and it's just calling itself, Scheme implementations are historically especially smart about this, because we can just reuse the same procedure where it was!

Which means, here's a very terse looking loop which looks kinda recursive but in terms of computational work is merely iterative:

;; Let's get loopin!
(test-call "500000000"                   ; expected value
           (lambda ()
             (let loop ((n 0))           ; start loop at 0
               (if (< n 500000000)       ; if we're less than 500 million...
                   (loop (1+ n))         ; keep looping and increment n
                   n))))                 ; otherwise we're done, return n

Once again, this program is less than 1 kilobyte when compiled, and carries with it everything a Hoot-compiled program needs to execute. That's cheap in terms of file size. But how fast is it?

Well consider that Guile is actually fairly speedy. It's not the fastest Scheme out there (their role in programming language research means that Schemes are a well trodden territory for optimizing languages), but it's quite fast. Shockingly then, the Hoot-compiled WebAssembly for the above executes, on local tests, is five times faster than native Guile!

Of course, looping over addition is a very limited benchmark. Still, we're happy for this early indicator that Scheme to WebAssembly is likely to be very performant.

Hoot is also a mini WebAssembly toolkit!

One of the interesting aspects of Hoot is that it takes advantage of Guile's compiler tower. Hoot is able to use all the optimizations that Guile already provides, because Hoot's compiler transforms Scheme code from Guile's CPS Soup intermediate language directly to WebAssembly.

(Guile's compiler tower has some limited support for other toplevel languages which are not Scheme; this also means that those too can compile directly to WebAssembly.)

One nice aspect of WebAssembly is that its textual representation is already very lispy, using parenthetical s-expression notation. This means that WebAssembly is already ripe territory for lispers to embrace, and Hoot has done just that.

Rather than use external tooling like binaryen or wabt, Hoot provides its own suite of tools for parsing, assembling, disassembling, and dumping WebAssembly directly. This means Hoot isn't just useful as a way of compiling Scheme to WebAssembly, it's a useful WebAssembly toolkit in and of itself!

What Hoot means for other languages

For a long time, programmers were given one real option to program in web browsers: Javascript. WebAssembly's initial introduction has provided a ripe ground for several other non-garbage-collected languages to enter the space, such as C, C++, Rust, and Zig.

But there is a large and beautiful array of programming languages available to programmers in the world. Many of those languages have been given less than ideal paths to participating on the web. Compiling an entire language environment or virtual machine to WebAssembly is perhaps better than transpiling to Javascript, but it is still an expensive operation.

This blogpost has primarily been centered around Scheme (Spritely's core tooling is written in Guile Scheme, and Hoot is a project to compile Scheme to WebAssembly, so this isn't too surprising). But the Spritely Institute is just that, a research institution. Hoot is breaking ground in some new spaces in WebAssembly land (and we are participating in the WebAssembly standards processes accordingly).

The web should be for everyone. We hope Hoot paves a path so that more languages may enter the web, including Python, Lua, Haskell, ML, and friends.

What Hoot means for Spritely

We showed some exciting demos in this blogpost, but it remains true that Hoot is still early. Some Scheme simple procedures can compile, but we have a ways to go. But we are on track, and the future looks bright. Hoot has been made possible due to a generous grant from Consensys/Metamask. This grant was given was a knowingly ambitious project; we are grateful for their support and are excited to be delivering towards those goals.

On the immediate roadmap is the compilation of the Scheme standard r7rs-small, with the goal to get Hoot compiled programs to appear on the r7rs benchmarks page. After that, it's onto getting Goblins itself to run in the browser and in the (increasingly) many places WebAssembly is supported. We are excited to have an optimistic path to get Spritely's tooling in the hands of as many users as possible.

We look forward to delivering more updates about Hoot as the project progresses. The future of WebAssembly (and Spritely) is bright!

Spritely Goblins v0.11.0 released: time travel distributed debugging and more!

2023-05-08T08:00:00Z

We are delighted to announce the v0.11.0 release of our distributed programming environment, Spritely Goblins!

Highlights from this release:

A new time-traveling distributed debugger
Substantially improved documentation and docstrings
Several more actor-lib modules ported from Racket to Guile
Updates to OCapN code

Goblins is Spritely's distributed object programming environment, providing an intuitive security model, automatic local transactions for synchronous programming, and asynchronous programming allowing collaboration over the network with the same level of ease as local asynchronous programming. Goblins takes the pain out of reasoning about distributed and secure peer-to-peer programming: with Goblins, that's the default mode of operation!

(Releases of Goblins v0.11.0 are available for both Guile and Racket alike, but new features are aimed at the Guile version of Goblins. More about the Racket version at the end!)

A time traveling distributed debugger for Goblins!

Goblins now includes a time traveling distributed debugger! Yeah, you heard that right!

The new Goblins debugger allows programmers to inspect and debug distributed computations that happen across many vats (communicating event loops.) The time travel feature allows for visiting past events and inspecting program state at the time the event happened.

These tools are implemented as “meta-commands” for Guile's REPL, complementing the tools that Guile already provides for sequential debugging. In true Scheme fashion, this means that debugging happens live, while the program is running, allowing programmers to fix bugs and try again without having to stop and recompile/reboot their program.

Documentation overhaul

Documentation has received a major overhaul in this release:

All key procedures now have docstrings, allowing for easier discovery of API usage during development
The manual's tutorial has been cleaned up for clarity and correctness
All core API procedures are now documented in the manual
All modules in the actor-lib are now documented in the manual
The manual now has an index, so finding procedures is much easier

OCapN / CapTP updates

Work has continued on Goblins' implementation of OCapN, the Object Capability Network, the set of network abstractions which provide the fluid experience of distributed networked programming.

Goblins' network architecture is still in active development; this release breaks compatibility with the 0.10 release of Goblins.

Semantics closer to CapTP draft specification

Goblins' implementation of the OCapN version of CapTP (the Capability Transfer Protocol) is aimed to be used as the starting point for revisions for the OCapN group. We have submitted a draft version of the protocol to the OCapN group. Goblins 0.11.0 gets us closer to the aims of that draft; we are hoping within the next few releases to get full alignment and group consensus towards a unified OCapN protocol, and the 0.11.0-goblins release of OCapN and CapTP shipped with Goblins gets us closer to that goal.

Add acyclic distributed GC support to CapTP

Goblins 0.11.0 now ships support for acyclic distributed garbage collection, meaning that references to cooperative freeing of objects shared over the network but no longer needed is possible

More actor-lib modules ported from Racket to Guile

More of Goblins' "standard library" is now available in the Guile version of Goblins, ported from Racket:

(goblins actor-lib pushdown) is a simple pushdown automata implementation, useful for various kinds of state machines, especially in some games
(goblins actor-lib opportunistic) allows for proxying object behavior, opportunistically using $ for near objects but using <- otherwise (known as select-swear in the Racket version)
(goblins actor-lib simple-mint) is a very simple example "bank" of sorts, ported from the example in An Ode to the Granovetter Diagram
(goblins actor-lib let-on) provides convenient code for promise resolution which strongly resembles let in Scheme, but with the behavior of on in Goblins

Goblins, Racket edition v0.11.0

The Guile version of Goblins is now the primary version of Goblins, but we continue to support the Racket version of Goblins. In particular, the Racket version of Goblins has also received a v0.11.0 release, bringing its network code up to the same state as the Guile version and maintaining compatibility.

Onward and upward!

And that's not all! For more information on Goblins v0.11.0, read the NEWS file!

This release is brought to you by the following contributors: David Thompson, Juliana Sims, Christine Lemmer-Webber, Jessica Tallon, Vivianne Langdon, and Geoffrey J. Teale. Thank you for your work!

We hope you enjoy this release of Spritely Goblins! And if you're building something cool with Goblins, let us know on our community forum. (Use OCAPN2023 for the Invite Code when you join.)

Introducing: A distributed debugger for Goblins with time travel!

2023-04-12T12:00:00Z

Debugging distributed programs is difficult, but we’ve been hard at work developing tools to make it easier within the Goblins object capability system. While this work is still in its early stages, we’re excited to share the progress we’ve made so far!

These tools are implemented as “meta-commands” for Guile's REPL, complementing the tools that Guile already provides for sequential debugging. In true Scheme fashion, this means that debugging happens live, while the program is running, allowing programmers to fix bugs and try again without having to stop and recompile/reboot their program.

Seeing is believing, so let’s walk through building a simple program and using the new debugger to find and fix a small bug. We’re going to make the world’s simplest dungeon crawling RPG. It will allow a player to attack a monster to deal damage, and that’s about it.

The first step is to start up a Guile REPL and import the Goblins modules that we need:

scheme@(guile-user)> ,use (goblins)
scheme@(guile-user)> ,use (goblins actor-lib cell)
scheme@(guile-user)> ,use (goblins actor-lib let-on)
scheme@(guile-user)> ,use (goblins actor-lib methods)

The (goblins) module provides the core Goblins API. The other three modules are part of Goblins’ handy standard library of actor abstractions. In Guile's REPL, expressions that start with a comma invoke a meta-command. Guile provides a number of these commands, like ,use. We’ll be seeing commands specific to Goblins later.

Next, we need to create some vats where our actors can live. Since there is overhead in collecting debug data, vat logging is currently opt-in on a per-vat basis. The #:log? #t keyword argument enables logging.

scheme@(guile-user)> (define p-vat (spawn-vat #:name 'Player  #:log? #t))
scheme@(guile-user)> (define m-vat (spawn-vat #:name 'Monster #:log? #t))
scheme@(guile-user)> (define c-vat (spawn-vat #:name 'Cave    #:log? #t))

Now we can define some actors to live in our vats. To do that, we first need to define constructor procedures. We only need two kinds of actors for this demo: Characters and dungeons. Characters have a name, some amount of hit points (HP), and a strength value. They can be damaged by attacks and die when their HP reaches 0.

scheme@(guile-user)>
(define (^character _bcom name max-hp strength)
  ;; HP is reduced when the character is attacked, so we use a cell
  ;; actor to store a counter that can be modified.
  (let ((hp (spawn ^cell max-hp)))
    (methods
     ;; Getters
     ((name) name)
     ((strength) strength)
     ((hp) ($ hp))
     ;; Deal combat damage
     ((damage amount)
      ($ hp (- ($ hp) amount))
      #t) ; attacks always hit
     ;; Check if character is dead
     ((dead?)
      (zero? ($ hp))))))

Dungeons are places where a player character fights a monster character. For simplicity, this is going to be a rather one-sided battle and we’ll only implement the player’s turn. A player turn starts by dealing damage to the monster and ends by returning a log summarizing what happened. If the monster is killed, the log will say so.

scheme@(guile-user)>
(define (^dungeon _bcom player monster)
  (methods
   ((player-turn)
    (let-on ((player-name (<- player 'name))
             (monster-name (<- monster 'name))
             (strength (<- player 'strength)))
      ;; Deal damage to the monster and check to see if it's dead.
      (let*-on ((hit? (<- monster 'damage strength))
                (dead? (<- monster 'dead?)))
        (let ((damage-msg (format #f "~a deals ~a damage to ~a!"
                                  player-name strength monster-name)))
          ;; Log a message if the monster is dead.
          (if dead?
              (let ((dead-msg (format #f "~a has been defeated!"
                                      monster-name)))
                (list damage-msg dead-msg))
              (list damage-msg))))))))

To show off the “distributed” part of “distributed debugger”, we’ll spawn actors across all three of the vats we defined earlier. The player character “Alice” will live in the Player vat, p-vat. The monster, a basilisk, will live in the Monster vat, m-vat. The dungeon will live in the Cave vat, c-vat. This simulates how, in a real game built using a distributed architecture, different actors in the game world would be hosted on many different computers.

scheme@(guile-user)> (define alice
                       (with-vat p-vat (spawn ^character "Alice" 100 30)))
scheme@(guile-user)> (define basilisk
                       (with-vat m-vat (spawn ^character "Basilisk" 20 15)))
scheme@(guile-user)> (define dungeon
                       (with-vat c-vat (spawn ^dungeon alice basilisk)))

With our actors spawned, we are ready to play this trivial game. But first, we’ll use a Goblins-specific REPL command to enter a new context for evaluating code:

scheme@(guile-user)> ,enter-vat c-vat
Entering vat 'Cave'.  Type ',q' to exit.  Type ',help goblins' for help.
goblins/Cave@(guile-user) [1]>

Note the change in the REPL prompt to indicate that we are now in the Goblins vat named “Cave” and we are in a nested REPL session that is one level deep. The ,enter-vat command is provided by Goblins to allow code (and other Goblins REPL commands) to be evaluated within the context of a specific vat. This makes Goblins operators (such as spawn, $, and <-) “just work” since they require a vat in order to do their job.

We're now ready to have Alice take a turn. Alice will attack the basilisk, dealing 30 damage. The basilisk only has 20 HP, so it should die and we should see a log message saying so. Getting a result requires communicating with the Player and Monster vats, so we’ll use the the ,vat-resolve command to wait for the asynchronous operation to complete and print out the results:

goblins/Cave@(guile-user) [1]> ,vat-resolve ($ dungeon 'player-turn)
("Alice deals 30 damage to Basilisk!")

Hmm, that’s not right! The log says that Alice dealt 30 damage, but it should also say that the basilisk was defeated. Let’s debug and see if we can spot the error. For starters, we can run ,vat-tail to see the most recent messages that the Cave vat has processed:

goblins/Cave@(guile-user) [1]> ,vat-tail
Churn 7:
  36: (receive listen #<local-promise>)
Churn 8:
  37: (receive message #<local-object ^resolver> fulfill #f)
  38: (receive message #<local-object ^on-listener> fulfill #f)
  39: (receive message #<local-object fulfilled-handler> #f)
  40: (receive message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
  41: (receive message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
  42: (receive message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
  43: (receive message #<local-object ^on-listener> fulfill ("Alice deals 30 damage to Basilisk!"))
  44: (receive message #<local-object fulfilled-handler> ("Alice deals 30 damage to Basilisk!"))
  45: (receive message #<local-object finally-handler>)

The events have been printed in chronological order. There is some useful information in here, but only events in the Cave vat are shown and there’s nothing that describes how these events relate to one another. However, we now know that event 45 is the most recent. This event represents the terminal point of the ,vat-resolve command we evaluated above.

To view the events related to another event, across all relevant vats, we will use the ,vat-tree and ,vat-graph commands. ,vat-tree outputs a textual representation:

goblins/Cave@(guile-user) [1]> ,vat-tree 45
Vat Cave, 2: (message #<local-object ^call-with-vat>)
├▸ Vat Player, 4: (message #<local-object ^character> name)
│  └▸ Vat Cave, 10: (message #<local-object ^resolver> fulfill "Alice")
├▸ Vat Monster, 5: (message #<local-object ^character> name)
│  └▸ Vat Cave, 16: (message #<local-object ^resolver> fulfill "Basilisk")
│     └▸ Vat Cave, 19: (message #<local-object ^resolver> fulfill ("Alice" "Basilisk" 30))
│        └▸ Vat Monster, 23: (message #<local-object ^character> damage 30)
│           └▸ Vat Cave, 28: (message #<local-object ^resolver> fulfill #t)
│              └▸ Vat Monster, 32: (message #<local-object ^character> dead?)
│                 └▸ Vat Cave, 37: (message #<local-object ^resolver> fulfill #f)
│                    └▸ Vat Cave, 40: (message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
│                       └▸ Vat Cave, 41: (message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
└▸ Vat Player, 6: (message #<local-object ^character> strength)
   └▸ Vat Cave, 13: (message #<local-object ^resolver> fulfill 30)

And ,vat-graph outputs a graphical Lamport causality diagram (thanks to Graphviz):

goblins/Cave@(guile-user) [1]> ,vat-graph 45

In each of these views, we can see that the message (<- monster 'dead?) (with the debug representation of msg ^character dead? in the ,vat-graph output) resolves to #f when we expected #t. Did you miss it? Here’s the relevant bit of the ,vat-tree output above:

│              └▸ Vat Monster, 32: (message #<local-object ^character> dead?)
│                 └▸ Vat Cave, 37: (message #<local-object ^resolver> fulfill #f)

But why did this happen? The monster should have been dead. Let’s travel back in time and find out. The first step is to truly enter the debugger so we can walk backwards in time to when the dead? check happened. To do this, we’ll use the ,vat-debug command, which takes an event ID argument and starts a new sub-REPL with all the context we need to debug the problem.

goblins/Cave@(guile-user) [1]> ,vat-debug 45
Entering a new prompt. Type `,q' to exit.
goblins/Cave@(guile-user) [2]>

Note that the [1] in the prompt became a [2] because we are in yet another sub-REPL. To get our bearings, we’ll use the ,vat-trace command to get a distributed backtrace showing the direct causes of event 45:

goblins/Cave@(guile-user) [2]> ,vat-trace
In vat Cave:
  Churn 2:
    2: (message #<local-object ^call-with-vat>)
    4: (message #<local-object ^character> name)
In vat Monster:
  Churn 2:
    6: (message #<local-object ^resolver> fulfill "Basilisk")
In vat Cave:
  Churn 5:
    17: (message #<local-object ^on-listener> fulfill "Basilisk")
    18: (message #<local-object fulfilled-handler> "Basilisk")
    19: (message #<local-object ^resolver> fulfill ("Alice" "Basilisk" 30))
    20: (message #<local-object ^on-listener> fulfill ("Alice" "Basilisk" 30))
    21: (message #<local-object fulfilled-handler> ("Alice" "Basilisk" 30))
    22: (message #<local-object ^character> damage 30)
In vat Monster:
  Churn 3:
    24: (message #<local-object ^resolver> fulfill #t)
In vat Cave:
  Churn 7:
    29: (message #<local-object ^on-listener> fulfill #t)
    30: (message #<local-object fulfilled-handler> #t)
    31: (message #<local-object ^character> dead?)
In vat Monster:
  Churn 4:
    33: (message #<local-object ^resolver> fulfill #f)
In vat Cave:
  Churn 8:
    38: (message #<local-object ^on-listener> fulfill #f)
    39: (message #<local-object fulfilled-handler> #f)
    40: (message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
    41: (message #<local-object ^resolver> fulfill ("Alice deals 30 damage to Basilisk!"))
    43: (message #<local-object ^on-listener> fulfill ("Alice deals 30 damage to Basilisk!"))
    45: (message #<local-object finally-handler>)

Much like a regular backtrace in Guile’s sequential debugger, output is in order from outermost call to innermost. In other words, the most recent event is the last event printed. Our debugging session starts focused on event 45, but we can shift our focus with the ,vat-up, ,vat-down, and ,vat-jump commands.

,vat-up moves up one event, from 45 to 43 in this case:

goblins/Cave@(guile-user) [2]> ,vat-up
Now in vat Cave, event 43:
  (receive message #<local-object ^on-listener> fulfill ("Alice deals 30 damage to Basilisk!"))

,vat-down moves down one event, back to event 45 in this case:

goblins/Cave@(guile-user) [2]> ,vat-down
Now in vat Cave, event 45:
  (receive message #<local-object finally-handler>)

,vat-jump moves to a particular event ID, so we can save ourselves the tedium of typing ,vat-up or ,vat-down a bunch of times if the event we're trying to inspect is far away from our current position. In fact, event 31 looks particularly interesting, since that’s when the dead? check happened. Let's jump there now:

goblins/Cave@(guile-user) [2]> ,vat-jump 31
Now in vat Cave, event 31:
  (send message #<local-object ^character> dead?)

What ,vat-trace is hiding, for brevity’s sake, is that this “send” event in the Cave vat has a corresponding “receive” event in the Monster vat. Let’s step down the trace and see where we end up:

goblins/Cave@(guile-user) [2]> ,vat-down
Now in vat Monster, event 32:
  (receive message #<local-object ^character> dead?)

We’re now focused on an event in vat Monster, where the basilisk lives. This means that we can travel back in time and send messages to the basilisk as it existed at event 32. To do this, we'll use the ,vat-peek command. Let's see what its HP looked like at this point:

goblins/Cave@(guile-user) [2]> ,vat-peek basilisk 'hp
-10

Hmm, zero should be the lowest HP value. The dead? method is specifically checking if the character’s HP is zero. It’s a safe bet that the bug is in the damage method, which happened around event 22, so let’s confirm our suspicion. We’ll jump there, move down into an event in the Monster vat, and check out the basilisk's HP:

goblins/Cave@(guile-user) [2]> ,vat-jump 22
Now in vat Cave, event 22:
  (send message #<local-object ^character> damage 30)
goblins/Cave@(guile-user) [2]> ,vat-down
Now in vat Monster, event 23:
  (receive message #<local-object ^character> damage 30)
goblins/Cave@(guile-user) [2]> ,vat-peek basilisk 'hp
20

Thanks to time travel, we can see that the starting HP value is 20, like it should be. The basilisk’s health was indeed initialized correctly. Moving down one more event, we can see for sure that this is the point where we entered a problematic state:

goblins/Cave@(guile-user) [2]> ,vat-down
Now in vat Monster, event 24:
  (send message #<local-object ^resolver> fulfill #t)
goblins/Cave@(guile-user) [2]> ,vat-peek basilisk 'hp
-10

A note for the curious: Goblins uses transactional heaps to manage actor state, enabling us to take snapshots of a vat before each event is processed. Querying these snapshots allows users to travel through time while debugging. We’re using very simple code for demonstration purposes, but we believe that this feature will make debugging real programs, with complex logic and state, much more feasible.

So, what we’re missing is some code to enforce a lower bound on the HP value. Let’s update our character constructor to fix that issue (exiting the nested REPLs and getting back to the top-level Guile prompt):

goblins/Cave@(guile-user) [2]> ,quit
goblins/Cave@(guile-user) [1]> ,quit
scheme@(guile-user)>
(define (^character _bcom name max-hp strength)
  ;; HP is reduced when the character is attacked, so we use a cell
  ;; actor to store a counter that can be modified.
  (let ((hp (spawn ^cell max-hp)))
    (methods
     ;; Getters
     ((name) name)
     ((strength) strength)
     ((hp) ($ hp))
     ;; Deal combat damage
     ((damage amount)
      ;; HP can't go below 0.
      ($ hp (max (- ($ hp) amount) 0))
      #t) ; attacks always hit
     ;; Check if character is dead
     ((dead?)
      (zero? ($ hp))))))

A call to max has been added to the damage method to ensure that zero is the lowest HP value. All that’s left to do now is rebuild the actors and verify the fix:

scheme@(guile-user)> (define alice
                       (with-vat p-vat (spawn ^character "Alice" 100 30)))
scheme@(guile-user)> (define basilisk
                       (with-vat m-vat (spawn ^character "Basilisk" 20 15)))
scheme@(guile-user)> (define dungeon
                       (with-vat c-vat (spawn ^dungeon alice basilisk)))
scheme@(guile-user)> ,enter-vat c-vat
Entering vat 'Cave'.  Type ',q' to exit.  Type ',help goblins' for help.
goblins/Cave@(guile-user) [1]> ,vat-resolve ($ dungeon 'player-turn)
("Alice deals 30 damage to Basilisk!" "Basilisk has been defeated!")

We see the Basilisk has been defeated message now. Problem solved!

We hope this small demo has given you a taste of the exciting things to come in the next Guile Goblins release. If you want to use these tools right now, all the code used for this demo is available in the main branch of the Goblins Git repository. While it is already quite useful, right now the debugger only supports multiple vats in a single Guile process. In the near future, we plan to add network debugging via CapTP. We’d also like to take this opportunity to give credit to E’s Causeway debugger as a significant source of inspiration and prior art that was crucial for this work.

If you'd like to try this yourself, here's the cut-and-paste text you can use...

 ,use (goblins)
 ,use (goblins actor-lib cell)
 ,use (goblins actor-lib let-on)
 ,use (goblins actor-lib methods)

 (define p-vat (spawn-vat #:name 'Player  #:log? #t))
 (define m-vat (spawn-vat #:name 'Monster #:log? #t))
 (define c-vat (spawn-vat #:name 'Cave    #:log? #t))

(define (^character _bcom name max-hp strength)
  ;; HP is reduced when the character is attacked, so we use a cell
  ;; actor to store a counter that can be modified.
  (let ((hp (spawn ^cell max-hp)))
    (methods
     ;; Getters
     ((name) name)
     ((strength) strength)
     ((hp) ($ hp))
     ;; Deal combat damage
     ((damage amount)
      ($ hp (- ($ hp) amount))
      #t) ; attacks always hit
     ;; Check if character is dead
     ((dead?)
      (zero? ($ hp))))))

(define (^dungeon _bcom player monster)
  (methods
   ((player-turn)
    (let-on ((player-name (<- player 'name))
             (monster-name (<- monster 'name))
             (strength (<- player 'strength)))
      ;; Deal damage to the monster and check to see if it's dead.
      (let*-on ((hit? (<- monster 'damage strength))
                (dead? (<- monster 'dead?)))
        (let ((damage-msg (format #f "~a deals ~a damage to ~a!"
                                  player-name strength monster-name)))
          ;; Log a message if the monster is dead.
          (if dead?
              (let ((dead-msg (format #f "~a has been defeated!"
                                      monster-name)))
                (list damage-msg dead-msg))
              (list damage-msg))))))))

 (define alice    (with-vat p-vat (spawn ^character "Alice" 100 30)))
 (define basilisk (with-vat m-vat (spawn ^character "Basilisk" 20 15)))
 (define dungeon  (with-vat c-vat (spawn ^dungeon alice basilisk)))

,enter-vat c-vat

,vat-resolve ($ dungeon 'player-turn)

,vat-tail

,vat-tree 45

,vat-graph 45

,vat-debug 45

,vat-trace

,vat-up

,vat-down

,vat-jump 31

,vat-down

,vat-peek basilisk 'hp

,vat-jump 22

,vat-down

,vat-peek basilisk 'hp

,vat-down

,vat-peek basilisk 'hp

,quit
,quit
(define (^character _bcom name max-hp strength)
  ;; HP is reduced when the character is attacked, so we use a cell
  ;; actor to store a counter that can be modified.
  (let ((hp (spawn ^cell max-hp)))
    (methods
     ;; Getters
     ((name) name)
     ((strength) strength)
     ((hp) ($ hp))
     ;; Deal combat damage
     ((damage amount)
      ;; HP can't go below 0.
      ($ hp (max (- ($ hp) amount) 0))
      #t) ; attacks always hit
     ;; Check if character is dead
     ((dead?)
      (zero? ($ hp))))))


 (define alice     (with-vat p-vat (spawn ^character "Alice" 100 30)))
 (define basilisk  (with-vat m-vat (spawn ^character "Basilisk" 20 15)))
 (define dungeon   (with-vat c-vat (spawn ^dungeon alice basilisk)))

,enter-vat c-vat

,vat-resolve ($ dungeon 'player-turn)

Spritely Networked Communities Institute

LWN on Spritely

[Contract] Goblins (Scheme/Wasm) Integrations Engineer

Developer Community Forum open for everyone!

How to get help with technical issues for Spritely Institute technologies

Guile Hoot v0.3.0 released!

Highlights

R7RS-small

Non-standard interfaces

Compiler

Runtime

Toolchain

Browser compatibility

Get Hoot 0.3.0!

FOSDEM 2024

Get in touch!

Meet us at GuixDays & FOSDEM!

Let's talk about partnering

Request a f2f meeting in Brussels

Spritely Goblins v0.12.0 Released: Two new netlayers join the OCapN family and more!

Prelay Netlayer

TCP-TLS Netlayer

A new Queue actor

Improved CapTP support

Getting the releases

FOSDEM 2024

[EFF] Spritely and Veilid: Exciting Projects Building the Peer-to-Peer Web

Scheme in Scheme on Wasm in the browser

The case of the missing REPL

Scheme in Scheme

Scheme in Scheme on Wasm

Scheming ahead

Having a Hoot at the Lisp Game Jam: A Tale of Two Daves

How did you come to learn about the jam?

David Wilson

Dave Thompson

How long have you been participating in this or other jams?

David Wilson

Dave Thompson

How did you start running the jam?

Dave Thompson

Can you tell us a bit about this year's entry?

David Wilson

Dave Thompson

What inspired your game?

David Wilson

Dave Thompson

What tools did you use to make your game and why?

David Wilson

Dave Thompson

What challenges did you encounter in development?

David Wilson

Dave Thompson

What about successes or wins?

David Wilson

Dave Thompson

What do you hope to see from Hoot going forward?

David Wilson

Dave Thompson

Do you have any advice or favorite resources you'd like to share with future participants?

David Wilson

Dave Thompson

Are there any other questions you wish you'd been asked about the jam?

David Wilson

Dave Thompson

Further reading

Building interactive web pages with Guile Hoot

Hello, world!

HTML as Scheme data

HTML templating with Scheme

Building a virtual DOM

To-do app

Looking forward

Guile Hoot v0.2.0 released!

Highlights

R7RS-small

Non-standard interfaces

Toolchain

Browser compatibility

Get Hoot 0.2.0!