Mandy: ActivityPub on Goblins

ActivityPub is the protocol that powers the Fediverse. Not only does it allow different instances of the same app to federate with one another, it also allows different apps to federate. For example, a post on a video hosting app could federate to a microblogging app. ActivityPub does a good job of this and is a leap forward from what came before it.

For those unfamiliar, ActivityPub is a decentralized social networking protocol standardized under the W3C. Both Spritely’s Executive Director, Christine Lemmer-Webber and myself (Jessica Tallon) worked on standardizing ActivityPub. The ActivityPub specification left holes in for identity, distributed storage, and more. Since then Spritely has been a continuation of this work, researching and developing the next generation of social web infrastructure.

But where does this leave ActivityPub? Has it been abandoned by Spritely as a stepping stone? No! We’ve long had a project (codenamed Mandy) on our roadmap to implement ActivityPub on top of Goblins. If you open up the ActivityPub specification you’ll actually see mention of actors. The protocol itself is designed with the actor model in mind. Since Goblins is an implementation of the actor model, they should be a natural fit.

Goblins actors over HTTP

ActivityPub is a protocol on top of HTTP, but Goblins doesn’t use HTTP. So, the first step was to make Goblins actors available over HTTP. Fortunately, hooking this up was quite easy. There are many different ways we could do this, but for this prototype I took a fairly simple approach.

Guile has a built in web server. not only that but fibers (the concurrency system Goblins uses) has a backend for this. It means we can pretty quickly start handling requests.

The webserver can be started using the run-server procedure. It takes in a symbol which specifies an implementation ('http would be the built in HTTP server, 'fibers is the one provided by Fibers):

(run-server handler 'fibers)

The handler is a procedure which takes a request and a body and expects the HTTP response as the return value. When writing a typical HTTP server in Fibers we’d suspend the fiber until the response is ready. However, Goblins code is built around sending messages and waiting for promises to resolve. To bridge the API between these two different philosophies, we use a channel.

Channels allow two fibers to send a message to one another. Reading or writing to a channel causes the fiber to suspend until a message is available. We can then send a message to one of our actors and use on to listen to the response, once we have the response we can write our response to our channel.

Goblins vats are event loops which run on a fiber. These event loops manage a queue of messages sent to actors spawned within that vat and are processed sequentially. If we were to just write to the channel, we would suspend underlying fiber for the vat. When the vat’s fiber suspends, it stops it from processing other messages within the queue. To ensure that we’re not blocking the vat by suspending it, we’ll use the helper procedure syscaller-free-fiber which gives us a new fiber outside the vat which can be safely suspended.

(define vat (spawn-vat))
(define (^web-server bcom router)
  (define (handler . args)
    (define response-ch (make-channel))
    (with-vat vat
      (on (apply <- router args)
          (lambda (response)
            (syscaller-free-fiber
             (lambda ()
               (put-message response-ch (vector 'ok response))))
            *unspecified*)
          #:catch
          (lambda (err)
            (syscaller-free-fiber
             (lambda ()
               (put-message response-ch (vector 'err err))))
            *unspecified*)))
    (match (get-message response-ch)
      [#(ok (content-type response))
       (values `((content-type . (,content-type))) response)]
      [#(ok (content-type response) headers)
       (values `((content-type . ,content-type) ,@headers) response)]
      [#(ok response)
       (values '((content-type . (text/plain))) response)]
      [#(err err) (error "Oh no!")]))
  (syscaller-free-fiber
   (lambda ()
     (run-server handler 'fibers `(#:addr ,(inet-pton AF_INET "0.0.0.0")))))
  (lambda () 'running))

(define web-server (with-vat vat (spawn ^web-server (spawn ^router))))

This is a slightly simpler version than the one used in the prototype, but it shows how we’re making asynchronous actors which can return promises accessible to HTTP requests. From the code above, we’ve already bridged into our Goblins actors. This is a pretty flexible bridge as this ^router actor just takes in a request and provides a response, we could dispatch this request in any number of ways. For our prototype, this is the approach we took:

(define-values (registry locator)
  (call-with-vat vat spawn-nonce-registry-and-locator))

(define (^router bcom)
  (lambda (request body)
    (define request-url (request-uri request))
    (match (string-split (uri-path (request-uri request)) #\/)
      [("" "static" filename)
       (static-file (string-append "mandy/web/static/" filename))]
      [("" "object" id)
       (let ((object (<- registry 'fetch (base32-decode id))))
         (<- object 'request))])))

Most web frameworks have a special-purpose routing language that uses strings, which is an inexpressive anti-pattern. We’re fortunate in Scheme to have a powerful pattern matcher that we can use instead. In this case we’re matching a filename for our static files and we’re also making some objects available at /object/<base32-encoded-id>.

We can see above the router we’re spawning something called the nonce registry. This is an actor which provides a mechanism to register any number of actors against some identifier and look them up. The actor handles salting and hashing the IDs and even persistence. This works great for registering ActivityPub objects. Each one gets a unique ID which can be used for lookup later.

These IDs are bytevectors, so we base32 encode them to convert them into a textual form that can be included in a URI. We then just need to add a route to our match clause to look them up. You may notice we’re using <- to send messages which means we get a promise in return. This isn’t a problem for our web server though as the on handler will wait until it’s resolved.

The prototype has more routes and handles slightly more situations than the snippet of code shown above, but the principles introduced are the same.

How does ActivityPub work?

Let’s take a step back and look at ActivityPub itself both because how it works will be useful to keep in mind the rest of the post, and to see if we can see the actor model within the specification.

ActivityPub is actually not too tricky. It has concepts like inboxes and outboxes that you’re probably familiar with from email. It also has activities which describe something the user is “doing”. Activities are the building block of the protocol. Finally, it has objects which are things like Notes, Images, Videos, etc.

ActivityPub is actually two protocols in one. There’s the client-to-server protocol and then the federated server-to-server protocol. These protocols are actually very similar and for the most part mirror one another, but unfortunately the client-to-server protocol gets little love from ActivityPub implementations. Even so, let’s take a look at how I might go about posting a Note (think toot/tweet/microblog text of choice) to my good friend Christine:

POST /outbox/ HTTP/1.1
Host: tsyesika.se
Authorization: Bearer XXXXXXXXXXX
Content-Type: application/ld+json; profile="https://www.w3.org/ns/activitystreams"

{
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Create",
    "to": ["https://dustycloud.org/"],
    "object": {
        "type": "Note",
        "content": "Ohai Christine!"
    }
}

The JSON object above represents a Create activity which basically is posting something. Other activities might be Like, Share, Delete, etc. Most activities are transitive (the activity has an object) and our Create activity is no exception. The object inside is a Note with some content.

The activity itself is posted to my outbox as an HTTP POST. The server will assign IDs to both objects and assign me (Jessica) as the author of the note. If you were to do a GET on the outbox, you’d see something like this:

GET /outbox/ HTTP/1.1
Host: tsyesika.se
Authorization: Bearer XXXXXXXXXXX
Content-Type: application/ld+json; profile="https://www.w3.org/ns/activitystreams"

{
    "@context" : "https://www.w3.org/ns/activitystreams",
    "id" : "https://tsyesika.se/outbox",
    "type" : "OrderedCollection",
    "name": "Jessica's Outbox"
    "totalItems" : 1,
    "items": {
        {
            "@context": "https://www.w3.org/ns/activitystreams",
            "type": "Create",
            "id": "https://tsyesika.se/objects/79402654-a9e5-4356-a50d-5109fedbaacc"
            "actor": "https://tsyesika.se"
            "to": ["https://dustycloud.org/"],
            "object": {
                "type": "Note",
                "attributedTo": "https://tsyesika.se"
                "id": "https://tsyesika.se/objects/2f614e93-1fe7-4a8a-ba39-f9e4468ed77f"
                "content": "Ohai Christine!"
            }
        }
    }
}

Reading from a user’s outbox gives you all the things they’ve posted (often it’s paginated, but that was left unimplemented in the prototype). You can see that the note we posted is the only item and the server has assigned IDs the activity, note, and author.

The server also should deliver this activity and note to Christine by looking up her inbox and doing an HTTP POST to it with the activity. I’m not going to go any further into federation but it’s very similar to the client-to-server API.

ActivityPub meets Goblins

If you’ve worked with Goblins or other actor frameworks you might be thinking “these activities look an awful lot like messages between different objects” and you’d be right.

That Create activity we posted above could be written something like this:

(define note (spawn ^note #:content "Ohai Christine!"))
(<- tsyesika-outbox         ; The outbox
    'create                 ; A method called create
    #:object note           ; Create an object
    #:to (list christine))  ; Send it to Christine.

The outbox can then implement whatever things it needs to for its Create activity (assigning an ID), federating the post out, etc. Just like any other Goblins actor would implement a method.

The prototype I’ve built does a fairly simple transformation from the unmarshalled JSON data. The JSON data is accepted and parsed to an association list. Then there is then an unmarshalling step where any nested objects get converted into their corresponding Goblins actors. The result is an activity actor which looks like this:

(define-actor (^as2:activity bcom parent #:key actor object target result origin
                             instrument)
  (extend-methods parent
    ((actor) actor)
    ((object) object)
    ((target) target)
    ((origin) origin)
    ((to-method)
     (string->symbol (string-downcase (assoc-ref ($ parent 'to-data) "type"))))
    ((to-data)
     (append (filter-data `(("actor" . ,actor)
                            ("object" . ,object)
                            ("target" . ,target)
                            ("result" . ,result)
                            ("origin" . ,origin)
                            ("instrument" . ,instrument)))
             ($ parent 'to-data)))))

ActivityPub is based on Activity Streams 2.0 which has a hierarchical structure. The Activity type extends from a base type of Object. This is represented in the Mandy prototype as parent.

We can then use this very simple procedure which takes an activity and converts it to a message:

(define* (activity->message activity #:key send-to)
  (define method-name ($ activity 'to-method))
  (define object ($ activity 'object))
  (define to
    (if send-to
        send-to
        ($ activity 'object)))

  (list to
        method-name
        #:object ($ activity 'object)
        #:actor ($ activity 'actor)
        #:target ($ activity 'target)
        #:self activity))

This produces our message as a list with its method name and a bunch of keyword arguments. The methods can then be defined as normal Goblins methods. If all the keywords aren’t needed for that method behavior, it can include #:allow-other-keys so that everything else can be ignored.

As an example of this let’s see the Collection type which is basically a list of objects. Here’s the implementation in the prototype:

(define* (^as2:collection bcom parent #:optional [items (make-gset)])
  (extend-methods parent
    ((add #:key object #:allow-other-keys)
     (bcom (^as2:collection bcom parent (gset-add items object))))
    ((remove #:key object #:allow-other-keys)
     (bcom (^as2:collection bcom parent (gset-remove items object))))
    ((move #:key object target #:allow-other-keys)
     (define new-items (gset-remove items object))
     ($ target 'add #:object object)
     (bcom (^as2:collection bcom parent new-items)))))

We can see it supports add, remove and move methods. The specification defines the behavior of add as:

Indicates that the actor has added the object to the target. If the target property is not explicitly specified, the target would need to be determined implicitly by context. The origin can be used to identify the context from which the object originated.

In this case, there might be two things which would be important to this method, the first being the #:object keyword and the second being the #:target. Since the collection is being sent the add message, it’s being assumed in the above code that the sender has figured the collection out. The add method then just needs to care about the object, in which case it specifies object as the only key and ignores anything else. Finally, the behavior is straightforward it adds the object to the collection.

Hopefully the above shows how we can take these ActivityPub activities and transform them into Goblins messages. This gives us the desired Goblins ergonomics while implementing ActivityPub objects.

Going further

The prototype that I implemented was a demo trying to explore some of both the Goblins actor HTTP interface and how ActivityPub might be implemented in an actor framework like Goblins. Having helped co-author ActivityPub and then develop Goblins, I’ve had musings of how this implementation might look, but it’s been very exciting to see that they work in practice.

This demo has explored both a HTTP interface with Goblins actors and ActivityPub. I think each one has a lot of potential for future work and I’d love to see Goblins applied in building websites. Goblins could be used both to build the backend of websites by handling the HTTP requests themselves, and in the browser by using Hoot.

There’s a many aspects of an ActivityPub implementation left to explore, for instance Goblins’ persistence system would be well suited to be our database. We could explore adding federation (Goblins being a distributed framework would be well suited to implement). Hopefully in the future we’ll be able to build on this experiment more.

If you found this blog post interesting, both myself and Christine Lemmer-Webber will be giving a FOSDEM 2026 talk on this. We’d love to see you there if you’re attending, but if not the video will be posted shortly after the event.

Thanks to our supporters

Your support makes our work possible! If you like what we do, please consider becoming a Spritely supporter today!

Diamond tier

• Aeva Palecek
• David Anderson
• Holmes Wilson
• Lassi Kiuru

Gold tier

• Alex Sassmannshausen
• Juan Lizarraga Cubillos

Silver tier

• Brian Neltner
• Brit Butler
• Charlie McMackin
• Dan Connolly
• Danny OBrien
• Deb Nicholson
• Eric Bavier
• Eric Schultz
• Evangelo Stavro Prodromou
• Evgeni Ku
• Glenn Thompson
• James Luke
• Jonathan Frederickson
• Jonathan Wright
• Joshua Simmons
• Justin Sheehy
• Michel Lind
• Mike Ledoux
• Nathan TeBlunthuis
• Nia Bickford
• Noah Beasley
• Steve Sprang
• Travis Smith
• Travis Vachon

Bronze tier

• Alan Zimmerman
• Aria Stewart
• BJ Bolender
• Ben Hamill
• Benjamin Grimm-Lebsanft
• Brooke Vibber
• Brooklyn Zelenka
• Carl A
• Crazypedia No
• François Joulaud
• Grant Gould
• Gregory Buhtz
• Ivan Sagalaev
• James Smith
• Jamie Baross
• Jason Wodicka
• Jeff Forcier
• Marty McGuire
• Mason DeVries
• Michael Orbinpost
• Nelson Pavlosky
• Philipp Nassua
• Robin Heggelund Hansen
• Rodion Goritskov
• Ron Welch
• Stefan Magdalinski
• Stephen Herrick
• Steven De Herdt
• Thomas Talbot
• William Murphy
• a b
• chee rabbits
• r g
• terra tauri