JS Sidecar

Up Next

Soon

• Allow passing additional dependencies which can be used in the scripts
  • This would probably be paths to somewhere on the filesystem that contains a library with package.json and all
  • I think supporting this would require supporting the entire node_modules lookup algorithm. The resolve.exports library may be helpful here, or it might also work to just use import
• Support callbacks from the JS side to the Rust side
  • e.g. to do expensive lookups, mutate state, etc.

Later/Maybe

• Consider using worker_threads instead of cluster
  • This should use a bit less memory (though probably not much less), but is complicated by the lack of ability to send sockets across thread boundaries, unlike cluster.
  • The bigger advantage is that it will make it easier to maintain a single code cache, and share data between threads in a zero-copy manner, using SharedArrayBuffers.
  • One approach is to have a single master thread create a new MessagePort pair for each connection. A thread gets the port and then the master shuttles buffers back and forth between them.
    • Mostly implemented here https://www.phind.com/search?cache=zdy35n834rpmds1cmiiszwt8.
    • Feels bad for latency though
  • Another way would be for each thread to have its own socket and then the Rust side communicates with each one and handles the load balancing. This would also potentially make it easier to share other things like persistent contexts but that is probably over complicating it.
    • This isn't great as a general approach but works well for this case because we know that there's just a single client (the Rust process) with exact knowledge of how things are structured.
    • I'll probably go this route.
  • Need to benchmark these approaches
• Support multiple engines (Bun, Deno)
  • Bun doesn't support node:cluster yet
  • Deno should work with minimal changes
• Run entire projects
  • with node_modules and so on
• Support running Boa or QuickJS in-process with the same interface
• Allow control over which Node built-in libraries can be imported
  • Currently none can be imported, this wouldn't be hard to change but making it configurable would be great.
• Network sandbox
  • This can be done most easily by replacing fetch with a function that wraps fetch and checks the URL before passing on the request
• Filesystem sandbox
  • Same as network, but for the fs functions. This is a bit more difficult to get right since there are so many functions.
  • Ideally we could just run the isolate in a chroot, but I don't think that's possible.

Done

• Benchmarks
• Support V8 code cache when evaluating code
  • Need some kind of content-addressable LRU caching so we don't hold on to code cache values forever
• Customizable number of workers
• Improve Rust API for an engine connection
  • Single function to run a script and wait for its result
• Start a single engine
• JS code to handle requests, run code, and respond
• Send requests to the engine, get a response
• Support one-way communication of a script back to the Rust side (e.g. progress messages)
• Ability to reuse a context across multiple requests without reinitializing it each time.
• Hook into console so that it goes back to Rust in a way that is linked to each request.
• Run a pool of engines and load balance requests between them.

Rust Side

• Start and maintain a pool of engines
• Allow passing JS code to the engine, either as strings or as filesystem references
• Support callbacks which

JS Side

• The core is a small application which takes in requests
• Support auto-importing modules from the filesystem before running the expression.
• Support compilation snapshots, if possible
  • Looks like Node.js support this via createCachedData. It doesn't cache stored data but does cache functions.
• Library which can callback to the Rust side to take actions

Why not Deno?

• In my experience in the past this worked out ok, but
  • you have to set up a lot of runtime stuff yourself
  • Updating all the Deno crates together is a pain and there were often breaking changes to be handled.
• Some of this might be better now as Deno itself has matured, but overall it seemed that embedding a "full-featured Deno" is not really as easy as it should be.

Why not bindings to QuickJS/Boa/etc?

• Runtime compatibility. Some applications may need to do things that only really work in the most popular JS engines, such as talking to Postgres.
• Harder to include arbitrary NPM packages or similar, without bundling
• For applications that don't need this, I do think it's worth providing a mode that just uses bindings and won't have to start a separate sidecar process.
• As QuickJS gets more WinterCG compatibility this also may be less of an issue.