Glance

Up Next

Soon

• Notifications
• Customize layout of the data items
  • Get rid of the existing format for items, and make this into a block-based thing with rows and columns, where blocks can be text/image/chart/etc.
  • Block types
    • Text
    • Chart
    • Row
    • Column
    • Expansion Pane
    • Image
  • Blocks can reference a separate data block using JSON pointer syntax
  • Text can be markdown
  • Probably keep "title" and maybe subtitle but the rest should be new

Later

• Calculate a hash of all data fields and store/compare in the database to avoid unneeded writes when nothing has changed. Can possibly use the new MERGE syntax to help with this.
• Publish live on web
• Menubar App
• Item actions
  • See Apps can define actions to be done.
• Detail pages can be a web app served by the app itself
• Main Item display can be a simple chart or other viz
• Item detail can be a vega viz or something
  • See Details
• Let apps expose their config through the platform
  • Things like your location for getting the weather
• SSE update notification

Done

• REST endpoints to add/update/delete an app — Jul 29th, 2024
• REST endpoints to add/update items within an app — Jul 29th, 2024
• Simple auth system so that I can publish this readonly but still log in and make changes — Feb 14th, 2024
• Platform can run apps on a schedule — Dec 12th, 2023
• Web app reloads data on a schedule — Dec 11th, 2023
• Dismissible items — Nov 28th, 2023
• Render simple UI — Nov 28th, 2023
• Figure out more structured layout for app data — Nov 27th, 2023
• HTTP Server — Nov 27th, 2023
• Read items from each data file
• Watch app data directory
• Write a simple app that always has data to show (Weather or HN API or something super basic)
• Create app helper library
  • Common paths, JSON data definition, etc.
• Firm up designs
• Design JSON schema for data

App Data Format

• This is a JSON file with some information about the app and its current data.
• App Title
• URL to reverse proxy to for detailed web UI
  • And index page path, if applicable
• Data Items Array
  • Something like this
  • Item UUID
  • Item type
  • Description
    • This is HTML that gets injected into the page.
  • Chart?
  • Structured data
    • This can be used to allow more customization of the displayed data
  • Notification - optional
    • Notification also has a UUID so that a single item can show multiple notifications even if it already exists.
    • Notification title and text
    • Icon/image?
  • Persistent or dismissable
    • Dismissable items can be closed by the user until they are updated again, if ever.
  • Updated time
  • Details
    • This can be a path under the reverse proxy URL, if present
    • Or some defined visualization
      • This will be something like a SQLite database, a set of queries to run, and maybe some Vega viz to render?

Backend Platform

• Watch the data directory
• On load and when a file updates, read the file and see what the data items are
• Serve a web app which shows all the data
• Track things like notifications, dismissible item state, and so on.

UI

• Menubar app
• More detailed page for use in a browser/phone.
• Good case for some kind of microcontroller with an OLED
• Should be possible to run all these on a different computer from the platform.

• A mini-app template project that can be forked
• A main server
  • Manage and start the mini-apps
  • Set environment for the mini-apps with some well-known keys:
    • directory for each app to create its SQLite database and other stuff like that
  • Provide a way the mini-apps can call to push new data
    • indicate that a notification should be shown for important stuff
    • push new data to be shown when the app is opened
    • some way to define how to lay out the UI for that data
    • Could be HTTP endpoints that the server exposes
    • Items can be persistent until the app remove them, or acknowledgeable, where the stay until the user dismisses them.
    • Actually just using the filesystem would work well
      • Data goes in a sqlite database
      • schema file defines both visualizations and the queries to run to create the data for them.
      • Server watches for changes in the schema file.
      • How do we read all the data in an efficient manner?
        • Having to read the dashboard databases all at once when the dashboard opens could be messy.
        • OTOH it could be massively parallelized and the reads would generally be very quick, just a few rows per DB.
        • In reality, having all the summary data in the JSON itself will be fine. There won't be that much data.
      • Notifications would probably still need an endpoint?
        • Could have a field in the JSON that would let an app set a notification, with a UUID or something to handle idempotency.
  • A web-based UI
    • Maybe both as a menu-bar app and as a normal app
    • Lists and manages the mini-apps
    • Displays the pushed data

• Some way for mini-apps to expose a more in-depth look into their data.
  • Probably do this through some kind of reverse proxy that just assigns some path to proxy through to the mini-app so it can serve its own UI.
• Apps can define actions to be done.
  • Each item can have its own actions
  • Actions have a label, maybe an icon, styling, and a way to do the action
    • HTTP endpoint to hit.
    • Website to visit for more details
    • Command to run
• Auto-deploy from Git/HTTP sources?
• LLM summarization of current stuff?
• AI Agents which can send their status and results to the app (this probably wouldn't require any new functionality)

• In some ways, it would be simplest for the app to be a single file which gets started by a worker. The file can export certain functions to deliver the relevant functionality.
• A full-fledged app is much more flexible, but also somewhat harder to distribute and harder to proxy to. I think in the end this will be a better solution though.
  • Distribution: this can be a zip file. Run bun to install deps and then start it. This also technically allows the server to be written in any language, though that adds complexity.
• Could just have them run separately, which would be simplest in some ways. Some apps would run persistently, but many apps would just be a cron job. The cron jobs could also be done as an internal system in the platform.

• Consul for service registration
• Vault for pulling secrets like webhook keys
• Nomad for running the mini apps and pulling in the secrets into the containers
• caddy or nginx for the web server, using dynamically generated templates from consul service records?
• Task scheduler to poke services every so often, or should this be built in to the apps themselves?
  • How do we manage concurrency here? Probably by just having each app that needs it manage its own queue. Could do this using effectum to make it easy.
  • Would it be useful to provide a shell library that could allow writing each service as just a simple function or as close as possible? I think this would be helpful. Create the database connections, pull in the proper secrets, and so on. Apps that needs more functionality could use an http router or SvelteKit or something. But a simple approach like this could potentially allow for creating mini-apps within some other UI again, perhaps even with dashboard building blocks.

• mini apps that are actually full apps
• Mini apps that are actually just executable functions but with some extra functionality compared to what's there now in Ergo.
• So the former is just a matter of hosting, proxy, etc. The latter is where the design comes in.

• single function to execute the actual code
• Optional schedule on which the function runs.
• UI defined with simple building blocks
  • Queries to run against the stored data, something like a stored procedure
  • Charts to display those queries
  • controls to update the parameters for the queries
  • A way to define all this easily, not necessarily no-code, but low-code to start, similar to Gradio
• compared to the current method, the function should be uploaded from a file and executed in something external like Node, Deno, or Bun.
  • This allows for it to actually be tested and developed externally, without much fuss.
  • Optionally, perhaps even update these functions from a Git repository, where pushing to the master branch updates the function to be run.
  • Then we could just do a shebang to decide in what context it actually runs (with the inherent security problems this has? Maybe a whitelist of safe executors, and this is especially appealing with Deno and Bun(?) which have permissions systems).
  • Also look into WASM execution now that WASIX and similar standards are making that easier.
  • Provide an SQLite database path for each function so that it can implement its own persistence.
• An app should provide four functions: query, act, cron (optional) and buildUI.
  • Query gets data from the database for the UI to show
  • Act performs actions based on button clicks
  • Cron, if implemented, will do scheduled data updates
  • buildUI shows what kind of dashboards to show
  • Something like this anyway. Need to figure out the relation between query and buildUI, maybe they should be merged.
• Application package defines:
• sqlite database connection with predetermined location
• Sqlite migration system
• A way to define UI via code
• A way to send notifications through the app host
• a job queue

• The main downside of this is that it's harder to make a dashboard with status from various apps all in one.
• But this could be done through a common endpoint that returns data and ui definitions.
• So in this paradigm we have:
  • server that exists independently and with its own UI
  • a separate dashboards app that apps can register with
  • A /dashboard endpoint on each server that tells the dashboard what to display.
  • Or an endpoint on the dashboards app through which the servers can push data. This is probably the way to go since it doesn't enforce any particular structure for each app.