Filigree

Written — Updated
  • Filigree is a web app framework based on Rust and SvelteKit, which includes common components such as authentication and background workers, and also makes it easier to set up data models and generate SQL, endpoints, and so on to use them.
  • Task List

    • Up Next

      • Generate tests for models
      • Tests for users without proper permissions for each action
      • Tests for login/logout
    • Soon

      • Endpoints for retrieving and updating your own user
      • List all permissions in a file with descriptions
      • Need a default role of admin which has all the permissions. (Or actually just add a new admin permission so there's one for the global admin and one for the team admin)
      • signup endpoint to create a new org that also creates default roles, adds a user, etc.
      • Ability to make signup open to the public or admin-only
      • Configurable "create" permission
        • for non-individual models, the owner permission or the write permission
        • for individual models, can be a separate global permission or always allow
        • How to properly define configuration for these since they have two settings?
          • Maybe make a tagged enum for permissions model, with different values for each
      • Password forgot/reset endpoints
      • Integrate with email sending services
      • MVP Basic email sending, enough to do password management and such
      • Auto-add appropriate dependencies to Cargo.toml
    • Later/Maybe

      • Template System
        • Easier way to add new fields to user/org model than redefining the whole thing
        • New runs diff against previous run and only apply diffed changes
          • When running the CLI, save the generated templates (post-formatter) to a state directory
          • On each run, diff the generated templates against the previous state in that directory
          • Apply that diff to the actual output file
            • Need a patch style that matches against nearby lines instead of just using line numbers.
          • Point out conflicts as they occur, probably use git style merge conflict markers
          • diffy supports three-way merge so that's also worth looking at
          • This system allows the user to make changes to the generated files but still update the config.
          • Need a command to rewrite a particular file
          • Need a command to just regenerate the state cache, for when changing formatter settings
        • Data-driven default org contents, even if this just a JSON blob to start
          • This can piggyback off of the fixture system
          • Considering not doing this since it may be simpler to just do it in a function.
        • Some way to easily add hooks to CRUD operations
          • Developer should probably just edit the generated code here.
        • Initial migration should only be written once
        • New models should be added in new migrations
        • Columns changes to models should generate new migrations
        • Models can define extra permissions
        • Models can define custom functions/actions
          • Might not do this since it may be worse DX than just adding to generated code
      • Authentication and Registration
        • OAuth
        • Passwordless email auth
        • Password requirements
          • length
          • mix of character types
        • Prevent reuse of recent passwords
        • Expire passwords that need a rehash due to upgraded standards
        • Expire passwords after a certain amount of time
          • This isn't recommended practice but some enterprise clients require this in order to sign with you.
        • Passkeys
        • Authenticator app
        • Authenticator Account recovery codes
        • Email-based user verification
        • Account self-deletion
        • Optional TOS acceptance at registration
        • Service accounts
        • Record info about sessions (geoip, user-agent, etc.)
        • Allow logging out other sessions
      • Permissions
        • For models with global permissions, add permissions check in route layer before hitting database
        • Project-based Access Control
        • API Key lookup query needs to change to make permissions a subset of the user's permissions when inherituserpermissions is false
        • Feature flags
        • Add "all" model permissions so that some roles have permissions on everything even when new models are added.
          • This needs some more thought though. Some models like user, org, etc don't want to have this blanket rule applied.
      • Block Storage
        • Block storage abstraction
          • revisit opendal vs. object_store crates
        • Block storage: Make it easy to return an object as a Response body
        • Block storage: Option for Uploads via signed URL
      • Workers
        • Define background jobs and generate a helper to enqueue one
        • Outbox pattern support for running background jobs
        • Define scheduled tasks that run a job on a cron schedule
      • Notifications
        • Notifications: Other types of notifications?
        • Notifications Internal notifications
          • e.g. Discord webhook notifying myself about events
          • This should have notification templates and also destinations for the notifications
        • Email template system
      • Bootstrap the real database from fixtures? Maybe not, this is simpler to do in code and tie in to the "new org" code.
      • Generate Typescript interfaces and JS code to talk to the API
      • Easy setup for commercialization
      • Collect and export metrics to some tool
        • Looks like Honeycomb supports Prometheus format
      • Delete log table for possibility of undelete later
      • Option for auditing on all operations
      • Ability for admins to assume permissions of other users
        • This needs to be audited as well even if nothing else is audited
      • Password confirmation flow for "dangerous" actions
      • User profile photos
    • Done

      • Generate fixtures for tests — Jan 5th, 2024
      • Write functions needed to bootstrap test data — Jan 4th, 2024
        • These should basically be the version that the actual app will use, just deferring all the surrounding functionality that isn't needed to write basic tests
        • Add org
        • Add roles
        • Add user to org with roles/permissions
        • Add new org (using all the above)
        • Add API key for user
      • Password login/logout endpoints — Dec 29th, 2023
      • Get server to start — Dec 27th, 2023
      • Add permissions checks to endpoints — Dec 27th, 2023
      • Add app-specific AuthInfo object — Dec 23rd, 2023
      • Add AuthQueries implementation — Dec 23rd, 2023
      • Select queries need to return the current permission level on each object for project/object-level permissions — Dec 22nd, 2023
      • Make more files "generated once" — Dec 22nd, 2023
        • endpoints should only be generated once, and should always be generated even if they are disabled
        • Move files that are regenerated to "generated" subdir
      • Add middleware layer to do permissions check — Dec 22nd, 2023
      • Add authz middleware to do generic predicate check — Dec 22nd, 2023
      • query to load user/roles/org for auth info — Dec 21st, 2023
      • Generate router for all model endpoints — Dec 21st, 2023
      • Create endpoints for the model — Dec 20th, 2023
      • Bearer token auth middleware — Dec 20th, 2023
      • Session cookie auth middleware — Dec 20th, 2023
      • Figure out querystring filter parameters — Dec 18th, 2023
      • Pagination — Dec 18th, 2023
      • Rename team to organization
      • Create Rust structures for each model
      • Bootstrap CLI utility
      • Figure out main config format
      • Figure out model definition
      • Create team, user, role, permissions, etc. tables
      • Create SQL migration
      • Create SQL queries to read/write the model
  • This should be made modular somehow so it's easy to maintain and add new stuff.
    • Modules define dependencies
  • Config
    • formatters to use
    • SQL dialect (postgres or sqlite)
  • Model definition
    • Write in TOML
    • Each field has
      • SQL type
      • nullable (default not null)
      • Rust type (can be inferred from the SQL type, but can be overridden)
    • Define Indexes and primary key
      • indexes can maybe just be defined in raw sql
    • Can have child relationships, one -to-one and one-to-many
    • Can have JSON fields, which deserialize to either JSON values or other Rust structs
      • The rust structs can be defined in the config as strings or something like that
    • List of other query functions to generate (find by field, etc.)
    • Permissions
      • Automatic checking on all operations
      • Ability to define which permissions are needed for what
    • some kind of hook on update? or is this better done by just changing the generated code myself
  • Consider some kind of codegen for even easier crud endpoint generation
    • Need ability to generate migrations that add/remove columns too
      • this can definitely be manual to start though.
  • Authentication
    • Need a user/account mapping to support multiple auth methods per user.
  • Authorization
    • Concepts
      • Organization
      • User
        • Users can potentially be in multiple organizations
      • Role
        • Basically a collection of permissions
      • Group
        • This is similar to Role
        • Might not do this, but it can be useful to make this different when more advanced sharing options are created.
      • Actor - all the things (users, roles, groups) that can have permissions on objects
        • When looking at permissions we gather up all the actors that apply to the current user, and use that whole list for permissions checks
    • The permissions table is then a list of the permissions each actor has.
    • Permissions
      • These should be definable per model once more than one is supported.
      • Tables
        • Global permissions
          • A many-to-many mapping of actor to permission
        • Object permissions
          • A mapping of actor to object and permission
          • For a project-based model, the projects are also just objects
      • Models
        • Role-based Access Control (RBAC)
          • Each role has permissions on whether it can read/write/admin certain types of objects.
          • This is the first one we'll support
          • Might be useful to have two sets of roles, some which can have the global permissions and are only created by administrators, and others which are generic groups of people for managing sharing of projects and objects
            • Need to think about how much this needs to be in the data model vs. just something in the UI. Probably just a single boolean flag is sufficient.
        • Resource-based access control
          • Read/write/admin on individual objects
          • Each role/user has specific permissions for each object
          • More complex, need to make sure the relevant entries get added for each object when it is created
          • Best for a model where objects are not shared by default.
        • Project-based Access Control
          • This combines with the above concepts but is a middle level of granularity compared to the all-or-nothing of plain RBAC and the often-overly-granular resource-based system.
          • A project acts as a container for objects, and actors can have permissions on the project instead of the individual objects.
          • Will probably implement this after RBAC, just need additional functionality for project management and ability to
          • Both individuals and roles can be added to projects
        • Attribute-based Access Control (ABAC)
          • Objects have attributes and actors
        • Public sharing
          • Some sites are built around publicly sharing your stuff.
          • This is basically the same as the resource-based access control model, but with the ability to share with an "anyone" user, and then all the permissions checks check for that actor ID as well.
      • Types of default permissions
        • creator only until shared
        • Creator gets write access, everyone else gets read access
        • everyone can see, only creator can write
        • Everyone in the project can see
        • Everyone in the project can edit
  • Pricing Plan Support
    • Add pricing tiers and permissions that go along with those tiers - this should mostly be data driven
      • Need ability for numeric values as well (e.g. max 3 projects, 500MB storage, up to 60 minutes of something, etc.)
    • Need ability to do grandfathering of pricing tiers and permissions
      • Essentially this means that both price plans and permissions sets should be versioned, and separately.
      • And there should be the ability to update existing plans and all previous versions

Thanks for reading! If you have any questions or comments, please send me a note on Twitter.