Pic Store

Tasks

• Up Next

• Soon

  • Real 2x image support
    • Currently you can just configure a double-size image in the conversion profile, but if you only want to use it for 2x there's no way to figure that out when querying the resulting image. Should add an option to generate a 2x image and then mark the resulting image as such in the metadata so they can be better merged in the picture tag.
  • Quick upload endpoint
    • This should allow creating a base image and uploading it with a single call. Some of the other image metadata such as alt text can be set later when using this mode.
    • /projects/:project_id/quick_upload
    • query string parameters:
      • upload profile, if not the default
      • alt text — defaults to empty
      • filename — defaults to empty string (so just the base ID)
      • output location name — this was already optional, so same logic as before
  • Replace current model and auth stuff with Filigree
    • This will help bring in a bunch of the missing endpoints.

• Later

  • Side Helper Tasks
    • CLI app that can hash an image and get the corresponding <picture> tag or other URL for it, and upload it if needed.
    • Some sort of GUI version of the same, maybe even something that can be triggered straight from macOS Finder.
      • Good excuse to try Tauri
  • Generate <picture> tag corresponding to the base image and its output images. (The API currently exposes the info to let this happen, but doesn't generate the HTML on its own.)
  • When adding a storage location, allow testing read/write access to it
  • Make it possible to delete obsolete output images
  • Support HEIC as input format

  • More tests

    • Upload a base image and see that the output images get created
    • Targeted tests for each image type
      • Read PNG, JPEG, Avif, Webp
      • Convert image to PNG, JPEG, Avif, Webp
    • Check that image conversion has a reasonable result
    • Upload to some kind of mock S3
    • Set upload locations, conversion profile, etc. and read them back
    • Test all the permissions, checking for authorized and unauthorized operations
    • Set up e2e test framework
      • Scaffold base data
      • Testing functionality, new database and so on
      • Create temporary folder to hold output images
      • Some method of checking that output images actually look ok. The image-compare crate may work well here.

• Done

  • Update reconvert endpoint to create a new set of output images corresponding to the current conversion profile, instead of using the old set of outputs — Nov 19th, 2023
  • Use WebP lossless encoding when quality is set to 100 — Nov 19th, 2023
  • Allow setting image encoder quality in the conversion profile — Nov 19th, 2023
  • Add some simple logic to conversions
    • Only upsize
    • Only downsize
    • Only do this conversion if source image is a particular format
      • e.g. only generate JPEGs if the source image is also a JPEG.
  • Update S3 auth to support instance metadata auth method
    • This is mostly just adding the config and setting it to not do anything
  • Endpoint to check by hash if an image already exists
  • Ensure that replacing a base image properly replaces the output images
  • When replacing a base image, delete the current output images associated with it.
    • We'll want an option here for how long to keep the old images around
    • Forever should be an option
    • Resolved: Images of the same size/format are overwritten, the rest are retained and marked for manual deletion.
  • Check out tracing-tree as a replacement for bunyan formatter
  • Integrate DB into Axum
  • Set up storage providers
  • Accept HTTP uploads of images
  • Put uploaded images into S3 (can we stream them directly?)
  • Download files to convert from S3
  • Do conversion on files and save them back to S3
  • Graceful shutdown for server on SIGINT
  • Bootstrapping command
    • team
    • user
    • project
    • storage locations
    • conversion profile
    • upload profile

Goals

• Image Upload
• Convert and resize
• Helper library for image upload from front-end
• Helper library (vite plugin maybe?) for using uploaded images
  • This should also check by hash and upload if needed.

Codecs

• AVIF
  • ravif for encoding
  • libavif for decoding since it's more tolerant of slightly out-of-spec AVIF headers than other libraries
• WebP webp
• PNG and JPG - image

• Schedule jobs for later times
• Multiple channels and workers only pull the channels they're interested in
• Automatically keep job alive while it runs
• Retry on failure
• Nice to have
  • Checkpoints with payload updates
  • Run everything in a Diesel transaction and include the job completion at the end of the transaction
    • Just implement outbox pattern for this. Best to add helpers for it to Effectum
  • Likewise with checkpoints
• Currently using a modified version of sqlxmq, changed to work with Diesel. While I like the queue code, maintaining a fork like this isn't a great idea so I should find something else. (Switched to Effectum )

Workflows

• Image Upload

  • Add base image
  • Upload image
    • If presigned url upload
      • Client uploads the image
      • Client calls the server to indicate that it is finished
    • If direct upload, then accept the upload
  • Look at the conversion profile to determine which conversions need to take place
  • Enqueue the conversions
  • Task queue does the conversions
  • When they are all done, image is marked as done.

• Text Overlay

  • Given an existing base image

Roadmap

• Support conversion profiles
• Auth key validation
• Permissions table
• Simple web UI

Permissions Lookup

• Each object has a linked "permissioned object," which is the object on which we need to actually check that the permission is present.
• This means that given the roles, we need to
  • Option 1
    • Left join on the permissions table using a subquery that checks if any of the user's roles have the required permission on the linked permissioned object.
    • Select an "allowed" field on whether that returns null or not
    • This feels bulky...
  • Option 2
    • Siimilar left join, but just return nothing if it's not allowed.
    • This has the problem that we can't distinguish a missing object from a lack of permissions.
  • Option 3
    • Fetch the object
    • Do the permissions check on the relevant field in a separate query.
    • This is clean but
      • requires two queries
      • requires remembering to do the permissions check
  • An ideal solution, I think, would be option 1, but in a way that it can be automated as part of the fetch process.
    • Could do this with a function that takes some arguments
      • Table to read
      • Operation requested
      • The field on which the operation must be allowed (i.e. project_id)
      • the user info (roles and such)

Lazy Conversion

• Might end up doing this for v2
• Store the raw image and convert it as requests come in. Requires a little more coordination since we may end up needing to handle concurrent requests for an image format that is currently being generated.