HYTRADBOI Conference

Written — Updated
  • Incremental Execution

    • Using incremental execution for realtime machine learning
      • They were looking at storing unstructured data in the database and extracting data from it using a model.
      • The data ends up in separate columns in the same row as the unstructured data, with confidence values for each one.
      • Their UI then lets you do corrections which feed back into the model and immediately recalculate values from it.
      • This then feeds into incremental materialized views that update only the parts which need updating.
    • Incremental stream processing with Differential Datalog.
      • https://github.com/vmware/differential-datalog
      • VMWare was running queries to check consistency constraints within their networks.
      • For better performance they wanted to have queries that update incrementally, on only the data that has changed.
      • On top of Datalog, they added more data types such as structs and collections, and fragments of imperative code as well.
      • They’re compiling DDLog to use differential dataflow for the compute engine, and are now working on their own compute engine.
  • Airtable
    • GUI vs SQL
    • No need for extra join table for many-to-many relationships. It's handled in the background
      • probably by all relationships being many to many internally? Or just array fields maybe, but then the hiding ofthe implementation is the point.
    • The nice thing about Airtable is that the GUI primitives turn into a cheap business logic without much effort and no real GUI-building required.
    • Unlike a spreadsheet, you're still using database concepts such as actual relations between rows in tables and uniqueness constraints, which makes things more scalable and prevents the all-too-familiar gigantic unmaintainable Excel spreadsheet.
    • You can add triggers to perform certain tasks in the database when changing certain pieces of data.
      • The trigger scripts can be actual Javascript with access to a full API.
  • Ultorg: a user interface for relational databases
    • This is a general purpose database application, but with better priimitives for building functionality on top of it than the normal database tool.
    • Can show data for joins on each row.

Thanks for reading! If you have any questions or comments, please send me a note on Twitter.