-
Spreadsheets
Databases
AWS
Implementing a Spreadsheet Event Log on DynamoDB
In the distant past, before I got sucked into a seemingly never ending series on databases, I said that I was going to start formalizing the format for my cloud based, serverless, event sourced spreadsheet. I realize now that I’ve said very little on how I’m going to implement the central component of my spreadsheet, the event log.
-
Databases
DynamoDB Database Grid View
DynamoDB is AWS’s flagship, serverless, horizontally scalable, NoSQL database. The work that ultimately led to the release of DynamoDB in 2012, started in 2004. Amazon was growing rapidly and finding it hard to scale their relational databases. The development of DynamoDB was driven by the observation that 70% of Amazon’s queries were key-value lookups using a primary key to return a single row. Another 20% returned a set of rows from a single table.
-
Databases
MongoDB Database Grid View
Last time we dipped our toes into the waters of schemaless databases by using a JSONB column in Postgres to store a set of custom fields. After a few attempts we got it working. However, it didn’t offer many benefits compared with the denormalized relational database optimizations we previously tried, while bringing in plenty of additional friction of its own.
-
Databases
JSON Relational Database Grid View
When we aggresively denormalized our Grid View relational database, we were able to achieve our aim of a single table solution. However, it came with a lot of unintended consequences. In order to support up to 100 custom fields of each type, we added 400 custom field columns to our issue table, most of which were NULL. In theory the overhead should have been minimal. In practice, we ran into all kinds of implementation and operational problems.
-
Databases
Denormalized Relational Database Grid View
We’ve been good. We’ve followed the rules. Our database is fully normalized and we have all the referential integrity checks we need in place. And yet. Our queries seem overly complex. There’s a constant battle to try and keep queries scalable. Despite all that, performance is not what we’d like.
-
Spreadsheets
Merging and Importing Spreadsheet Snapshots
Last time, we looked at the added complexity that comes when you start inserting and deleting rows and columns from your spreadsheet. Spreadsheet snapshots are made up of multiple segments. Once you start inserting and deleting things, those segments are in different coordinate spaces. You need to transform the earlier segments into the coordinate space of the most recent as you load them.
-
Databases
Custom Fields with a Normalized Relational Database
Last time we discovered that it’s relatively easy to build a Grid View application using a Normalized Relational Database. True, it was a toy example with a small number of fixed fields. However, given some reasonable functional limitations, we showed how it could scale to manage large collections with 100,000 items or more.
-
Databases
Normalized Relational Database Grid View
Let me take you back to a time before NoSQL, when E.F. Codd’s relational rules and normal forms were the last word in database design. Data was modelled logically, without redundant duplication, with integrity enforced by the database.
-
Databases
Cloud Architecture
The Ubiquitous Database Grid View
You’re building a full stack application that enables teams of people to do … something. In order to manage the process of doing something, the teams collaborate by creating, filling in, finding, sorting and archiving forms. There will be some industry specific bits of workflow that let you convince yourself that you’re not building the same thing over and over again.
-
Spreadsheets
Cloud Architecture
Making Spreadsheet Snapshots work with Insert and Delete
When you’re implementing a cloud spreadsheet, it’s tempting to think of it as just another kind of database. Each row of the spreadsheet is equivalent to a row in a database. Each column in the spreadsheet is equivalent to a column in a database. Yes, spreadsheets don’t have schemas. Yes, spreadsheets can have lots of columns. However, there are plenty of examples of NoSQL databases that are schemaless and have wide column stores.