Kafka for One Machine: An Embedded Event Log With Offsets, Compaction and No Broker
A desktop app wants an undo history that survives a crash. An edge gateway buffers sensor readings while its uplink is down and has to replay them, in order, once the link is back. An AI agent writes down every action it takes so a bad run can be replayed later. All three are the same data structure: an append-only sequence of records, plus readers that remember where they stopped. Kafka is that structure, and also a cluster, partitions and the people who tune them.
Mapping Every Dependency to the Features That Break If It Disappears Tomorrow
Run npm ls --all on a mid-sized web app and you get a tree that scrolls for pages. It’s accurate and complete, and on the morning a maintainer deletes a package it tells you almost nothing. You can see that tiny-util sits three levels down. You can’t see whether losing it breaks the checkout page or a script nobody has run since spring.
A dependency tree answers “what do I depend on”. The question that matters is “what breaks, and for whom, if this package vanishes or turns hostile”. They need different data. The tree comes from a lockfile. The answer comes from your own source: which functions call the package, which routes, commands and jobs those functions serve, and how much code would change to replace it. A tool that produced that for every package would turn dependency management from an inventory into something a product owner can read.
Most Log Volume Is a Few Templates Repeated, So Reduce Logs Locally Before Shipping Them
Take a day of production logs from one service, mask the numbers, IDs and quoted strings in every line, and sort the resulting patterns by how often they occur. The top of that list is short and dull: health checks, cache hits, a retry notice from a client library nobody owns. Each of those lines was serialized, shipped, parsed and indexed millions of times to say the same thing, and you paid for every copy. Run it on your own logs. The claim is easy to check, so don’t take it on faith.
One Binary, One File, No Daemon: The SQLite and nginx Pattern as a Product Strategy
Compare two quickstarts. The first says to install Docker, bring up a compose file with a database, a cache and a worker, create an account, and paste a token into an env file. The second says to download one file and run it. Both tools might solve the same problem. Only one of them survives an evaluation that lasts ten minutes.
The second quickstart is a product decision, made in the architecture. SQLite and nginx made it decades ago, and Caddy, DuckDB, ripgrep, jq and Hugo have made it again in their own corners. Small surface area, few dependencies and one artifact to copy work as an adoption strategy. For a one-person or AI-assisted project they also work as the only scope control that holds up. That’s the claim, and the rest of this post tries to break it.
Redis Semantics Without Redis: An Embedded State Engine for Counters, Leases and TTLs
A command-line tool should run only one copy of its nightly export at a time. A small web app wants to cap each user at 60 requests a minute. A desktop app needs to remember when it last synced. All three are state: an integer, a timestamp, an owner. All three usually get solved by adding Redis, because Redis has the verbs for them: INCR, SET NX PX, EXPIRE. The app needed the verbs and got a server along with them, a port and a persistence config included.
Spreadsheet Recalculation Inside a Database: Derived Values That Track Their Own Dependencies
A line item’s quantity goes from 4 to 5. The order subtotal should change, and so should the tax, the order total, the customer’s lifetime value and the regional revenue figure on a dashboard. Which of those actually update depends on which code path made the edit, because each one gets recomputed somewhere different: the checkout handler, the invoice renderer, a nightly job, a SQL view somebody wrote for finance. Every copy of the formula is another chance for two screens to disagree.
SQL and Key-Value Over the Same Bytes: An Old Database Idea Worth Making Small
A sensor node writes a reading every few seconds. The firmware wants one thing from storage: put a record by key, get it back by key, as fast as the flash allows. The dashboard on the gateway wants something else, like average temperature by sensor by hour, joined to a table of sites. The usual answer is two databases with a translation job between them, a key-value store on the device and SQLite or Postgres upstream. Then comes a year of debugging the translation: a unit that changed, a row that arrived twice, a field the device encodes one way and the gateway reads another.
Store Changes Instead of Current Rows: An Embedded Database Built Around History
A support ticket asks two plain questions: which plan was customer 4411 on when the August invoice went out, and who moved them off it? The customers table can’t answer either. It holds today’s row. The audit table someone added two years ago holds most of the story, apart from the week a data fix ran a bulk UPDATE in raw SQL and skipped the ORM callbacks that write the audit rows.
Verified Absence Is Evidence: A Database for Recording That You Checked and Nothing Was There
An auditor asks you to show that customer 1187 was screened against a sanctions list on March 3. The screening job writes a row when it finds a match, and customer 1187 has no rows. That fits two histories: the job ran and found nothing, or the job never ran. The database can’t tell them apart, so you hold no evidence either way.
This is the closed-world assumption doing what SQL has always done: a row that doesn’t exist is false. RDF and OWL went the other way with an open-world assumption, where a missing statement means unknown. Neither has a place for the sentence you actually want to store: I looked here, in this way, at this time, and found nothing.
Screening App Ideas Before You Write Code: Four Questions, Starting With Who Pays
App ideas cost nothing. Every one of them looks shippable in a weekend, and then the weekend turns into three months of auth flows, sync bugs, review notes and a paywall that took longer than the app.
So before Xcode or Android Studio opens, put the idea through four questions and write the answers down. Who pays? What problem does it solve? Can you say what it does in one sentence? Why would anyone switch, and what stops Apple from copying it? Ask them in that order. Money goes first because it kills weak ideas fastest, while a kill still costs an evening.