Your Markdown
is already a database.

mq-db parses Markdown into a flat block list with an interval index (Nested Set / Pre-Post Order), turning heading hierarchy into O(1) integer comparisons. Query it with SQL or mq, persist it to a compact custom page-file format — no SQLite dependency.

install mq-db → view source

sql> every block nested under a heading, in one comparison under()

-- all text/code nested under the "Architecture" heading
SELECT b.block_type, b.content
FROM blocks b
WHERE under(b.pre, b.post,
  (SELECT pre  FROM blocks WHERE block_type = 'heading' AND content = 'Architecture'),
  (SELECT post FROM blocks WHERE block_type = 'heading' AND content = 'Architecture'))
  AND b.block_type IN ('paragraph', 'code')
ORDER BY b.pre;

Every block gets a pre/post pair at index time — "is X inside Y?" becomes one integer range comparison, not a tree walk.

Why mq-db

No tree walk

Nested Set (pre/post) encoding assigned at index time turns "is X inside Y?" into a single interval comparison — under(pre, post, anc_pre, anc_post).

Three index layers

A BitmapIndex on block type, a BTreeIndex on pre/post, and a HashIndex on content/lang/depth push SQL predicates down before a block is materialized.

Two query engines

A custom sqlparser-based SQL evaluator, and mq via mq-lang — run either against the same block storage. No SQLite dependency.

Zone-map pruning

Per-document statistics skip irrelevant files before a single block is scanned, ahead of the index layers.

O(1)

ancestor / descendant checks

index layers — bitmap · btree · hash

8 KB

fixed pages, checksummed

external db dependencies

SQL, joins included

documents and blocks are plain virtual tables — join them like any relational schema, plus self-joins over the interval index for structural checks.

documents ⋈ blocks JOIN

-- documents that contain Python code
SELECT DISTINCT d.path
FROM documents d
JOIN blocks b ON b.document_id = d.id
WHERE b.block_type = 'code' AND b.lang = 'python';

fig. 1 — a plain relational join across documents and blocks.

self-join over pre/post JOIN ×2

-- H2 headings immediately followed by a list
SELECT d.path, h.content AS heading
FROM blocks h
JOIN blocks nxt ON nxt.document_id = h.document_id
                 AND nxt.pre = h.pre + 1
JOIN documents d ON d.id = h.document_id
WHERE h.block_type = 'heading' AND h.depth = 2
  AND nxt.block_type = 'list';

fig. 2 — a structural self-join, one row apart in pre-order.

mq() scalar function inline mq

-- run an mq program against Markdown content, in SQL
SELECT mq('.h1 | to_text', content) AS title
FROM blocks
WHERE block_type = 'code' AND lang = 'markdown';

fig. 3 — mq embedded as a scalar function inside a SQL projection.

CASE + aggregates GROUP BY

-- bucket headings by depth, summarize per bucket
SELECT
  CASE WHEN depth <= 1 THEN 'top-level' ELSE 'nested' END AS bucket,
  count(*),
  group_concat(initcap(trim(content)), ', ') AS headings
FROM blocks
WHERE block_type = 'heading'
GROUP BY CASE WHEN depth <= 1 THEN 'top-level' ELSE 'nested' END;

fig. 4 — CASE expressions and aggregates over heading depth.

Index

9 entries

Flat block storage

Every Markdown element becomes a typed Block with row-polymorphic properties.

O(1) hierarchy queries

Interval index (pre/post) makes ancestor/descendant checks a single comparison.

Three-layer indexes

Bitmap, BTree, and Hash indexes for fast SQL predicate pushdown.

Zone maps

Per-document statistics skip irrelevant files before scanning any blocks.

Dual query engines

SQL via a custom sqlparser-based evaluator, and mq via mq-lang.

DDL support

CREATE TABLE, INSERT INTO, DROP TABLE for in-memory custom tables.

Function library

String, numeric, null-handling, CASE, and aggregate functions comparable to a general-purpose RDBMS.

Custom page storage

8 KB fixed pages, checksums, atomic writes to a single-file format.

CLI + REPL + TUI

Full terminal experience, plus an HTTP server for SQL and mq over JSON.

Install

# downloads, verifies (SHA256), and installs to ~/.local/bin

curl -fsSL https://raw.githubusercontent.com/harehare/mq-db/main/bin/install.sh | bash

mq-db index docs/ --recursive && mq-db sql "SELECT block_type, count(*) FROM blocks GROUP BY block_type"

Your Markdown is already a database.