Your Markdown
is already a database.

mq-db parses Markdown into a flat block list with an interval index (Nested Set / Pre-Post Order), turning heading hierarchy into O(1) integer comparisons. Query it with SQL or mq, persist it to a compact custom page-file format — no SQLite dependency.

install mq-db → view source

the source, annotated at index time

README.md block_type · pre / post
# mq-db heading · 1 / 18
mq-db parses Markdown into a flat block list with an interval index. paragraph · 2 / 3
## Architecture heading · 4 / 17
Every heading opens an interval; blocks inherit pre/post bounds. paragraph · 5 / 6
- Bitmap index on block_type list_item · 8 / 9
- BTree index on (pre, post) list_item · 10 / 11
- Hash index on content, lang, depth list_item · 12 / 13
```sql SELECT block_type, count(*) FROM blocks GROUP BY 1; ``` code · 15 / 16

1 every block gets a pre/post pair at index time — "is X inside Y?" becomes one integer range comparison, not a tree walk.

2 the query planner picks a BitmapIndex, BTreeIndex, or HashIndex automatically, based on the predicate.

Hierarchy checks
without a tree walk.

Most Markdown tooling answers "what's inside this section?" by walking a tree. mq-db assigns every block a pre/post pair (Nested Set encoding) at index time, so an ancestor check becomes a single interval comparison — under(pre, post, anc_pre, anc_post).

Three secondary index layers push SQL predicates down before a single block is materialized: a BitmapIndex on block type, a BTreeIndex on pre/post, and a HashIndex on content/lang/depth.

sql> RAG extraction under a section under() 
-- all text/code nested under the "Architecture" heading
SELECT b.block_type, b.content
FROM blocks b
WHERE under(b.pre, b.post,
  (SELECT pre  FROM blocks WHERE block_type = 'heading' AND content = 'Architecture'),
  (SELECT post FROM blocks WHERE block_type = 'heading' AND content = 'Architecture'))
  AND b.block_type IN ('paragraph', 'code')
ORDER BY b.pre;

fig. 1 — scoping a query to everything nested under a heading, via a single interval comparison.

O(1) ancestry

Interval index turns "is X inside Y?" into an integer range check.

Dual engines

A custom sqlparser-based SQL evaluator, and mq via mq-lang — no SQLite.

Zone pruning

Per-document stats skip irrelevant files before any block is scanned.

SQL, joins included

documents and blocks are plain virtual tables — join them like any relational schema, plus self-joins over the interval index for structural checks.

documents ⋈ blocks JOIN 
-- documents that contain Python code
SELECT DISTINCT d.path
FROM documents d
JOIN blocks b ON b.document_id = d.id
WHERE b.block_type = 'code' AND b.lang = 'python';

fig. 2 — a plain relational join across documents and blocks.

self-join over pre/post JOIN ×2 
-- H2 headings immediately followed by a list
SELECT d.path, h.content AS heading
FROM blocks h
JOIN blocks nxt ON nxt.document_id = h.document_id
                 AND nxt.pre = h.pre + 1
JOIN documents d ON d.id = h.document_id
WHERE h.block_type = 'heading' AND h.depth = 2
  AND nxt.block_type = 'list';

fig. 3 — a structural self-join, one row apart in pre-order.

mq() scalar function inline mq 
-- run an mq program against Markdown content, in SQL
SELECT mq('.h1 | to_text', content) AS title
FROM blocks
WHERE block_type = 'code' AND lang = 'markdown';

fig. 4 — mq embedded as a scalar function inside a SQL projection.

CASE + aggregates GROUP BY 
-- bucket headings by depth, summarize per bucket
SELECT
  CASE WHEN depth <= 1 THEN 'top-level' ELSE 'nested' END AS bucket,
  count(*),
  group_concat(initcap(trim(content)), ', ') AS headings
FROM blocks
WHERE block_type = 'heading'
GROUP BY CASE WHEN depth <= 1 THEN 'top-level' ELSE 'nested' END;

fig. 5 — CASE expressions and aggregates over heading depth.

Index

9 entries
01

Flat block storage

Every Markdown element becomes a typed Block with row-polymorphic properties.

02

O(1) hierarchy queries

Interval index (pre/post) makes ancestor/descendant checks a single comparison.

03

Three-layer indexes

Bitmap, BTree, and Hash indexes for fast SQL predicate pushdown.

04

Zone maps

Per-document statistics skip irrelevant files before scanning any blocks.

05

Dual query engines

SQL via a custom sqlparser-based evaluator, and mq via mq-lang.

06

DDL support

CREATE TABLE, INSERT INTO, DROP TABLE for in-memory custom tables.

07

Function library

String, numeric, null-handling, CASE, and aggregate functions comparable to a general-purpose RDBMS.

08

Custom page storage

8 KB fixed pages, checksums, atomic writes to a single-file format.

09

CLI + REPL + TUI

Full terminal experience, plus an HTTP server for SQL and mq over JSON.

Install

# downloads, verifies (SHA256), and installs to ~/.local/bin

curl -fsSL https://raw.githubusercontent.com/harehare/mq-db/main/bin/install.sh | bash

mq-db index docs/ --recursive && mq-db sql "SELECT block_type, count(*) FROM blocks GROUP BY block_type"