Recently, I’ve been interested in the DuckDB project (like a SQLite geared towards data applications). And one of the amazing features is that it has many data importers included without requiring extra dependencies. This means it can natively read and parse JSON as a database table, among many other formats.

I work extensively with JSON day to day, and I often reach for jq when exploring documents. I love jq, but I find it hard to use. The syntax is super powerful, but I have to study the docs anytime I want to do anything beyond just selecting fields.

Once I learned DuckDB could read JSON files directly into memory, I realized that I could use it for many of the things where I’m currently using jq. In contrast to the complicated and custom jq syntax, I’m very familiar with SQL and use it almost daily.

Is there a standard way of referencing objects by identity in JSON? For example, so that graphs and other data structures with lots of (possibly circular) references can be sanely serialized/loaded?

From: JSON: Standard way of referencing an object by identity (for, eg, circular references)? – Stack Overflow

Then there’s an expired yet interesting Internet-Draft used in JSON Schema and Swagger and a Python module giving us this example:

{
“foo”: {“$ref”: “#/bar”},
“bar”: true
}

that would be resolved as:

{
  "foo": true,
  "bar": true
}