{"id":11513,"date":"2024-03-24T18:43:09","date_gmt":"2024-03-24T17:43:09","guid":{"rendered":"https:\/\/monodes.com\/predaelli\/?p=11513"},"modified":"2024-03-24T18:43:12","modified_gmt":"2024-03-24T17:43:12","slug":"duckdb-as-the-new-jq","status":"publish","type":"post","link":"https:\/\/monodes.com\/predaelli\/2024\/03\/24\/duckdb-as-the-new-jq\/","title":{"rendered":"DuckDB as the New jq"},"content":{"rendered":"\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Recently, I\u2019ve been interested in the <a href=\"https:\/\/duckdb.org\/\">DuckDB<\/a> project (like a <a href=\"https:\/\/www.sqlite.org\/\">SQLite<\/a> geared towards data applications). And one of the amazing features is that it has many data importers included without requiring extra dependencies. This means it can natively read and parse JSON as a database table, among many other formats.<\/p>\n\n\n\n<p>I work extensively with JSON day to day, and I often reach for <a href=\"https:\/\/jqlang.github.io\/jq\/\">jq<\/a> when exploring documents. I love <code class=\"\" data-line=\"\">jq<\/code>, but I find it hard to use. The syntax is super powerful, but I have to study the docs anytime I want to do anything beyond just selecting fields.<\/p>\n\n\n\n<p>Once I learned DuckDB could read JSON files directly into memory, I realized that I could use it for many of the things where I\u2019m currently using <code class=\"\" data-line=\"\">jq<\/code>. In contrast to the complicated and custom <code class=\"\" data-line=\"\">jq<\/code> syntax, I\u2019m very familiar with SQL and use it almost daily.<\/p>\n<cite>Source: <em><a href=\"https:\/\/www.pgrs.net\/2024\/03\/21\/duckdb-as-the-new-jq\/\">DuckDB as the New jq<\/a><\/em><\/cite><\/blockquote>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p class=\"excerpt\">Recently, I\u2019ve been interested in the DuckDB project (like a SQLite geared towards data applications). And one of the amazing features is that it has many data importers included without requiring extra dependencies. This means it can natively read and parse JSON as a database table, among many other formats. I work extensively with JSON&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"https:\/\/monodes.com\/predaelli\/2024\/03\/24\/duckdb-as-the-new-jq\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"quote","meta":{"inline_featured_image":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"federated","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[14],"tags":[84,354,349],"class_list":["post-11513","post","type-post","status-publish","format-quote","hentry","category-software-libero","tag-json","tag-sql","tag-sqlite","post_format-post-format-quote"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6daft-2ZH","jetpack-related-posts":[{"id":8667,"url":"https:\/\/monodes.com\/predaelli\/2021\/09\/08\/useful-tools\/","url_meta":{"origin":11513,"position":0},"title":"Useful tools","author":"Paolo Redaelli","date":"2021-09-08","format":false,"excerpt":"htmlq, like jq, but for HTML. Uses CSS selectors to extract bits content from HTML files. Mozilla's MDN has a good reference for CSS selector syntax. jq is a lightweight and flexible command-line JSON processor.","rel":"","context":"In &quot;Software Libero&quot;","block_context":{"text":"Software Libero","link":"https:\/\/monodes.com\/predaelli\/category\/software\/software-libero\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":15394,"url":"https:\/\/monodes.com\/predaelli\/2026\/03\/29\/nobody-gets-fired-for-picking-json-but-maybe-they-should\/","url_meta":{"origin":11513,"position":1},"title":"Nobody Gets Fired for Picking JSON, but Maybe They Should?","author":"Paolo Redaelli","date":"2026-03-29","format":"link","excerpt":"Nobody Gets Fired for Picking JSON, but Maybe They Should? By Miguel Young de la Sota Nobody Gets Fired for Picking JSON, but Maybe They Should? JSON is extremely popular but deeply flawed. This article discusses the details of JSON\u2019s design, how it\u2019s used (and misused), and how seemingly helpful\u2026","rel":"","context":"In &quot;Javascript&quot;","block_context":{"text":"Javascript","link":"https:\/\/monodes.com\/predaelli\/category\/javascript\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":9720,"url":"https:\/\/monodes.com\/predaelli\/2022\/10\/15\/uniform-exchange-format-uxf\/","url_meta":{"origin":11513,"position":2},"title":"Uniform eXchange Format (uxf)","author":"Paolo Redaelli","date":"2022-10-15","format":false,"excerpt":"Uniform eXchange Format (uxf) is a plain text human readable optionally typed storage format that supports custom types. It may serve as a convenient alternative to csv, ini, json, sqlite, toml, xml, or yaml. https:\/\/github.com\/mark-summerfield\/uxf Looks smart and interesting! Expecially this part: Implmenting a UXF pretty printer whould be doable\u2026","rel":"","context":"In &quot;Software&quot;","block_context":{"text":"Software","link":"https:\/\/monodes.com\/predaelli\/category\/software\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":924,"url":"https:\/\/monodes.com\/predaelli\/2016\/01\/26\/json-references\/","url_meta":{"origin":11513,"position":3},"title":"JSON references","author":"Paolo Redaelli","date":"2016-01-26","format":false,"excerpt":"Is there a standard way of referencing objects by identity in JSON? For example, so that graphs and other data structures with lots of (possibly circular) references can be sanely serialized\/loaded? From: JSON: Standard way of referencing an object by identity (for, eg, circular references)? - Stack Overflow Then there's\u2026","rel":"","context":"In &quot;Javascript&quot;","block_context":{"text":"Javascript","link":"https:\/\/monodes.com\/predaelli\/category\/javascript\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":9401,"url":"https:\/\/monodes.com\/predaelli\/2022\/05\/26\/the-simdjson-library\/","url_meta":{"origin":11513,"position":4},"title":"The simdjson library","author":"Paolo Redaelli","date":"2022-05-26","format":false,"excerpt":"The simdjson library Parsing gigabytes of JSON per second JSON is everywhere on the Internet. Servers spend a lot of time parsing it. The simdjson library uses commonly available SIMD instructions and microparallel algorithms to break speed records.","rel":"","context":"In &quot;Agenda&quot;","block_context":{"text":"Agenda","link":"https:\/\/monodes.com\/predaelli\/category\/agenda\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":15395,"url":"https:\/\/monodes.com\/predaelli\/2026\/03\/29\/protobuf-advolvendum-est\/","url_meta":{"origin":11513,"position":5},"title":"ProtoBuf advolvendum est","author":"Paolo Redaelli","date":"2026-03-29","format":false,"excerpt":"The proficient Miguel Young de la Sota suggests to use Protocol Buffers1 instead of JSON in his article \"Nobody Gets Fired for Picking JSON, but Maybe They Should?\" I shall add to my advolvenda2","rel":"","context":"In &quot;Advolvenda&quot;","block_context":{"text":"Advolvenda","link":"https:\/\/monodes.com\/predaelli\/category\/eiffel\/liberty-eiffel\/advolvenda\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts\/11513","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/comments?post=11513"}],"version-history":[{"count":0,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts\/11513\/revisions"}],"wp:attachment":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/media?parent=11513"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/categories?post=11513"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/tags?post=11513"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}