{"id":9813,"date":"2022-11-09T20:40:00","date_gmt":"2022-11-09T19:40:00","guid":{"rendered":"https:\/\/monodes.com\/predaelli\/?p=9813"},"modified":"2022-11-09T09:57:31","modified_gmt":"2022-11-09T08:57:31","slug":"ascii-oh-ascii-wherefore-art-thou-ascii","status":"publish","type":"post","link":"https:\/\/monodes.com\/predaelli\/2022\/11\/09\/ascii-oh-ascii-wherefore-art-thou-ascii\/","title":{"rendered":"ASCII, oh ASCII! Wherefore art thou, ASCII?"},"content":{"rendered":"\n

ASCII, oh ASCII! Wherefore art thou, ASCII?<\/p>The original line copied by an infamous English poet<\/cite><\/blockquote>\n\n\n\n

Puns aside, in the XXI century there are still need to stick to plain, old 7 bit ASCII character table. Many industrial applications stick to it for its simplicity. Unicode is often an overkill in that situations.<\/p>\n\n\n\n\n\n\n\n

So how to find if a stream of text is Unicode? POSIX systems (GNU\/Linux, *BSD, MacOs and many, many others) have the file utility, which is as simple as “file filename” that will likely answer like:<\/p>\n\n\n\n

DEST_CLI.CSV.20220901-105314.backup: UTF-8 Unicode text, with CRLF line terminators\nfoo: UTF-8 Unicode text<\/pre>\n\n\n\n
In case of debug we will need to find where Unicode characters are actually used. Ifound inspiration in How to Find Non-ASCII Characters in Text Files in Linux<\/a> that has been graciously updated less than three days ago (lucky me!), slightly modifying the command; it is as simple as:<\/p>\n\n\n\n
grep --color='auto' -P -n "[\\x80-\\xFF]" foo<\/code><\/pre>\n\n\n\n
Sometimes we just need to convert a UTF-8 file to ASCII (best-effort)<\/a>. In many cases the iconv does the job, piping your data to <\/p>\n\n\n\n
iconv -f utf-8 -t ascii\/\/TRANSLIT<\/code><\/pre>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
ASCII, oh ASCII! Wherefore art thou, ASCII? The original line copied by an infamous English poet Puns aside, in the XXI century there are still need to stick to plain, old 7 bit ASCII character table. Many industrial applications stick to it for its simplicity. Unicode is often an overkill in that situations.<\/p>\n
Read more →<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[278],"tags":[],"class_list":["post-9813","post","type-post","status-publish","format-standard","hentry","category-tricks"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6daft-2yh","jetpack-related-posts":[{"id":2268,"url":"https:\/\/monodes.com\/predaelli\/2017\/03\/11\/a-programmers-introduction-to-unicode\/","url_meta":{"origin":9813,"position":0},"title":"A Programmer\u2019s Introduction to Unicode","author":"Paolo Redaelli","date":"2017-03-11","format":"link","excerpt":"\u00abA Programmer\u2019s Introduction to Unicode\u00bb from Nathan Reed\u2019s coding blog A Programmer\u2019s Introduction to Unicode March 3, 2017 \u00b7 Coding \u00b7 7 Comments \uff35\uff4e\uff49\uff43\uff4f\uff44\uff45! \ud83c\udd64\ud83c\udd5d\ud83c\udd58\ud83c\udd52\ud83c\udd5e\ud83c\udd53\ud83c\udd54\u203d \ud83c\uddfa\u200c\ud83c\uddf3\u200c\ud83c\uddee\u200c\ud83c\udde8\u200c\ud83c\uddf4\u200c\ud83c\udde9\u200c\ud83c\uddea! \ud83d\ude04 The very name strikes fear and awe into the hearts of programmers worldwide. We all know we ought to \u201csupport Unicode\u201d in our\u2026","rel":"","context":"In "Senza categoria"","block_context":{"text":"Senza categoria","link":"https:\/\/monodes.com\/predaelli\/category\/senza-categoria\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":11195,"url":"https:\/\/monodes.com\/predaelli\/2024\/01\/14\/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses\/","url_meta":{"origin":9813,"position":1},"title":"The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)","author":"Paolo Redaelli","date":"2024-01-14","format":"link","excerpt":"Ever wonder about that mysterious Content-Type tag? You know, the one you\u2019re supposed to put in HTML and you never quite know what it should be? Did you ever get an email from your friends in\u2026 The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character\u2026","rel":"","context":"In "Documentations"","block_context":{"text":"Documentations","link":"https:\/\/monodes.com\/predaelli\/category\/documentations\/"},"img":{"alt_text":"\u05d2","src":"https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2024\/01\/gimel.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":4600,"url":"https:\/\/monodes.com\/predaelli\/2018\/09\/29\/come-si-pronuncia-ascii-come-si-pronuncia\/","url_meta":{"origin":9813,"position":2},"title":"Come si pronuncia ASCII – Come si pronuncia","author":"Paolo Redaelli","date":"2018-09-29","format":false,"excerpt":"Come si pronuncia ASCII - Come si pronuncia Te pensa che si dice \"aschi\". Io ho sempre pronunciato come \"esci\"...","rel":"","context":"In "Senza categoria"","block_context":{"text":"Senza categoria","link":"https:\/\/monodes.com\/predaelli\/category\/senza-categoria\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":15394,"url":"https:\/\/monodes.com\/predaelli\/2026\/03\/29\/nobody-gets-fired-for-picking-json-but-maybe-they-should\/","url_meta":{"origin":9813,"position":3},"title":"Nobody Gets Fired for Picking JSON, but Maybe They Should?","author":"Paolo Redaelli","date":"2026-03-29","format":"link","excerpt":"Nobody Gets Fired for Picking JSON, but Maybe They Should? By Miguel Young de la Sota Nobody Gets Fired for Picking JSON, but Maybe They Should? JSON is extremely popular but deeply flawed. This article discusses the details of JSON\u2019s design, how it\u2019s used (and misused), and how seemingly helpful\u2026","rel":"","context":"In "Javascript"","block_context":{"text":"Javascript","link":"https:\/\/monodes.com\/predaelli\/category\/javascript\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3228,"url":"https:\/\/monodes.com\/predaelli\/2017\/07\/20\/reverse-engineering-one-line-of-javascript\/","url_meta":{"origin":9813,"position":4},"title":"Reverse Engineering One Line of JavaScript","author":"Paolo Redaelli","date":"2017-07-20","format":false,"excerpt":"ASCII animations in 1 line of #JS. #JavaScript is really the assembler of the Web","rel":"","context":"In "Javascript"","block_context":{"text":"Javascript","link":"https:\/\/monodes.com\/predaelli\/category\/javascript\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2455,"url":"https:\/\/monodes.com\/predaelli\/2017\/05\/15\/2455\/","url_meta":{"origin":9813,"position":5},"title":"Semantic Versioning 2.0.0 | Semantic\u2026","author":"Paolo Redaelli","date":"2017-05-15","format":"link","excerpt":"Semantic Versioning 2.0.0 | Semantic Versioning Software versioning labels, the right way... :) Semantic Versioning 2.0.0 Summary Given a version number MAJOR.MINOR.PATCH, increment the: MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards-compatible manner, and PATCH version when you make backwards-compatible bug\u2026","rel":"","context":"In "Documentations"","block_context":{"text":"Documentations","link":"https:\/\/monodes.com\/predaelli\/category\/documentations\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts\/9813","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/comments?post=9813"}],"version-history":[{"count":0,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts\/9813\/revisions"}],"wp:attachment":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/media?parent=9813"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/categories?post=9813"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/tags?post=9813"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}