Another interesting article to “convert to Eiffel”…. It’s Time to Say Goodbye to These Obsolete Python Libraries

It’s Time to Say Goodbye to These Obsolete Python Libraries

Martin

Jul 20, 2022

With every Python release, there are new modules being added and new and better ways of doing things get introduced. We all get used to using the good old Python libraries and to certain way of doing things, but it’s time upgrade and make use of the new and improved modules and their features.

Pathlib

pathlib is definitely one of the bigger, recent additions to Python’s standard library. It’s been part of standard library since Python 3.4, yet a lot of people still use os module for filesystem operations.

pathlib has however many advantages over old os.path – while os module represents paths in raw string format, pathlib uses object-oriented style, which makes it more readable and natural to write:

<span class="token keyword">from</span> pathlib <span class="token keyword">import</span> Path
<span class="token keyword">import</span> os<span class="token punctuation">.</span>path

<span class="token comment"># Old, Unreadable</span>
two_dirs_up <span class="token operator">=</span> os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>dirname<span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>dirname<span class="token punctuation">(</span>os<span class="token punctuation">.</span>path<span class="token punctuation">.</span>abspath<span class="token punctuation">(</span>__file__<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
<span class="token comment"># New, Readable</span>
two_dirs_up <span class="token operator">=</span> Path<span class="token punctuation">(</span>__file__<span class="token punctuation">)</span><span class="token punctuation">.</span>resolve<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>parent<span class="token punctuation">.</span>parent

The fact that paths are treated as objects rather than strings also makes it possible to create the object once and then lookup its attributes or make operations on it:

readme <span class="token operator">=</span> Path<span class="token punctuation">(</span><span class="token string">"README.md"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>resolve<span class="token punctuation">(</span><span class="token punctuation">)</span>

<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Absolute path: </span><span class="token interpolation"><span class="token punctuation">{</span>readme<span class="token punctuation">.</span>absolute<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Absolute path: /home/martin/some/path/README.md</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"File name: </span><span class="token interpolation"><span class="token punctuation">{</span>readme<span class="token punctuation">.</span>name<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># File name: README.md</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Path root: </span><span class="token interpolation"><span class="token punctuation">{</span>readme<span class="token punctuation">.</span>root<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Path root: /</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Parent directory: </span><span class="token interpolation"><span class="token punctuation">{</span>readme<span class="token punctuation">.</span>parent<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Parent directory: /home/martin/some/path</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"File extension: </span><span class="token interpolation"><span class="token punctuation">{</span>readme<span class="token punctuation">.</span>suffix<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># File extension: .md</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Is it absolute: </span><span class="token interpolation"><span class="token punctuation">{</span>readme<span class="token punctuation">.</span>is_absolute<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Is it absolute: True</span>

The one feature that I love the most about pathlib though, is possibility to use the / (“division”) operator to join paths:

<span class="token comment"># Operators:</span>
etc <span class="token operator">=</span> Path<span class="token punctuation">(</span><span class="token string">'/etc'</span><span class="token punctuation">)</span>

joined <span class="token operator">=</span> etc <span class="token operator">/</span> <span class="token string">"cron.d"</span> <span class="token operator">/</span> <span class="token string">"anacron"</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Exists? - </span><span class="token interpolation"><span class="token punctuation">{</span>joined<span class="token punctuation">.</span>exists<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Exists? - True</span>

This makes handling of paths so easy and really is a chef’s kiss 👌.

With that said, it’s important to note that pathlib is only replacement for os.path and not a whole os module. It however includes also functionality from glob module, so if you’re used to using os.path in combination with glob.glob, then you can just forget that those 2 exist.

In the above snippets we presented some handy path manipulations and object attributes, but pathlib also includes all the methods that you’re used to from os.path, such as:

<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Working directory: </span><span class="token interpolation"><span class="token punctuation">{</span>Path<span class="token punctuation">.</span>cwd<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>  <span class="token comment"># same as os.getcwd()</span>
<span class="token comment"># Working directory: /home/martin/some/path</span>
Path<span class="token punctuation">.</span>mkdir<span class="token punctuation">(</span>Path<span class="token punctuation">.</span>cwd<span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token string">"new_dir"</span><span class="token punctuation">,</span> exist_ok<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span>  <span class="token comment"># same as os.makedirs()</span>
<span class="token keyword">print</span><span class="token punctuation">(</span>Path<span class="token punctuation">(</span><span class="token string">"README.md"</span><span class="token punctuation">)</span><span class="token punctuation">.</span>resolve<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>  <span class="token comment"># same as os.path.abspath()</span>
<span class="token comment"># /home/martin/some/path/README.md</span>
<span class="token keyword">print</span><span class="token punctuation">(</span>Path<span class="token punctuation">.</span>home<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>  <span class="token comment"># same as os.path.expanduser()</span>
<span class="token comment"># /home/martin</span>

For full mapping of os.path functions to new ones in pathlib see docs .

For more examples of how great pathlib is, check out nice write-up by Trey Hunner.

Secrets

Speaking of os module, another part of it that you should stop using is os.urandom. Instead, you should use new secrets module available since Python 3.6:

<span class="token comment"># Old:</span>
<span class="token keyword">import</span> os

length <span class="token operator">=</span> <span class="token number">64</span>

value <span class="token operator">=</span> os<span class="token punctuation">.</span>urandom<span class="token punctuation">(</span>length<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Bytes: </span><span class="token interpolation"><span class="token punctuation">{</span>value<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Bytes: b'\xfa\xf3...\xf2\x1b\xf5\xb6'</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Hex: </span><span class="token interpolation"><span class="token punctuation">{</span>value<span class="token punctuation">.</span><span class="token builtin">hex</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Hex: faf3cc656370e31a938e7...33d9b023c3c24f1bf5</span>

<span class="token comment"># New:</span>
<span class="token keyword">import</span> secrets

value <span class="token operator">=</span> secrets<span class="token punctuation">.</span>token_bytes<span class="token punctuation">(</span>length<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Bytes: </span><span class="token interpolation"><span class="token punctuation">{</span>value<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Bytes: b'U\xe9n\x87...\x85>\x04j:\xb0'</span>
value <span class="token operator">=</span> secrets<span class="token punctuation">.</span>token_hex<span class="token punctuation">(</span>length<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Hex: </span><span class="token interpolation"><span class="token punctuation">{</span>value<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Hex: fb5dd85e7d73f7a08b8e3...4fd9f95beb08d77391</span>

Using os.urandom isn’t actually the problem here though, the reason secrets module got introduced is because people were using random module for generating passwords and such, even though random module doesn’t produce cryptographically safe tokens.

As per docs, random module should not be used for security purposes. You should use either secrets or os.urandom, but the secrets module is definitely preferable, considering that it’s newer and includes some utility/convenience methods for hexadecimal tokens as well as URL safe tokens.

Zoneinfo

Until Python 3.9, there wasn’t builtin library for timezone manipulation, so everyone was using pytz, but now we have zoneinfo in standard library, so it’s time to switch!

<span class="token keyword">from</span> datetime <span class="token keyword">import</span> datetime
<span class="token keyword">import</span> pytz  <span class="token comment"># pip install pytz</span>

dt <span class="token operator">=</span> datetime<span class="token punctuation">(</span><span class="token number">2022</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">,</span> <span class="token number">4</span><span class="token punctuation">)</span>
nyc <span class="token operator">=</span> pytz<span class="token punctuation">.</span>timezone<span class="token punctuation">(</span><span class="token string">"America/New_York"</span><span class="token punctuation">)</span>

localized <span class="token operator">=</span> nyc<span class="token punctuation">.</span>localize<span class="token punctuation">(</span>dt<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Datetime: </span><span class="token interpolation"><span class="token punctuation">{</span>localized<span class="token punctuation">}</span></span><span class="token string">, Timezone: </span><span class="token interpolation"><span class="token punctuation">{</span>localized<span class="token punctuation">.</span>tzname<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">, TZ Info: </span><span class="token interpolation"><span class="token punctuation">{</span>localized<span class="token punctuation">.</span>tzinfo<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>

<span class="token comment"># New:</span>
<span class="token keyword">from</span> zoneinfo <span class="token keyword">import</span> ZoneInfo

nyc <span class="token operator">=</span> ZoneInfo<span class="token punctuation">(</span><span class="token string">"America/New_York"</span><span class="token punctuation">)</span>
localized <span class="token operator">=</span> datetime<span class="token punctuation">(</span><span class="token number">2022</span><span class="token punctuation">,</span> <span class="token number">6</span><span class="token punctuation">,</span> <span class="token number">4</span><span class="token punctuation">,</span> tzinfo<span class="token operator">=</span>nyc<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Datetime: </span><span class="token interpolation"><span class="token punctuation">{</span>localized<span class="token punctuation">}</span></span><span class="token string">, Timezone: </span><span class="token interpolation"><span class="token punctuation">{</span>localized<span class="token punctuation">.</span>tzname<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">, TZ Info: </span><span class="token interpolation"><span class="token punctuation">{</span>localized<span class="token punctuation">.</span>tzinfo<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Datetime: 2022-06-04 00:00:00-04:00, Timezone: EDT, TZ Info: America/New_York</span>

The datetime module delegates all timezone manipulation to abstract base class datetime.tzinfo. This abstract base class needs a concrete implementation – before introducing this module that would most likely come from pytz. Now that we have zoneinfo in standard library we can use that instead.

Using zoneinfo however has one caveat – it assumes that there’s time zone data available on the system, which is the case on UNIX systems. If your system doesn’t have timezone data though, then you should use tzdata package which is a first-party library maintained by the CPython core developers, which contains IANA time zone database.

Dataclasses

An important addition to Python 3.7 was dataclasses package which is a replacement for namedtuple.

You might be wondering why would you need to replace namedtuple? So, these are some reasons why you should consider switching to dataclasses:

Can be mutable,
By default provides __repr__, __eq__, __init__, __hash__ magic methods,
Allows to specify default values,
Supports inheritance.

Additionally, dataclasses also support __frozen__ and __slots__ (from 3.10) attributes to give feature parity with named tuples.

And switching really shouldn’t be too difficult, as you only need to change the definitions:

<span class="token comment"># Old:</span>
<span class="token comment"># from collections import namedtuple</span>
<span class="token keyword">from</span> typing <span class="token keyword">import</span> NamedTuple
<span class="token keyword">import</span> sys

User <span class="token operator">=</span> NamedTuple<span class="token punctuation">(</span><span class="token string">"User"</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token punctuation">(</span><span class="token string">"name"</span><span class="token punctuation">,</span> <span class="token builtin">str</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"surname"</span><span class="token punctuation">,</span> <span class="token builtin">str</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token string">"password"</span><span class="token punctuation">,</span> <span class="token builtin">bytes</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span>

u <span class="token operator">=</span> User<span class="token punctuation">(</span><span class="token string">"John"</span><span class="token punctuation">,</span> <span class="token string">"Doe"</span><span class="token punctuation">,</span> <span class="token string">b'tfeL+uD...\xd2'</span><span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Size: </span><span class="token interpolation"><span class="token punctuation">{</span>sys<span class="token punctuation">.</span>getsizeof<span class="token punctuation">(</span>u<span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Size: 64</span>

<span class="token comment"># New:</span>
<span class="token keyword">from</span> dataclasses <span class="token keyword">import</span> dataclass

<span class="token decorator annotation punctuation">@dataclass</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token keyword">class</span> <span class="token class-name">User</span><span class="token punctuation">:</span>
    name<span class="token punctuation">:</span> <span class="token builtin">str</span>
    surname<span class="token punctuation">:</span> <span class="token builtin">str</span>
    password<span class="token punctuation">:</span> <span class="token builtin">bytes</span>

u <span class="token operator">=</span> User<span class="token punctuation">(</span><span class="token string">"John"</span><span class="token punctuation">,</span> <span class="token string">"Doe"</span><span class="token punctuation">,</span> <span class="token string">b'tfeL+uD...\xd2'</span><span class="token punctuation">)</span>

<span class="token keyword">print</span><span class="token punctuation">(</span>u<span class="token punctuation">)</span>
<span class="token comment"># User(name='John', surname='Doe', password=b'tfeL+uD...\xd2')</span>

<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Size: </span><span class="token interpolation"><span class="token punctuation">{</span>sys<span class="token punctuation">.</span>getsizeof<span class="token punctuation">(</span>u<span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">, </span><span class="token interpolation"><span class="token punctuation">{</span>sys<span class="token punctuation">.</span>getsizeof<span class="token punctuation">(</span>u<span class="token punctuation">)</span> <span class="token operator">+</span> sys<span class="token punctuation">.</span>getsizeof<span class="token punctuation">(</span><span class="token builtin">vars</span><span class="token punctuation">(</span>u<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Size: 48, 152</span>

In the above code we also included a size comparison, as that’s one of the bigger differences between the namedtuple and dataclasses. As you can see, named tuples have significantly smaller size, which is due to dataclasses using dict to represent attributes.

As for the speed comparison, the access time for attributes should be mostly the same, or not significant enough to matter unless you plan to create millions of instances:

<span class="token keyword">import</span> timeit

setup <span class="token operator">=</span> <span class="token triple-quoted-string string">'''
from typing import NamedTuple
User = NamedTuple("User", [("name", str), ("surname", str), ("password", bytes)])
u = User("John", "Doe", b'')
'''</span>

<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Access speed: </span><span class="token interpolation"><span class="token punctuation">{</span><span class="token builtin">min</span><span class="token punctuation">(</span>timeit<span class="token punctuation">.</span>repeat<span class="token punctuation">(</span><span class="token string">'u.name'</span><span class="token punctuation">,</span> setup<span class="token operator">=</span>setup<span class="token punctuation">,</span> number<span class="token operator">=</span><span class="token number">10000000</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Access speed: 0.16838401100540068</span>

setup <span class="token operator">=</span> <span class="token triple-quoted-string string">'''
from dataclasses import dataclass

@dataclass(slots=True)
class User:
    name: str
    surname: str
    password: bytes

u = User("John", "Doe", b'')
'''</span>

<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Access speed: </span><span class="token interpolation"><span class="token punctuation">{</span><span class="token builtin">min</span><span class="token punctuation">(</span>timeit<span class="token punctuation">.</span>repeat<span class="token punctuation">(</span><span class="token string">'u.name'</span><span class="token punctuation">,</span> setup<span class="token operator">=</span>setup<span class="token punctuation">,</span> number<span class="token operator">=</span><span class="token number">10000000</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>
<span class="token comment"># Access speed: 0.17728697300481144</span>

If the above persuaded you switch to dataclasses, but you’re stuck in 3.6 or earlier you can grab a backport from https://pypi.org/project/dataclasses/.

Conversely, if you don’t want to switch and really want to use named tuples for some reason, then you should at very least NamedTuple from typing module instead of the one from collections:

<span class="token comment"># Bad:</span>
<span class="token keyword">from</span> collections <span class="token keyword">import</span> namedtuple
Point <span class="token operator">=</span> namedtuple<span class="token punctuation">(</span><span class="token string">"Point"</span><span class="token punctuation">,</span> <span class="token punctuation">[</span><span class="token string">"x"</span><span class="token punctuation">,</span> <span class="token string">"y"</span><span class="token punctuation">]</span><span class="token punctuation">)</span>

<span class="token comment"># Better:</span>
<span class="token keyword">from</span> typing <span class="token keyword">import</span> NamedTuple
<span class="token keyword">class</span> <span class="token class-name">Point</span><span class="token punctuation">(</span>NamedTuple<span class="token punctuation">)</span><span class="token punctuation">:</span>
    x<span class="token punctuation">:</span> <span class="token builtin">float</span>
    y<span class="token punctuation">:</span> <span class="token builtin">float</span>

Finally, if you don’t use either namedtuple nor dataclasses you might want to consider going straight to Pydantic.

Proper Logging

This isn’t a recent addition to standard library, but it bears repeating – you should use proper logging instead of print statements. It’s fine to use print if you’re debugging an issue locally, but for any production-ready program that will run without user intervention, proper logging is a must.

Especially considering that setting up Python logging is as easy as:

<span class="token keyword">import</span> logging
logging<span class="token punctuation">.</span>basicConfig<span class="token punctuation">(</span>
    filename<span class="token operator">=</span><span class="token string">'application.log'</span><span class="token punctuation">,</span>
    level<span class="token operator">=</span>logging<span class="token punctuation">.</span>WARNING<span class="token punctuation">,</span>
    <span class="token builtin">format</span><span class="token operator">=</span><span class="token string">'[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s'</span><span class="token punctuation">,</span>
    datefmt<span class="token operator">=</span><span class="token string">'%H:%M:%S'</span>
<span class="token punctuation">)</span>

logging<span class="token punctuation">.</span>error<span class="token punctuation">(</span><span class="token string">"Some serious error occurred."</span><span class="token punctuation">)</span>
<span class="token comment"># [12:52:35] {<stdin>:1} ERROR - Some serious error occurred.</span>
logging<span class="token punctuation">.</span>warning<span class="token punctuation">(</span><span class="token string">'Some warning.'</span><span class="token punctuation">)</span>
<span class="token comment"># [12:52:35] {<stdin>:1} WARNING - Some warning.</span>

Just the simple configuration above will give you superior debugging experience in comparison to print statements. On top of that you can further customize the logging library to log to different places, change log levels, automatically rotate logs, etc. For examples on how to set up all of that see my previous article Ultimate Guide to Python Debugging.

f-strings

Python includes quite a few ways to format strings including C-style formatting, f-strings, template strings or .format function. One of them – f-strings – the formatted string literals – are just superior, though. They’re more natural to write, more readable, and the fastest of the previously mentioned options.

Therefore, I think there’s no point arguing or explaining why you should use them. There are however a couple cases where f-strings cannot be used:

Only reason to ever use % formatting is for logging:

<span class="token keyword">import</span> logging

things <span class="token operator">=</span> <span class="token string">"something happened..."</span>

logger <span class="token operator">=</span> logging<span class="token punctuation">.</span>getLogger<span class="token punctuation">(</span>__name__<span class="token punctuation">)</span>
logger<span class="token punctuation">.</span>error<span class="token punctuation">(</span><span class="token string">"Message: %s"</span><span class="token punctuation">,</span> things<span class="token punctuation">)</span>  <span class="token comment"># Evaluated inside logger method</span>
logger<span class="token punctuation">.</span>error<span class="token punctuation">(</span><span class="token string-interpolation"><span class="token string">f"Message: </span><span class="token interpolation"><span class="token punctuation">{</span>things<span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span>  <span class="token comment"># Evaluated immediately</span>

In the example above if you use f-strings the expression would be evaluated immediately, while with C-style formatting, substitution will be deferred until it’s actually needed. This is important for grouping of messages, where all messages with the same template can be recorded as one. That doesn’t work with f-strings, because the template is populated with data before it’s passed to logger.

Also, there are things that f-strings simply cannot do. For example populating template at runtime – that is, dynamic formatting – that’s the reason f-strings are referred to as literal string formatting:

<span class="token comment"># Dynamically set both the template and its parameters</span>
<span class="token keyword">def</span> <span class="token function">func</span><span class="token punctuation">(</span>tpl<span class="token punctuation">:</span> <span class="token builtin">str</span><span class="token punctuation">,</span> param1<span class="token punctuation">:</span> <span class="token builtin">str</span><span class="token punctuation">,</span> param2<span class="token punctuation">:</span> <span class="token builtin">str</span><span class="token punctuation">)</span> <span class="token operator">-</span><span class="token operator">></span> <span class="token builtin">str</span><span class="token punctuation">:</span>
    <span class="token keyword">return</span> tpl<span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>param<span class="token operator">=</span>param1<span class="token punctuation">,</span> param2<span class="token operator">=</span>param2<span class="token punctuation">)</span>

some_template <span class="token operator">=</span> <span class="token string">"First template: {param1}, {param2}"</span>
another_template <span class="token operator">=</span> <span class="token string">"Other template: {param1} and {param2}"</span>

<span class="token keyword">print</span><span class="token punctuation">(</span>func<span class="token punctuation">(</span>some_template<span class="token punctuation">,</span> <span class="token string">"Hello"</span><span class="token punctuation">,</span> <span class="token string">"World"</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span>func<span class="token punctuation">(</span>another_template<span class="token punctuation">,</span> <span class="token string">"Hello"</span><span class="token punctuation">,</span> <span class="token string">"Python"</span><span class="token punctuation">)</span><span class="token punctuation">)</span>

<span class="token comment"># Dynamically reuse same template with different parameters.</span>
inputs <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token string">"Hello"</span><span class="token punctuation">,</span> <span class="token string">"World"</span><span class="token punctuation">,</span> <span class="token string">"!"</span><span class="token punctuation">]</span>
template <span class="token operator">=</span> <span class="token string">"Here's some dynamic value: {value}"</span>

<span class="token keyword">for</span> value <span class="token keyword">in</span> inputs<span class="token punctuation">:</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>template<span class="token punctuation">.</span><span class="token builtin">format</span><span class="token punctuation">(</span>value<span class="token operator">=</span>value<span class="token punctuation">)</span><span class="token punctuation">)</span>

Bottom line is though, use f-strings wherever possible because they’re more readable and more performant, be aware though that there are cases where other formatting style are still preferred and/or necessary.

Tomllib

TOML is widely used configuration format and is especially important to Python’s tooling and ecosystem, because if its usage for pyproject.toml configuration files. Until now, you’d have to use external libraries to manage TOML files, but starting with Python 3.11, there will be builtin library named tomllib which is based on tomli package.

So, as soon as you switch to Python 3.11, you should get into habit of using import tomllib instead of import tomli. It’s one less dependency to worry about!

<span class="token comment"># import tomli as tomllib</span>
<span class="token keyword">import</span> tomllib

<span class="token keyword">with</span> <span class="token builtin">open</span><span class="token punctuation">(</span><span class="token string">"pyproject.toml"</span><span class="token punctuation">,</span> <span class="token string">"rb"</span><span class="token punctuation">)</span> <span class="token keyword">as</span> f<span class="token punctuation">:</span>
    config <span class="token operator">=</span> tomllib<span class="token punctuation">.</span>load<span class="token punctuation">(</span>f<span class="token punctuation">)</span>
    <span class="token keyword">print</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span>
    <span class="token comment"># {'project': {'authors': [{'email': 'contact@martinheinz.dev',</span>
    <span class="token comment">#                           'name': 'Martin Heinz'}],</span>
    <span class="token comment">#              'dependencies': ['flask', 'requests'],</span>
    <span class="token comment">#              'description': 'Example Package',</span>
    <span class="token comment">#              'name': 'some-app',</span>
    <span class="token comment">#              'version': '0.1.0'}}</span>

toml_string <span class="token operator">=</span> <span class="token triple-quoted-string string">"""
[project]
name = "another-app"
description = "Example Package"
version = "0.1.1"
"""</span>

config <span class="token operator">=</span> tomllib<span class="token punctuation">.</span>loads<span class="token punctuation">(</span>toml_string<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span>config<span class="token punctuation">)</span>
<span class="token comment"># {'project': {'name': 'another-app', 'description': 'Example Package', 'version': '0.1.1'}}</span>

Setuptools

Last one is more of a deprecation notice:

> As Distutils is deprecated, any usage of functions or objects from distutils is similarly discouraged, and Setuptools aims to replace or deprecate all such uses.

It’s time to say goodbye to distutils package and switch to setuptools. setuptools docs provide guidance on how you should replace usages of distutils. Apart from that, also the PEP 632 provides migration advice for parts of distutils not covered by setuptools.

Conclusion

Every new Python release brings new features, so I’d recommend checking “New Modules”, “Deprecated Modules” and “Removed Modules” sections in Python release notes, which is a great way to stay up to date with major changes to Python’s standard library. This way you can continuously incorporate new features and best practices into your projects.

Now you might think that making all these changes and upgrades would require a lot of effort. In reality, you might be able to run pyupgrade on your project and upgrade the syntax to the latest Python version automatically where possible.