A Short and Clean Approach to Processing Iterables
Khuyen Tran
4 days ago·5 min read
Motivation
map and filter are two efficient Python methods to work with iterables. However, the code can look messy if you use both map and filter at the same time.
Image by Author
Wouldn’t be nice if you can use pipes | to apply multiple methods on an iterable like below?
Image by Author
The library Pipe allows you to do exactly that.
What is Pipe?
Pipe is a Python library that enables you to use pipes in Python. A pipe (|) passes the results of one method to another method.
I like Pipe becauseit makes my code look cleaner when applying multiple methods to a Python iterable. Since Pipe only provides a few methods, it is also very easy to learn Pipe. In this article, I will show you some methods I found the most useful.
To install Pipe, type:
pip install pipe
Where — Filter Elements in an Iterable
Similar to SQL, Pipe’s where method can also be used to filter elements in an iterable.
Image by Author
Select — Apply a Function to an Iterable
The select method is similar to the map method. select applies a method to each element of an iterable.
In the code below, I use select to multiply each element in the list by 2.
Image by Author
Now, you might wonder: Why do we need the methods where and select if they have the same functionalities as map and filter ?
It is because you can insert one method after another method using pipes. As a result, using pipes removes nested parentheses and makes the code more readable.
Image by Author
Unfold Iterables
chain — Chain a Sequence of Iterables
It can be a pain to work with a nested iterable. Luckily, you can use chain to chain a sequence of iterables.
Image by Author
Even though the iterable is less nested after applying chain, we still have a nested list. To deal with a deeply nested list, we can use traverse instead.
traverse — Recursively Unfold Iterables
The traverse method can be used to recursively unfold iterables. Thus, you can use this method to turn a deeply nested list into a flat list.
Image by Author
Let’s integrate this method with the select method to get the values of a dictionary and flatten the list.
Image by Author
Pretty cool!
Group Elements in a List
Sometimes, it might be useful to group elements in a list using a certain function. That could be easily done with the groupby method.
To see how this method works, let’s turn a list of numbers into a dictionary that groups numbers based on whether they are even or odd.
Image by Author
In the code above, we use groupby to group numbers into theEven group and theOdd group. The output after applying this method looks like the below:
[('Even', <itertools._grouper at 0x7fbea8030550>),
('Odd', <itertools._grouper at 0x7fbea80309a0>)]
Next, we use select to turn a list of tuples into a list of dictionaries whose keys are the first elements in the tuples and values are the second elements in the tuples.
Cool! To get only the values that are greater than 2, we can add the where method inside the select method:
Image by Author
Note that there are no longer 2 and 1 in the outputs.
dedup — Deduplicate Values Using a Key
The dedup method removes duplicates in a list.
Image by Author
That might not sound interesting since the set method can do the same thing. However, this method is more flexible since it enables you to get unique elements using a key.
For example, you can use this method to get a unique element that is smaller than 5 and another unique element that is larger than or equal to 5.
Image by Author
Now, let’s combine this method with select and where to get the values of a dictionary that has duplicated keys and None values.
Image by Author
In the code above, we:
remove items with the same name
get the values of count
only choose the values that are integers.
Within a few lines of code, we can apply multiple methods to an iterable while still keeping the code clean. Pretty cool, isn’t it?
Conclusion
Congratulations! You have just learned how to use pipe to keep your code clean and short. I hope this article will give you the knowledge to turn complicated operations on an iterable into one simple line of code.
Feel free to play and fork the source code of this article here:
Data-science/pipe.ipynb at master · khuyentran1401/Data-science
Collection of useful data science topics along with code and articles – Data-science/pipe.ipynb at master ·…
github.com
I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.
Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these:
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptRead More
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.