{"id":11883,"date":"2024-08-28T14:55:54","date_gmt":"2024-08-28T12:55:54","guid":{"rendered":"https:\/\/monodes.com\/predaelli\/?p=11883"},"modified":"2024-08-28T14:55:56","modified_gmt":"2024-08-28T12:55:56","slug":"python-shared-memory-in-multiprocessing","status":"publish","type":"post","link":"https:\/\/monodes.com\/predaelli\/2024\/08\/28\/python-shared-memory-in-multiprocessing\/","title":{"rendered":"Python Shared Memory in Multiprocessing"},"content":{"rendered":"\n<p><a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/\">Python Shared Memory in Multiprocessing<\/a> <\/p>\n\n\n\n<p><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"\" data-line=\"\">np_array&#039;s size=220.0MB\nWith SharedMemory: ...\nCurrent memory usage 0.11283MB; Peak: 0.156706MB\nTime elapsed: 0.99s\nNo SharedMemory: ...\nCurrent memory usage 0.026587MB; Peak: 467.558995MB\nTime elapsed: 5.48s\n<\/code><\/pre>\n\n\n\n<p>I think I shall go for shared memory! My Amiga formation years requires it!<\/p>\n\n\n\n<!--nextpage-->\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<h2 class=\"wp-block-heading\"><a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/\">Python Shared Memory in Multiprocessing<\/a><\/h2>\n\n\n\n<p>Mingze Gao, PhD   &#8211; Macquarie University June 8, 2020<\/p>\n\n\n\n<p>Python 3.8 introduced a new module <code class=\"\" data-line=\"\">multiprocessing.shared_memory<\/code> that provides shared memory for direct access across processes. My test shows that it significantly reduces the memory usage, which also speeds up the program by reducing the costs of copying and moving things around.<a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#fn1\"><sup>1<\/sup><\/a><\/p>\n\n\n\n<p><sup>1<\/sup>&nbsp;This test is performed on a 2017 12-inch MacBook with 1.3 GHz Dual-Core Intel Core i5 and 8 GB 1867 MHz LPDDR3 RAM.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Test<a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#test\"><\/a><\/h2>\n\n\n\n<p>In this test, I generated a 240MB <a href=\"https:\/\/numpy.org\/doc\/stable\/reference\/generated\/numpy.recarray.html\"><code class=\"\" data-line=\"\">numpy.recarray<\/code><\/a> from a <code class=\"\" data-line=\"\">pandas.DataFrame<\/code> with <code class=\"\" data-line=\"\">datetime<\/code>, <code class=\"\" data-line=\"\">int<\/code> and <code class=\"\" data-line=\"\">str<\/code> typed columns. I used <code class=\"\" data-line=\"\">numpy.recarray<\/code> because it can preserve the <code class=\"\" data-line=\"\">dtype<\/code> of each column, so that later I can reconstruct the same array from the buffer of shared memory.<\/p>\n\n\n\n<p>I performed a simple <code class=\"\" data-line=\"\">numpy.nansum<\/code> on the numeric column of the data using two methods. The first method uses <code class=\"\" data-line=\"\">multiprocessing.shared_memory<\/code> where the 4 spawned processes directly access the data in the shared memory. The second method passes the data to the spawned processes, which effectively means each process will have a separate copy of the data.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Test Result<\/h3>\n\n\n\n<figure class=\"wp-block-image\"><img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/mingze-gao.com\/images\/Python-SharedMemory-test.png?w=910&#038;ssl=1\" alt=\"\"\/><figcaption class=\"wp-element-caption\">Test result<\/figcaption><\/figure>\n\n\n\n<p>A quick run of <a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#test-code\">the test code below<\/a> shows that the first method based on <code class=\"\" data-line=\"\">shared_memory<\/code> uses minimal memory (peak usage is 0.33MB) and is much faster (2.09s) than the second one where the entire data is copied and passed into each process (peak memory usage of 1.8G and takes 216s). More importantly, the memory usage under the second method is consistently high.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Test Code<a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#test-code\"><\/a><\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"\" data-line=\"\">&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-1&quot;&gt;&lt;\/a&gt;from multiprocessing.shared_memory import SharedMemory\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-2&quot;&gt;&lt;\/a&gt;from multiprocessing.managers import SharedMemoryManager\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-3&quot;&gt;&lt;\/a&gt;from concurrent.futures import ProcessPoolExecutor, as_completed\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-4&quot;&gt;&lt;\/a&gt;from multiprocessing import current_process, cpu_count, Process\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-5&quot;&gt;&lt;\/a&gt;from datetime import datetime\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-6&quot;&gt;&lt;\/a&gt;import numpy as np\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-7&quot;&gt;&lt;\/a&gt;import pandas as pd\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-8&quot;&gt;&lt;\/a&gt;import tracemalloc\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-9&quot;&gt;&lt;\/a&gt;import time\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-10&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-11&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-12&quot;&gt;&lt;\/a&gt;def work_with_shared_memory(shm_name, shape, dtype):\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-13&quot;&gt;&lt;\/a&gt;    print(f&#039;With SharedMemory: {current_process()=}&#039;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-14&quot;&gt;&lt;\/a&gt;    # Locate the shared memory by its name\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-15&quot;&gt;&lt;\/a&gt;    shm = SharedMemory(shm_name)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-16&quot;&gt;&lt;\/a&gt;    # Create the np.recarray from the buffer of the shared memory\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-17&quot;&gt;&lt;\/a&gt;    np_array = np.recarray(shape=shape, dtype=dtype, buf=shm.buf)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-18&quot;&gt;&lt;\/a&gt;    return np.nansum(np_array.val)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-19&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-20&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-21&quot;&gt;&lt;\/a&gt;def work_no_shared_memory(np_array: np.recarray):\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-22&quot;&gt;&lt;\/a&gt;    print(f&#039;No SharedMemory: {current_process()=}&#039;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-23&quot;&gt;&lt;\/a&gt;    # Without shared memory, the np_array is copied into the child process\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-24&quot;&gt;&lt;\/a&gt;    return np.nansum(np_array.val)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-25&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-26&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-27&quot;&gt;&lt;\/a&gt;if __name__ == &quot;__main__&quot;:\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-28&quot;&gt;&lt;\/a&gt;    # Make a large data frame with date, float and character columns\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-29&quot;&gt;&lt;\/a&gt;    a = &#091;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-30&quot;&gt;&lt;\/a&gt;        (datetime.today(), 1, &#039;string&#039;),\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-31&quot;&gt;&lt;\/a&gt;        (datetime.today(), np.nan, &#039;abc&#039;),\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-32&quot;&gt;&lt;\/a&gt;    ] * 5000000\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-33&quot;&gt;&lt;\/a&gt;    df = pd.DataFrame(a, columns=&#091;&#039;date&#039;, &#039;val&#039;, &#039;character_col&#039;])\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-34&quot;&gt;&lt;\/a&gt;    # Convert into numpy recarray to preserve the dtypes \n<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"\" data-line=\"\">&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-35&quot;&gt;&lt;\/a&gt;    np_array = df.to_records(index=False, column_dtypes={&#039;character_col&#039;: &#039;S6&#039;})\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-36&quot;&gt;&lt;\/a&gt;    del df\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-37&quot;&gt;&lt;\/a&gt;    shape, dtype = np_array.shape, np_array.dtype\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-38&quot;&gt;&lt;\/a&gt;    print(f&quot;np_array&#039;s size={np_array.nbytes\/1e6}MB&quot;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-39&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-40&quot;&gt;&lt;\/a&gt;    # With shared memory\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-41&quot;&gt;&lt;\/a&gt;    # Start tracking memory usage\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-42&quot;&gt;&lt;\/a&gt;    tracemalloc.start()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-43&quot;&gt;&lt;\/a&gt;    start_time = time.time()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-44&quot;&gt;&lt;\/a&gt;    with SharedMemoryManager() as smm:\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-45&quot;&gt;&lt;\/a&gt;        # Create a shared memory of size np_arry.nbytes\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-46&quot;&gt;&lt;\/a&gt;        shm = smm.SharedMemory(np_array.nbytes)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-47&quot;&gt;&lt;\/a&gt;        # Create a np.recarray using the buffer of shm\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-48&quot;&gt;&lt;\/a&gt;        shm_np_array = np.recarray(shape=shape, dtype=dtype, buf=shm.buf)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-49&quot;&gt;&lt;\/a&gt;        # Copy the data into the shared memory\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-50&quot;&gt;&lt;\/a&gt;        np.copyto(shm_np_array, np_array)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-51&quot;&gt;&lt;\/a&gt;        # Spawn some processes to do some work\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-52&quot;&gt;&lt;\/a&gt;        with ProcessPoolExecutor(cpu_count()) as exe:\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-53&quot;&gt;&lt;\/a&gt;            fs = &#091;exe.submit(work_with_shared_memory, shm.name, shape, dtype)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-54&quot;&gt;&lt;\/a&gt;                  for _ in range(cpu_count())]\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-55&quot;&gt;&lt;\/a&gt;            for _ in as_completed(fs):\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-56&quot;&gt;&lt;\/a&gt;                pass\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-57&quot;&gt;&lt;\/a&gt;    # Check memory usage\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-58&quot;&gt;&lt;\/a&gt;    current, peak = tracemalloc.get_traced_memory()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-59&quot;&gt;&lt;\/a&gt;    print(f&quot;Current memory usage {current\/1e6}MB; Peak: {peak\/1e6}MB&quot;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-60&quot;&gt;&lt;\/a&gt;    print(f&#039;Time elapsed: {time.time()-start_time:.2f}s&#039;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-61&quot;&gt;&lt;\/a&gt;    tracemalloc.stop()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-62&quot;&gt;&lt;\/a&gt;\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-63&quot;&gt;&lt;\/a&gt;    # Without shared memory\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-64&quot;&gt;&lt;\/a&gt;    tracemalloc.start()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-65&quot;&gt;&lt;\/a&gt;    start_time = time.time()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-66&quot;&gt;&lt;\/a&gt;    with ProcessPoolExecutor(cpu_count()) as exe:\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-67&quot;&gt;&lt;\/a&gt;        fs = &#091;exe.submit(work_no_shared_memory, np_array)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-68&quot;&gt;&lt;\/a&gt;              for _ in range(cpu_count())]\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-69&quot;&gt;&lt;\/a&gt;        for _ in as_completed(fs):\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-70&quot;&gt;&lt;\/a&gt;            pass\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-71&quot;&gt;&lt;\/a&gt;    # Check memory usage\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-72&quot;&gt;&lt;\/a&gt;    current, peak = tracemalloc.get_traced_memory()\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-73&quot;&gt;&lt;\/a&gt;    print(f&quot;Current memory usage {current\/1e6}MB; Peak: {peak\/1e6}MB&quot;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-74&quot;&gt;&lt;\/a&gt;    print(f&#039;Time elapsed: {time.time()-start_time:.2f}s&#039;)\n&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#annotated-cell-1-75&quot;&gt;&lt;\/a&gt;    tracemalloc.stop()<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Note on Segfault<a href=\"https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#note-on-segfault\"><\/a><\/h2>\n\n\n\n<p>Warning<\/p>\n\n\n\n<p>A very important note about using <code class=\"\" data-line=\"\">multiprocessing.shared_memory<\/code>, as at June 2020, is that the <code class=\"\" data-line=\"\">numpy.ndarray<\/code> cannot have a <code class=\"\" data-line=\"\">dtype=dtype(&#039;O&#039;)<\/code>. That is, the <code class=\"\" data-line=\"\">dtype<\/code> cannot be <code class=\"\" data-line=\"\">dtype(object)<\/code>. If it is, there will be a segmentation fault when child processes try to access the shared memory and dereference it. It happens when the column contains strings.<\/p>\n\n\n\n<p>To solve this problem, you need to specify the <code class=\"\" data-line=\"\">dtype<\/code> in <code class=\"\" data-line=\"\">df.to_records()<\/code>. For example:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"\" data-line=\"\">&lt;a href=&quot;https:\/\/mingze-gao.com\/posts\/python-shared-memory-in-multiprocessing\/#cb1-1&quot;&gt;&lt;\/a&gt;np_array = df.to_records(index=False\uff0ccolumn_dtypes={&#039;character_col&#039;: &#039;S6&#039;})<\/code><\/pre>\n\n\n\n<p>Here, we specify that <code class=\"\" data-line=\"\">character_col<\/code> contains strings of length 6. If it contains Unicode, we can use <code class=\"\" data-line=\"\">&#039;U6&#039;<\/code> instead. Longer strings will then be truncated at the specified length. As such, there won\u2019t be anymore segfault<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p class=\"excerpt\">Python Shared Memory in Multiprocessing I think I shall go for shared memory! My Amiga formation years requires it!<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"https:\/\/monodes.com\/predaelli\/2024\/08\/28\/python-shared-memory-in-multiprocessing\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"federated","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[113],"tags":[],"class_list":["post-11883","post","type-post","status-publish","format-standard","hentry","category-python"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6daft-35F","jetpack-related-posts":[{"id":9066,"url":"https:\/\/monodes.com\/predaelli\/2022\/01\/14\/getting-started-with-cython-how-to-perform-1-7-billion-calculations-per-second-in-python\/","url_meta":{"origin":11883,"position":0},"title":"Getting started with Cython: How to perform >1.7 billion calculations per second in Python","author":"Paolo Redaelli","date":"2022-01-14","format":false,"excerpt":"Getting started with Cython: How to perform >1.7 billion calculations per second in Python Combine the ease of Python with the speed of C Mike HulsDec 8, 2021\u00b711 min read Cython will add an afterburner to your program (image by Oscar Sutton on Unsplash) The main advantage of Python is\u2026","rel":"","context":"In &quot;Python&quot;","block_context":{"text":"Python","link":"https:\/\/monodes.com\/predaelli\/category\/python\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":8964,"url":"https:\/\/monodes.com\/predaelli\/2021\/12\/06\/rusteiffel\/","url_meta":{"origin":11883,"position":1},"title":"RustEiffel?","author":"Paolo Redaelli","date":"2021-12-06","format":false,"excerpt":"Python 4.0 will never arrive\ud83e\udd1a\ud83d\ude14. Said by Python\u2019s creator And the reason is mainly Rust. Guido van Rossum (the Python programming language\u2019s inventor) stated in an interview a few days ago that it would be difficult for Python 4.0 to see the light of day since the programming language is\u2026","rel":"","context":"In &quot;Liberty Eiffel&quot;","block_context":{"text":"Liberty Eiffel","link":"https:\/\/monodes.com\/predaelli\/category\/eiffel\/liberty-eiffel\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2021\/12\/1Zw72DLdFFS_SoumO0Ho_Ew.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2021\/12\/1Zw72DLdFFS_SoumO0Ho_Ew.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2021\/12\/1Zw72DLdFFS_SoumO0Ho_Ew.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2021\/12\/1Zw72DLdFFS_SoumO0Ho_Ew.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":6494,"url":"https:\/\/monodes.com\/predaelli\/2020\/01\/25\/a-quick-dive-into-pythons-__slots__-noteworthy-the-journal-blog\/","url_meta":{"origin":11883,"position":2},"title":"A quick dive into Python\u2019s \u201c__slots__\u201d &#8211; Noteworthy &#8211; The Journal Blog","author":"Paolo Redaelli","date":"2020-01-25","format":false,"excerpt":"A quick dive into Python\u2019s \u201cslots\u201d - Noteworthy - The Journal Blog Or how to try mimik a strongly typed language in a weakly typed one or something like this.... A quick dive into Python\u2019s \u201c__slots__\u201d Stephen Jayakar Nov 10, 2018 \u00b7 6 min read or my first Medium article,\u2026","rel":"","context":"In &quot;Documentations&quot;","block_context":{"text":"Documentations","link":"https:\/\/monodes.com\/predaelli\/category\/documentations\/"},"img":{"alt_text":"Stephen Jayakar","src":"https:\/\/i0.wp.com\/miro.medium.com\/fit\/c\/96\/96\/2%2AvFNVadKjiDYAViJUxGzsTg.jpeg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":9674,"url":"https:\/\/monodes.com\/predaelli\/2022\/09\/28\/10-powerful-python-one-liners\/","url_meta":{"origin":11883,"position":3},"title":"10 Powerful Python One-Liners.","author":"Paolo Redaelli","date":"2022-09-28","format":false,"excerpt":"I shall Eiffellize those, one day or another: 10 Powerful Python One-Liners. Python one-liners can be just as\u2026 | by Ishaan Gupta | Sep, 2022 | Python in Plain English 10 Powerful Python One-Liners Python one-liners can be just as powerful as a long and tedious program written in another\u2026","rel":"","context":"In &quot;Agenda&quot;","block_context":{"text":"Agenda","link":"https:\/\/monodes.com\/predaelli\/category\/agenda\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2022\/09\/1jLSxNQvNqsYr02VB_wr96A.jpeg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2022\/09\/1jLSxNQvNqsYr02VB_wr96A.jpeg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2022\/09\/1jLSxNQvNqsYr02VB_wr96A.jpeg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":8612,"url":"https:\/\/monodes.com\/predaelli\/2022\/05\/30\/eiffel-rust-and-memory-management\/","url_meta":{"origin":11883,"position":4},"title":"Eiffel, Rust and memory management","author":"Paolo Redaelli","date":"2022-05-30","format":false,"excerpt":"It already passed one year almost two years since in \"Oh my lazyness!\" when I wrote I shall restart my efforts on Liberty EIffel and Monodes\u2026 \ud83d\ude41 I hope to integrate them in my current effort. While I haven't been able to integrate Eiffel in the endeavour that is keeping\u2026","rel":"","context":"In &quot;Eiffel&quot;","block_context":{"text":"Eiffel","link":"https:\/\/monodes.com\/predaelli\/category\/eiffel\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":14807,"url":"https:\/\/monodes.com\/predaelli\/2026\/01\/23\/the-xonsh-shell-python-powered-shell\/","url_meta":{"origin":11883,"position":5},"title":"The Xonsh Shell \u2014 Python-powered shell","author":"Paolo Redaelli","date":"2026-01-23","format":false,"excerpt":"The Xonsh Shell \u2014 Python-powered shell. Python shell. Python in the shell. Shell in Python. Shell and Python. Python and shell.Xonsh (sounds like \"consh\") is a modern, full-featured and cross-platform python shell. The language is a superset of Python 3.6+ with additional shell primitives that you are used to from\u2026","rel":"","context":"In &quot;Python&quot;","block_context":{"text":"Python","link":"https:\/\/monodes.com\/predaelli\/category\/python\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/monodes.com\/predaelli\/wp-content\/uploads\/sites\/4\/2026\/01\/xonsh.webp?fit=257%2C399&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts\/11883","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/comments?post=11883"}],"version-history":[{"count":0,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/posts\/11883\/revisions"}],"wp:attachment":[{"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/media?parent=11883"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/categories?post=11883"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/monodes.com\/predaelli\/wp-json\/wp\/v2\/tags?post=11883"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}