John Reese - Thinking Outside the GIL with AsyncIO and Multiprocessing - PyCon 2018
Why Stack Overflow recommends it
Stack Overflow users link it when answering questions about mixing IO-bound and CPU-bound work in Python and about choosing between threads, asyncio and multiprocessing.
Recommended by 3 people across 3 different months, 2021 to 2022.
What it is
A 30-minute PyCon 2018 talk by John Reese on redesigning a multithreaded Python application that was limited by the GIL. He combines asyncio and multiprocessing, using only the standard library, to build something that scales with demand. No prior asyncio or multiprocessing experience is assumed.
What people said
Strong rec rawrex · 2021-05 · Stack Overflow · on “How to run multiple threads within a for loop?”
If you have IO-bound [LINK] tasks here, such as access to storage and retrieving images, you can benefit from incorporating asyncio [LINK] features, in particular the aiofiles [LINK] module. Meanwhile, as was mentioned, the actual processing (the CPU-bound [LINK] part of the program) may benefit from using the multiprocessing [LINK] module, which overcomes limitations of the GIL [LINK] . For how to cooperate multiprocessing and asycnio , check this great talk [this video] by John Reese [LINK] ! Good luck!
Recommends Christo Goosen · 2022-01 · Stack Overflow · on “multiprocessing vs multithreading vs asyncio”
Pleasure. Yeah I happened to watch a youtube video where the author described its use. Was very insightful as it explained the purpose well. Definitely not a magic bullet and probably not the use case for everyone. Would perhaps be at the core of web server or low level network application. Basically just churn through as many requests as CPUs and the multiple event loops can handle. [this video]
Recommends lambdakappatheta · 2022-06 · Stack Overflow · on “What is a "thread" (really)?”
Just as a process represents a virtual computer, the thread abstraction represents a virtual processor. [LINK] So threads are an abstraction . Abstractions reduce complexity. Thus, the first question is what problem threads solve. The second question is how they can be implemented. As to the first question: Threads make implementing multitasking easier. The main idea behind this is that multitasking is unnecessary if every task can be assigned to a unique worker. Actually, for the time being, it's fine to generalize the definition even further and say that the thread abstraction represents a virtual worker. Now, imagine you have a robot that you want to give multiple tasks. Unfortunately, it