Abstract
Chapter 7 covers threading and asynchronous programming in Python. Threading is often the source of subtle bugs. Asynchronous code can be used for similar tasks, but it is an idiom that many Python programmers haven’t used because the program behaves quite differently to synchronous programming. This chapter focuses on the real-world use of concurrency to achieve a result, rather than demonstrations of a simple example or the limits of what asynchronous programming can do. The objective is working code that is usable in the real world and a thorough understanding of the trade-offs, not a stand-alone technology demonstration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
It can do other things, like detecting when a file is ready to be written to, but that’s not relevant for our needs.
- 2.
Assuming you’re using HTTP 0.9, 1.0, or 1.1. HTTP 2.0 and higher are binary protocols.
- 3.
This is a terrible assumption and would cause lots of intermittent bugs. To do this for real, we’d have to build up the response chunk by chunk.
- 4.
One reason to use PyPy is that it has a JIT compiler, making some code run faster. Compatibility with CPython isn’t 100%, mainly due to how compiled extensions are handled, but the performance of your programs may be very different under PyPy. It might be worth trying it, to see if your programs work, and if they’re faster or slower.
- 5.
Short for disassemble.
- 6.
An operation that is done in one step that cannot be split up.
- 7.
Some implementations of Python, especially those that run on an underlying virtual machine like Jython, do not implement the GIL as the VM offers the same guarantee. The overall effect is the same, but the specific details of bytecode and switching are different.
- 8.
Except for variable types that are designed for thread-safety, such as queues.
- 9.
Last In, First Out and First In, First Out.
- 10.
They also have some methods for introspecting the state of the queue, such as empty(), full(), and qsize(). It’s possible that the underlying queue will change between checking the state and the next instruction, though. These methods are only really useful when you have additional guarantees about the state of the program, so you know the queue won’t be changing.
- 11.
This is a contrived example, as there’s no reason to write a function that returns only its own argument, especially in an asynchronous way, but if we imagine that rather than returning the input, this is making a call to a web service that returns the data, this is more defensible. This is a trade-off between a useful function and something that’s more easily understood.
- 12.
With apologies to my friends Nathan and Ramon who maintain the Guillotina project, a specialist framework for very high-performance REST APIs in Python, who will advocate strongly for the benefits of asyncio in server code.
- 13.
Preemptive multitasking is when a central authority can interrupt tasks to give another task a chance to execute. In this case, the Python GIL enforces this with its switch interval. The alternative, cooperative multitasking, is the idea that tasks must voluntarily give up control to their peers. This is the basis of the name coroutine and also the reason that bugs in Windows 3.1 applications could inadvertently freeze the system so easily.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Matthew Wilkes
About this chapter
Cite this chapter
Wilkes, M. (2020). Parallelization and async. In: Advanced Python Development. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-5793-7_7
Download citation
DOI: https://doi.org/10.1007/978-1-4842-5793-7_7
Published:
Publisher Name: Apress, Berkeley, CA
Print ISBN: 978-1-4842-5792-0
Online ISBN: 978-1-4842-5793-7
eBook Packages: Professional and Applied ComputingApress Access BooksProfessional and Applied Computing (R0)