DjangoCon US 2022: Day 3
Async Django: The practical guide you’ve been awaiting for.
Async is exciting, but be aware that using async will make your application more complex.
import asyncio
: Python’s implementation of an async runtime that Django uses.import asyncio async def foo(): ... await bar() async def main(): tasks = [asyncio.create_task(foo()) for range(5)] # Important! Wait for all tasks to complete. await asyncio.gather(*tasks) asyncio.run(main())
Can you use this for background tasks?
- Kind off…if it fails, there is no built-in statuses, retires, or error handling.
- It depends…on how you are running Django (WSGI or ASGI). With WSGI, hitting an async view spins up an event loop, but will disappear when then view exits (like the example above).
Aggregate Views (what you did before GraphQL):
import httpx import asyncio async def aggregate_view(): # pseudo code async with httpx.AsyncClient() as client: response_a, response_b = asyncio.gather( client.get(view_a_url), client.get(view_b_url), ) return JsonResponse({ 'response_a': response_a.json(), 'response_b': response_b.json(), })
Chat App (four ways):
- Polling (HTMX): Simple, but doesn’t scale, isn’t responsive, and can lead to self DDOS.
- Long Polling (HTMX + Channels): Responsiveness, but creates a lot connections.
- Server-Sent Events (HTMX + Channels): Better, keeps the connection open.
- WebSockets (HTMX + Channels): Also keeps the connection open, but allows two-way communication.
Why large Django projects need a data (prefetching) layer
“DRY isn’t helpful if you need to be careful.”
Django REST Framework loves DRY, but has high change amplification as a side effect.
Change Amplification: The expected number of places in the codebase that needs to be modified during an atomic change to the software.
An example of this in DRF is needing to prefetch in serializers.
class MovieSerializer(ModelSerializer): # data that requires prefetching # You have to remeber to prefetch for each view using MovieSerializer. class MovieListView(ListAPIView): queryset = Movie.objects.prefetch_related("directors") serializer_class = MovieSerializer class MovieDetailView(ListAPIView): queryset = Movie.objects.prefetch_related("directors") serializer_class = MovieSerializer
You could do one of two things to combat this:
- Use non-DRF serializers that are very explicit about prefetching.
django-virtual-models
.
- Or keep DRF, but:
- Warn about missing prefetches for each serializer.
- Automatically run necessary prefetches.
- Automatically prevent unnecessary ones.
- Keep serializer nesting support.
- Keep
SerializerMethodField
support.
TLDR: Use tools to be explicit about the data you expect from the DB. Otherwise you’ll suffer from performance regressions and your read logic will break frequently.
A pythonic full-text search
- PostgreSQL added full text search in 2008.
- Django added full text search in 2016.
- Paolo has a great article outlining the various search features.
Home on the range with Django - getting comfortable with ranges and range fields
- Ranges are everywhere!
- Support for ranges was added for PostgreSQL Django in 2015.
- The typical approach is to us “start” and “stop” fields, but that gets complicated fast.
- The DB doesn’t know these two fields are related.
- You have to manually add bounding/validation/constraint logic.
- Queries can get weird/complicated.
- Ranges fix all this!
- 1 field that stores lower, upper, and boundary information.
- Automatically validates/constrains values.
- Easy/Intuitive queries.
- Ranges (Math):
- Ranges = Intervals, Inclusive = Closed, Exclusive = Open
- (exclusive, exclusive) - (1, 3) - 2
- [inclusive, exclusive) - [1, 3) - 1, 2
The default for Django ranges!
- (exclusive, inclusive] - (1, 3] - 2, 3
- [inclusive, inclusive] - [1, 3] - 1, 2, 3
- There are lots of very useful query filters!
__overlap
,__contains
,__adjacent_to
,__fully_lt/gt
, etc.
- Django supports a number of different value types via
pysycopg2.extra
.- You can also create custom types! Example: IP ranges!
- Pitfalls
- Limited Django Admin support.
- You have to use
Cast
when usingF()
. - You have to use
Lower
andUpper
database functions to access values.
- Resources
psycopg2.extras
django-range-merge
django-generate-series