In the world of web development, performance is everything. As more users flock to web applications, the demand for fast and efficient APIs has grown as well. One way to achieve this is through asynchronous programming for web development, which can handle multiple requests simultaneously and improve overall performance.
In this article, we’ll explore how to use asynchronous web development in Django to build high-performance APIs. Specifically, we’ll be using two powerful libraries — Django-Ninja and Django-Ninja-Extra — to help us achieve our goals.
Django-Ninja is a popular web framework that allows developers to quickly build APIs with minimal code. It’s built on top of Django and supports asynchronous views, which makes it a great choice for building high-performance APIs.
Django-Ninja-Extra, on the other hand, is a powerful library that extends Django-Ninja’s functionality by adding support for input and output validation, automatic API documentation generation, and more. With Django-Ninja-Extra, you can build even more robust APIs with less code.
New in Django
There are some exciting new features that allow developers to take advantage of asynchronous web development. One of the most significant additions is the ability to run ORM queries asynchronously.
With some exceptions, Django can now run ORM queries asynchronously, which means that developers can take advantage of the performance benefits of asynchronous programming while still using the powerful Django ORM. For example, you can now use an async for loop to iterate over a queryset and await database operations:
async for author in Author.objects.filter(name__startswith="A"):
book = await author.books.afirst()
In addition to queryset methods, Django also supports some asynchronous model methods that use the database. For example:
async def make_book(*args, **kwargs):
book = Book(...)
await book.asave(using="secondary")
async def make_book_with_tags(tags, *args, **kwargs):
book = await Book.objects.acreate(...)
await book.tags.aset(tags)It’s worth noting that transactions do not yet work in async mode. If you need transaction behavior, it’s recommended to write that piece of code as a single synchronous function and call it using sync_to_async().
These new features in Django 4.1 or newer allow developers to build faster and more efficient APIs using asynchronous programming for web development. In combination with powerful libraries like Django-Ninja and Django-Ninja-Extra, developers can take their Django API development to the next level. In the following sections, we’ll dive into how you can use these new features to build high-performance APIs.
Django Ninja — Fast Django REST Framework
Django Ninja is a web framework for building APIs with Django and Python 3.6+ type hints.
Django Ninja Extra
The Django Ninja Extra package offers a convenient, class-based approach to quickly building and configuring high-performance APIs. Utilizing the core features of Django Ninja, it allows for speedy development without sacrificing performance.”
Setting Up Your Project
Before we start building our API, we need to set up our project. Here are the steps to create a Django project and start an app:
1. To install Django, you can use pip:
pip install django2. To create a new Django project named “project”, you can run the following command:
django-admin startproject project3. To start a new Django app named “demo_app”, you can run the following command:
python manage.py startapp demo_appNow you have a new Django project named “project” and a new app named “demo_app” that you can use to start building your API using Django-Ninja and Django-Ninja-Extra.
Setting Up Django-Ninja-Extra
1. To use Django-Ninja-Extra, you will need to install it first. You can install it using pip:
pip install django-ninja-extra2. After installation, add ninja_extra to your INSTALLED_APPS
### project/settings.py
INSTALLED_APPS = [
...,
'ninja_extra',
]3. In your django project next to urls.py create new api.py file:
### project/api.py
from ninja_extra import NinjaExtraAPI
api = NinjaExtraAPI(
version="1.0.0",
title="Project API",
)4. Now go to urls.py and add the following:
### project/urls.py
from django.contrib import admin
from django.urls import path
from project.api import api
urlpatterns = [
path('admin/', admin.site.urls),
path('api/', api.urls), # <--- Register API urls
]5. Run the migrations:
python manage.py migrate6. Now run the server:
python manage.py runserver7. Navigate to http://127.0.0.1:8000/api/docs and you should see the following:
8. Let’s continue to add models so we can test them in async api:
### demo_app/models.py
from django.db import models
class Tag(models.Model):
name = models.CharField(max_length=255)
def __str__(self):
return self.name
class Book(models.Model):
title = models.CharField(max_length=255)
tags = models.ManyToManyField(Tag)
def __str__(self):
return self.title9. Add demo_app to your INSTALLED_APPS to detect the models for migrations
### project/settings.py
INSTALLED_APPS = [
...,
'ninja_extra',
'demo_app',
]10. Make the migrations
python manage.py makemigrations demo_app11. Run the migrations:
python manage.py migrate Working with Data in Django-Ninja using Pydantic
Django Ninja allows you to define the schema of your responses both for validation and documentation purposes.
Schemas are very useful to define your validation rules and responses, but sometimes you need to reflect your database models into schemas and keep changes in sync.
ModelSchema
ModelSchema is a special base class that can automatically generate schemas from your models.
All you need is to set model and model_fields attributes on your schema Config
To get started, let’s take a look at the example code below:
1. In demo_app create new schemas.py file:
### demo_app/schemas.py
from typing import List, Optional
from ninja import ModelSchema, Schema
from pydantic import Field
from demo_app.models import Book, Tag
class BookTagSchema(ModelSchema):
name: str = Field(..., description="Tag name")
class Config:
model = Tag
model_fields = ["id", "name"]
class BookSchema(ModelSchema):
title: str = Field(..., description="Book title")
tags: List[BookTagSchema] = []
class Config:
model = Book
model_fields = ["id", "title", "tags"]
class BookCreateSchema(Schema):
title: str = Field(..., description="Book title")
tags: Optional[List[str]] = Field([], description="List of tag names")BookTagSchemais used to serialize and deserializeTagobjects. It defines theidandnamefields of aTag.BookSchemais used to serialize and deserializeBookobjects. It defines theid,title, andtagsfields of aBook. Thetagsfield is a list ofBookTagSchemaobjects.BookCreateSchemais used for creating newBookobjects. It defines thetitlefield, and an optionaltagsfield that is a list of tag names.
These Pydantic models will be used later in our API endpoints to handle requests and responses.
2. In demo_app create new api.py file:
### demo_app/api.py
from typing import List
from ninja.params import Body
from ninja_extra import api_controller, route
from demo_app.models import Book, Tag
from demo_app.schemas import BookCreateSchema, BookSchema
@api_controller("", tags=["Books"])
class BookController:
@route.get("books/", response={200: List[BookSchema]})
async def get_books(
self,
) -> List[Book]:
return [book async for book in Book.objects.all()]We are defining a Django Ninja Extra controller class, BookController, which has one GET route that returns a list of all books in the database. The controller is defined using the api_controller decorator from Django Ninja Extra.
The api_controller decorator is used to define a class-based controller in Django Ninja Extra. It takes several arguments to configure the routes and functionality of the controller. In this case, we are providing the prefix_or_class argument as an empty string, which means that this controller’s routes will not be grouped under any prefix. We are also providing the tags argument as ["Books"], which will be used for OpenAPI documentation purposes.
Inside the BookController class, we define the get_books method, which is decorated with the route.get decorator. This method returns a list of all books in the database by using the async for syntax to asynchronously iterate over all books returned by the Book.objects.all() query.
The BookController class also imports the List type from the typing module and the Body class from the ninja.params module. Additionally, it imports the Book and Tag models from the demo_app.models module, and the BookSchema and BookCreateSchema schemas from the demo_app.schemas module.
Using a class-based controller in Django Ninja Extra allows you to encapsulate related functionality and routes in a single class, which can be useful for organizing your codebase. By using the api_controller decorator, you can easily configure the routes and functionality of your controller.
3. In project/api.py register the controller:
### project/api.py
from ninja_extra import NinjaExtraAPI
from demo_app.api import BookController
api = NinjaExtraAPI(
version="1.0.0",
title="Project API",
)
api.register_controllers(BookController) # <-- register controller4. Navigate to the http://127.0.0.1:8000/api/docs:
5. New POST route
In addition to the GET route we created earlier, we’ll also create a POST route that allows us to create new books:
### demo_app/api.py
from typing import List, Tuple
from ninja.params import Body
from ninja_extra import api_controller, route
from demo_app.models import Book, Tag
from demo_app.schemas import BookCreateSchema, BookSchema
@api_controller("", tags=["Books"])
class BookController:
@route.get("books/", response={200: List[BookSchema]})
async def get_books(
self,
) -> List[Book]:
return [book async for book in Book.objects.all()]
@route.post("books/", response={201: BookSchema}) # <-- New route
async def create_book(
self,
payload: BookCreateSchema = Body(..., description="Book payload"),
) -> Tuple[int, Book]:
book = await Book.objects.acreate(title=payload.title)
tags = [await Tag.objects.acreate(name=name) for name in payload.tags]
await book.tags.aset(tags)
return 201, bookThe create_book function is a POST endpoint that creates a new book and its associated tags. It takes in a payload parameter, which is of type BookCreateSchema and represents the book data to be created. The response parameter specifies that the function returns an HTTP status code of 201 and the BookSchema representing the created book.
Within the function, the Book object is created using the acreate method, which asynchronously creates a new instance of Book and saves it to the database. The tags variable is then created as a list comprehension that creates a new Tag object for each tag name in the payload. The aset method is then used to set the tags of the book to the newly created Tag objects.
Finally, the function returns a tuple containing the HTTP status code 201 and the created Book object. The HTTP status code is specified as the first element of the tuple and the book object as the second element.
By using the Body parameter and specifying the description argument, we provide documentation to the users of the API, explaining what data the API endpoint expects. This makes it easier for developers to consume the API and ensures that the data sent to the endpoint is in the correct format.
Overall, the create_book function is a useful addition to the BookController class, as it provides the functionality for users to create a new book with its associated tags using a simple HTTP POST request.
6. Navigate to the docs:
Testing the BookController
To test our application, we’ll need a few additional packages:
pytest: a testing framework for Python that makes it easy to write and run tests.pytest-django: a plugin for pytest that provides support for testing Django applications.pytest-asyncio: a plugin for pytest that provides support for testing asynchronous code using Python’sasynciolibrary.
You can install these packages using pip:
pip install pytest pytest-django pytest-asyncioStarting with application testing!
When writing tests for our Django application, it’s important to have the proper requirements and configuration to support asynchronous testing. Here’s an example of how to set up the pyproject.toml configuration file to enable this support:
# project_base_directory/pyproject.toml
[tool.pytest.ini_options]
minversion = “7.0” addopts = “-ra -q –no-migrations –reuse-db –log-cli-level=INFO” # <– add your options here testpaths = [ “tests”, ] asyncio_mode = “auto” # <– enable asyncio support python_files = “tests.py test_*.py *_tests.py” DJANGO_SETTINGS_MODULE = “project.settings” # <– define your settings module here|
Let’s break down what each of these parameters do:
minversion: This sets the minimum version of pytest required to run the tests.addopts: This allows us to add additional options to the pytest command. In this case, we’re adding options to display a summary of test results (-r), run tests quietly (-q), skip migrations (--no-migrations), reuse the database (--reuse-db), and set the logging level toINFO(--log-cli-level=INFO).testpaths: This specifies the directories that pytest should search for tests.asyncio_mode: This enables asyncio support for pytest.python_files: This specifies the file patterns that pytest should look for when searching for tests.DJANGO_SETTINGS_MODULE: This sets the name of the settings module for the Django application.
By configuring our pyproject.toml file with these options, we can ensure that our Django application is properly set up for running asynchronous tests with pytest.
To start testing our application, we need to create some test files. First, we need to create a test directory inside our demo_appdirectory. Then, we’ll create a Python package named testsinside the test directory.
On Linux or macOS, we can do this using the following command in the terminal:
$ cd demo_app
$ mkdir tests
$ touch tests/__init__.pyOn Windows, we can use the following commands in the command prompt:
cd demo_app
mkdir tests
New-Item tests\__init__.py -ItemType FileNext, we need to create two files inside the test directory: a conftest.py file and a test_api.py file. The conftest.py file will contain some fixtures that we will use in our tests. The test_api.py file will contain our actual tests.
The conftest.py file is used to define fixtures that can be shared across multiple test files. Pytest will discover it automatically. We’ll create a fixture for creating a test client that we can use to make requests to our API. Add the following code to conftest.py:
# demo_app/tests/conftest.py
import asyncio
import pytest
from demo_app.models import Book
@pytest.fixture(autouse=True)
def enable_db_access_for_all_tests(db):
"""
This fixture enables database access for all tests.
"""
pass
@pytest.fixture(scope="session")
def event_loop():
"""
This fixture handles the event loop for all tests.
"""
loop = asyncio.get_event_loop_policy().new_event_loop()
yield loop
loop.close()
@pytest.fixture
async def books():
"""
This fixture creates two books and returns them.
"""
await Book.objects.acreate(title="test")
await Book.objects.acreate(title="test2")
return [book async for book in Book.objects.prefetch_related("tags").all()]Now, let’s create our test_api.py file. In this file, we will write tests for the BookController we created earlier. Add the following code to test_api.py:
# demo_app/tests/test_api.py
import pytest
from ninja_extra.testing import TestAsyncClient
from demo_app.api import BookController
from demo_app.schemas import BookSchema
pytestmark = pytest.mark.asyncio
class TestBookController:
@pytest.fixture(autouse=True)
def setup(self, books):
self.books = books
self.client = TestAsyncClient(BookController)
async def test_get(self):
response = await self.client.get("/books/")
assert response.status_code == 200
assert response.json() == [
BookSchema.from_orm(book).dict() for book in self.books
]The TestBookController class is responsible for testing the BookController class we defined earlier. This test class includes a setup fixture that initializes a TestAsyncClient instance and populates the self.books attribute with a list of Book objects.
The test_get method sends a GET request to the /books/ endpoint using the TestAsyncClient instance. It then asserts that the response status code is 200 and that the JSON response body contains a list of dictionaries representing the books in the database. This is achieved by converting each Book object in self.books to a dict using the BookSchema.from_orm() method.
By setting pytestmark to pytest.mark.asyncio, pytest is instructed to run the tests asynchronously, which is necessary since the get method in TestAsyncClient is itself an async function.
Now we are ready to run our tests. Simply run pytest from the command line inside the base directory. This will automatically discover and run all the tests in the tests directory.
$ pytest
In the next part of this series, we’ll explore more advanced testing techniques and write tests for more complex scenarios.
Conclusion
Django Ninja and Django Ninja Extra are powerful asynchronous web development tools that can help you rapidly build high-performance APIs using Django and Python. In this article, we’ve covered some basic concepts like serialization, validation, and routing, as well as more advanced techniques like using async ORM to improve performance.
So stay tuned for the next part of the asynchronous web development series, and in the meantime, start building your own high-performance APIs with Django Ninja and Django Ninja Extra!
References:
- Django Ninja: https://django-ninja.rest-framework.com/
- Django Ninja Extra: https://eadwincode.github.io/django-ninja-extra/
- Project repository: https://github.com/godd0t/django-ninja-extra-demo
