Project Environment: A very gentle introduction
Some classmates have told us they are concerned about the technical parts of the course.
Since this is not the subclass for CS majors, in general we’ll try to go slowly and give help.
On Friday, Kevin will give a tutorial on Django.
Role of Django
HTTP HTTP
Internet
Client/server pattern
HTTP
Site
HTTP
COMP3297: Software Engineering
2
COMP3297Project
Django is a framework that makes it easy to implement web application software without writing much code.
Role of a site
HTTP Request HTTP Response
In its simplest form:
a browser sends an HTTP request to a site for a page.
the site sends an HTTP response containing the requested page. This is the basic Request/Response Cycle.
For static resources, that is all that is needed.
But most pages for real-world sites must be generated dynamically because they are based on data that is constantly changing. So the cycle is a little more complicated.
COMP3297: Software Engineering
3
COMP3297Project
HTTP Request HTTP Response
URL Dispatch
URL Patterns
HttpRequest object
Django website
Template Loader
HttpResponse object
App
Views
URL Patterns
Models
Database
Django doesn’t see the raw HTTP Request and couldn’t
parse it anyway. It is a WSGI application.
Templates
(also supports ASGI, but development server doesn’t)
COMP3297: Software Engineering
4
HTTP HTTP
Client-server pattern
Site
HTTP
Internet
HTTP
Logic
Persistence
Database
COMP3297: Software Engineering
5
Web server
COMP3297Project
Application server
Role of Django
HTTP HTTP
Client-server pattern
Site
Internet
HTTP
COMP3297Project
HTTP
Logic
Persistence
Database
Web server
Application server
6
The application server calls the Django application to generate a response to a request.
It communicates through an application callable, supplying request data and a callback function for status code and headers. Django returns the response body as strings.
The application callable, is set up by default, as the application
object in the project’s wgsi.py module – you’ll see this in a moment. COMP3297: Software Engineering
Here is a rough architecture and components of a typical production environment.
Web server
-Builds and handles client requests
-Serves collected static content directly
-Forwards requests for dynamic content to a server (here gunicorn) that can connect to our Django application’s code via WSGI.
-Serves those responses -Reverse proxy agent -Load balancing -Caching
-Executes application logic to respond to request from gunicorn.
-Postgresql or similar for persistence.
COMP3297: Software Engineering 7
-Also an HTTP server but here acting as an intermediary (“application server”). Integrates well with Django.
-Handles HTTP requests from nginx
-Connects to Django through WSGI,
-Passes the translated request, calling Django’s request handler.
-Responds to nginx
The Development Server and default DBMS
Django provides a lightweight Web server to use during development. It is based on the the simple WSGI HTTP server from the Python Standard Library – implemented by the wsgiref.simple_server module.
It’s a subclass of wsgiref.simple_server.WSGIServer
You start it with the runserver command of the manage.py command-line utility.
By default it starts the server to run on port 8000 on IP address127.0.0.1.
It is not suitable for production but is great for development and is very convenient to work with.
Django also ships with a file-based database, SQLite, “in the box”. It is a good default for development. It is already installed for you and Django is configured to use it.
COMP3297: Software Engineering 8
Let’s write some short examples – but first:
Working on real-world development projects introduces new difficulties for us.
• We need to take control of project dependencies so that we can replicate our project’s required environment in production – we want to be able to deploy and have confidence that our software won’t break because of some incompatible version of a third-party package present on the target machine.
• We want members of our team to be able to work collaboratively on the code even though they may have different machines and installed packages.
• We want to be able to work on multiple projects on our own machines even though they might require different versions of Python, Django, and other packages.
COMP3297: Software Engineering 9
What we need
Isolation for packages so that if a project uses an older version of, say, Django, and another project needs the latest version, then we can still work on them at the same time.
The standard way to do this is to create a virtual environment for each project where third-party packages can be installed in isolation rather than system-wide.
We also want to track what we install and uninstall so that we can replicate the same environment and its critical dependencies when we deploy into production.
We’ll use pipenv as a packaging tool. It combines two widely used tools, pip and virtualenv, to achieve both of these goals and more. It does both package management and virtual environment management for us.
You will see it creates two files for this to specify dependencies and specific versions of those dependencies: Pipfile and Pipfile.lock
COMP3297: Software Engineering 10
Installing pipenv
There are installation instructions for your platform here:
https://pipenv.readthedocs.io/en/latest/install/
For example, for a user install:
Then install packages through pipenv, creating Pipfile and Pipfile.lock. Example:
COMP3297: Software Engineering 11
Using pipenv
…
COMP3297: Software Engineering 12
3.1.5 and minor updates
Resulting Pipfile
COMP3297: Software Engineering 13
Starting here after class
Two common layouts:
Either is fine. The one on the left is probably more popular and that is the one we will use in examples.
Let’s create the simplest Django site to find out what lives at each level of the project hierarchy.
Django Project Layout
COMP3297: Software Engineering 14
Make a new directory as the repository root for your new project and switch to it.
Then, just as we did earlier, create a virtual environment for the project and install the version of Django you want using pipenv.
Then activate the virtual environment:
COMP3297: Software Engineering 15
Creating a project: startproject
Make the django_project_root directory, here called demo, switch to it and start a new project there:
don’t miss the ‘.’
Repository root
the Repository level is where we will put critical files, particularly for deployment, such as README, the /docs/ directory, .gitignore, pipenv files, and the Django project root directory.
Django project root
Configuration root: Site-wide modules
COMP3297: Software Engineering
16
Configuration root modules
Configuration settings for project
Root URL configuration (URLconf). Maps URLs to views.
Entry point for application servers (e.g. gunicorn). wsgi.py contains the application callable.
COMP3297: Software Engineering
17
Run the development server as a check. Remember how?
From earlier:
“start it with the runserver command of the manage.py command-line utility. By default it starts the server to run on port 8000 on IP address127.0.0.1.”
COMP3297: Software Engineering
Visit http://127.0.0.1:8000/ from a browser
18
What about those “unapplied migrations” it complained about?
First, you need to know about apps:
A Django project consists of at least one app, together with the project-wide configuration modules. The apps in a project work together to power the overall web application.
An app itself is a small web application that is dedicated to a particular piece of the overall project functionality. Following principles of good design, we keep apps small, self-contained and tightly focused on that small area of responsibility.
One benefit of this approach is that our apps become good candidates for reuse in other projects.
COMP3297: Software Engineering 19
What about those “unapplied migrations” it complained about?
Django provides a several useful built-in apps to provide, for example, admin functionality and authentication functionality. Django installs some of those automatically in new projects.
When anything new is added that impacts the database, Django recognizes things have changed and reminds about migrations which might be needed to keep the database schema in sync.
Propagating changes to the database schema is called migration in Django.
COMP3297: Software Engineering 20
What about those “unapplied migrations” it complained about?
Look in settings.py to see the built-in apps that were added by Django. Those apps define models, and Django is reminding you that you haven’t made corresponding changes to the database schema yet.
21
You aren’t forced to keep the apps that were added – you can remove them if
you don’t need them. And you can also add additional built-in and third-party
apps that you do need. You add your own apps here, too.
COMP3297: Software Engineering
Creating an app: startapp
COMP3297: Software Engineering
Configuration file for admin
Configuration file for this app
Tracks changes to models.py so that database and models can be kept synchronized
These are the database models that will be translated into database tables for us.
App-specific tests 22
Simplest case:
HTTP Request HTTP Response
HttpResponse object
App
Views
URL Patterns
URL Dispatch
URL Patterns
Django website
HttpRequest object
On startup, Django loads all the URL patterns we have configured. When it receives a request, Django translates it.
The URL resolver tries to match the URL path. If it finds a matching pattern then, after possibly further matching, handling code is identified and called, passing the HttpRequest object as an argument.
As the simplest example, let’s just display a text message if the user doesn’t supply
anything to match – that is, we’ll match the empty string.
COMP3297: Software Engineering 23
How do we get Django to do it??
We want Django to do it as part of its normal cycle of execution. Then we only need to add code to be executed where we want specific behaviour.
We don’t call Django, Django just uses our code at those points.
In the simple case we want Django to take a request (say an HTTP GET) for a resource specified in the form of a URL and match it to a pattern that we specify so that Django can call the code we’ve written to handle such a request.
Then after Django has executed our code, we want Django to handle the response.
So, we only need to provide:
• details of a pattern to match,
• a callable to be executed when there is such a match. That callable will take an HttpRequest object and return an HttpResponse object
Remember, in Django, any such callable is known as a View. COMP3297: Software Engineering
24
The simple example: First create a view function
Edit the views.py file of the app to add the following:
This new function is a callable that takes an argument – it’s called with an HttpRequest object which we don’t use here – and returns an HttpResponse object. Here the response contains the text we want to display.
So, from earlier, it is a View as far as Django is concerned.
COMP3297: Software Engineering 25
Map the view to a URL in a URLconf
Now we tell Django what to do if our greetings app gets a match for the empty string. We need to add a urls.py file to the app to specify it (called a URLconf in Django):
This tells Django to call greetingView if the request does not specify anything particular – that is, if the route is empty.
COMP3297: Software Engineering 26
Finally we need to point the project level root URLconf to the greetings/urls URLconf to include it
Add these to what is already supplied by Django
include() strips whatever matched so far and sends the rest to the specified URLconf (here to greetings.urls)
COMP3297: Software Engineering
27
Check that it works
runserver again
COMP3297: Software Engineering 28