How to Webpack your Django!
Contents
Django is a very popular Python Web Framework used to build all sorts of web applications, but it can be tricky to know the best way to integrate your client-side code. In this post, I will show you how to integrate the popular module bundler “webpack” into a Django application allowing you to take advantage of the latest JavaScript versions (ES6 and later) and frameworks (Vue.js). I will show an example “Environment Random Quote” application to demonstration the integration. Due to the outlined architecture, this application is very easy to develop further and to deploy, and update, on production. As portrayed in Fig. 1, the use of the latest and greatest JavaScript frameworks and taking advantage of the great features of webpack will allow your Django application to soar!
What is webpack?
At its core, webpack is static module bundler for modern JavaScript applications.
– webpack documentation
Webpack was initially released in 2012. The usage increased greatly after a fantastic talk by Pete Hunt at OSCON 2014 on how Instagram uses webpack to efficiently bundle their React application. Webpack has many useful features and can be used to build applications in other frameworks such as Vue.js and Angular 2+. Node.js is required to install webpack. It supports all browsers that are ES5-compliant (IE8 and below are not supported), but a polyfill is available for older browsers. Webpack is extremely popular with millions of downloads from NPM (Node Package Manager) each month. Fig. 2 depicts how webpack handles different file types and generates bundles of files that the browser can interpret.
Webpack has five core concepts:
- Setting the mode of development, production or none enables the built-in optimizations that correspond to each environment.
- The app entry point(s) are define. This is the file(s) that will be used inside the base html as the starting point for the dependency graph.
- The output configuration property tells webpack how to name and where to place the bundles. These are the processed files that are then used in your application.
- Out of the box, webpack only understands JavaScript and JSON files, loaders allow webpack to process other types of file and convert them into valid modules.
- Plugins are used for a wide range of tasks like optimisation, asset management and injection of environment variables.
These core concepts are utilised by way of settings in the webpack configuration file webpack.config.js.
Webpack Benefits
There are benefits to using webpack at all stages of the software development process. During application development, it offers an advantage over including individual .js and .css files directly in your html file:
- The webpack dev server watches for any changes in your source files. If changes are detected, it then rebuilds the bundles and reloads the browser to show the updates. This leads to reduced development time as it is no longer needed to refresh the browser manually after each code change.
- Eslint is a configurable JavaScript linter. The webpack eslint-loader will run the linter prior to building the bundles in order to highlight code issues by quickly failing the build so that you are aware of the issue and can fix it.
- Babel is mainly used to convert ECMAScript 2015+ code into a backwards compatible version of JavaScript in current and older browsers or environments. The webpack babel-loader means that you can write code using the latest JavaScript version, and the loader will ensure that the code is compiled correctly for older browsers.
When it comes to production webpack can:
- By using a mode of “production”, bundle files can be minimised and comments removed in order to reduce the file size that the browser must download. This improves the initial page load speed of the application. A “.map” file containing links between the minimised code and the original code can also be generated using the “devtool” setting to help debugging.
- Multiple entry points can be defined which means that only application code relevant to the page requested is loaded into the browser. Therefore code for a web page that the user has not requested is not downloaded. This is called code splitting.
- Bundle files can be generated to include a hash in the file name which means that on application update, the browser knows to re-fetch the new file(s) from the server, and retain files that have not changed stored in the browser cache.
How does webpack integrate into Django?
I have found that the secret to making webpack and Django work well together is in the library django-webpack-loader. There are many good examples in the library README of how to configure this library. Django-webpack-loader allows us to transparently use webpack with Django.
One of the core principles of django-webpack-loader is to not manage webpack itself in order to give you the flexibility to run webpack the way you want.
This package allows both Django and webpack to develop side-by-side and for webpack to communicate using a metadata file called webpack-stats.json. This JSON file contains the location of the bundled files that webpack has generated. In the code below is shown example content:
{% load render_bundle from webpack_loader %}
{% render_bundle 'app' %}
The render_bundle will render the proper links, generated by the build, into your template.
“Environment Random Quote” Example Application
To illustrate the integration of webpack into Django, a small example application has been devised. As “Environment” is the theme of PyCon Ireland 2019, the application fetches a random Environment quote from the database and displays it on a webpage. Three steps are shown as part of an application development. Firstly the API is setup, second the “frontend” infrastructure is setup including webpack, and finally webpack is integrated and tested using a client-side code. The application client-side is written using Vue.js, with webpack bundling the files and Django providing an API. The application is available to download on Github here. The application has a database of Environment related quotes. As shown in Fig. 3, the current quote is displayed with the author and a new random quote is retrieved when the user clicks “Next”.
1. Set up the Django
In order to serve a random quote through the API, a Django application is set up. The only two requirements are Django itself and django-webpack-loader which can both be downloaded from PyPi. A Django model is used to store the quote “text” and quote “author” which creates the corresponding database entries:
from django.db import models
class Quote(models.Model):
text = models.CharField(max_length=1000, help_text="The quote text")
author = models.CharField(max_length=100, help_text="The quote author")
In order to populate the database with some initial data, a fixtures JSON file is defined with can be loaded into the database:
{
"model": "quotes.Quote",
"pk": 1,
"fields": {
"text": "What's the use of a fine house if you haven't got a tolerable planet to put it on.",
"author": "Henry David Thoreau"
}
}, ....
A request to URL
from django.shortcuts import render
from django.http import JsonResponse
from django.forms.models import model_to_dict
from .models import Quote
def random_quote(request):
quote = Quote.objects.order_by('?').first()
return JsonResponse(model_to_dict(quote))
def index(request):
"""Index on what to show when user comes to root URL"""
return render(request, 'quotes/index.html', {})
As shown in the code snippet above, an index.html file is references in the templates directory. This file populates the content which will be delivered by webpack into the div with id “app”. The js and the css is separated. Also the “vendor” code (bootstrap, vue.js) is separated to avoid re-downloading if another entry point is added:
{% load render_bundle from webpack_loader %}
<head>
Environment Random Quote
{% render_bundle 'vendor' 'css' %}
{% render_bundle 'app' 'css' %}
</head>
<body>
{% render_bundle 'vendor' 'js' %}
{% render_bundle 'app' 'js' %}
</body>
2. JavaScript and Webpack Setup
All of the client-side code for this project is in a “frontend” directory. The JavaScript package manager npm is used to install webpack. Npm uses the package.json file to keep track of the packages installed. The dependencies section of this file is shown in Fig. 4 along with the use for each dependency:
In the package.json file, along with dependencies is a “scripts” section. This section is used to define any custom scripts for the project. The snippet below allows for running “npm run build” that will generate the bundles with the default config file webpack.config.js.. For development a second script is added, running “npm run start” command runs the webpack-dev-server with the webpack.dev.js config. The webpack-dev-server runs alongside Django and will serve the bundle from memory and watch for changes in the source file:
"scripts": {
"build": "webpack",
"start": "webpack-dev-server --config webpack.dev.js"
}
The next step is to create the two configuration files: webpack.config.js and webpack.dev.js. In order to replicate the production environment for development, as much as possible of the configuration should be placed in a webpack.config.js. This configuration file is the core of webpack and contains all the logic on how webpack should behave.
The config configuration which is common for both is shown below. Our mode is “production” as we want to minimise the output code in this case. It handles the app entry entry point for our “app” which is a file called index.js which is the entry point into the Vue.js application. The output destination and file name are defined. The “hash” is used to give a new filename to the output files so that the browser knows that the file has changed. The “publicPath” setting is also needed to give a destination for the output image files which will be handled with the file-loader loader.
The optimisation section is used to split the “vendor” code into a separate bundle to the “app” code. This is called code splitting and it is very useful as our application expands and other entry points are added. The plugins are then specified. The BundleTracker plugin is used to track the “bundle” files that webpack generates in order that Django knows what to load. It writes to the file specificed with metadata. VueLoaderPlugin helps handle .vue files which consist of Single File Components (SFC) that allow the html, css and js to reside in the same file. CleanWebpackPlugin ensures that old build files are deleted from the directory when new ones are generated. Finally, the loaders used to handle the different files types. “.css” files are handled with either the style or the css loaders. Image files are handled with the file-loader. vue files with the vue-loader. As stated above, loaders allow webpack to handle file types beyond JavaScript and JSON.
const webpack = require('webpack');
const BundleTracker = require('webpack-bundle-tracker');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');
const VueLoaderPlugin = require('vue-loader/lib/plugin');
module.exports = {
mode 'production',
entry: { app: './apps/frontend/js/index.js', },
output: {
path: __dirname + '/apps/frontend/bundles',
filename: '[name].[hash].bundle.js',
publicPath: '/static/dist/'
},
plugins: [
new BundleTracker({filename: './apps/webpack-stats.json'}),
new CleanWebpackPlugin(),
new VueLoaderPlugin(),
],
optimization: {
splitChunks: {
cacheGroups: {
commons: {
test: /[\\/]node_modules[\\/]/,
chunks: 'initial',
name: 'vendor',},},},},
module: {
rules: [
{ test: /\.css$/,
use: ['style-loader', 'css-loader'], },
{ test: /\.vue$/,
use: 'vue-loader', },
{ test: /.*\.(png|jpe?g)$/i,
use: 'file-loader', },],},};
Finally the development specific config file is added. The mode of development ensures that the default webpack optimizations for development are enabled. The output publicPath is overwritten with a URL link to the bundles. The devServer setup includes the port that the local development server to run on. The hot-reload is enabled which specifies that only the part of the page that has changed should up updated. Additionally as there is a request from Django running on a different port, accessing from a different origin is allowed in this case:
const merge = require('webpack-merge');
const common = require('./webpack.config.js');
module.exports = merge(common, {
mode: 'development',
output: { publicPath: 'http://localhost:3000/static/dist/', },
devServer: {
port: 3000,
hot: true,
headers: { "Access-Control-Allow-Origin": "*" },
watchOptions: { ignored: /node_modules/ }, },
});
3. Integrate webpack into Django
In the django-webpack-loader project README there are instructions on how to use the package. It must be installed in both the Python and in JavaScript sides, and the Django settings file must be updated to let Django know to find additional static files. The code below tells Django where to find the bundles and the metadata on the webpack generated files:
WEBPACK_LOADER = {
'DEFAULT': {
'BUNDLE_DIR_NAME': 'dist/',
'STATS_FILE': os.path.join(BASE_DIR, 'webpack-stats.json')}}
As above, on build, webpack produces a file called “webpack-stats.json” which contains information on the the bundle(s) produced by webpack. The chunks are the different landing pages that for our application. In our case it is only “app”, but it is also possible to have multiple entry points. The filename with the hash is shown in this file so that it is available for Django to use:
{"status":"done","publicPath":"/static/dist/","chunks":{"app":[{"name":"app.341513f60d69919e8f18.bundle.js","publicPath":"/static/dist/app.341513f60d69919e8f18.bundle.js","path":"/home/davidg/quotes-display/apps/frontend/dist/app.341513f60d69919e8f18.bundle.js"}],"vendor":[{"name":"vendor.341513f60d69919e8f18.bundle.js","publicPath":"/static/dist/vendor.341513f60d69919e8f18.bundle.js","path":"/home/davidg/quotes-display/apps/frontend/dist/vendor.341513f60d69919e8f18.bundle.js"}]}}
In this case the entry file index.js imports App.vue. Vue is inported and the App.vue file too. The Vue instance binds onto the HTML element with id “app”, and renders the application:
import Vue from 'vue';
import App from './App.vue';
// eslint-disable-next-line no-new
new Vue({
el: '#app',
render: (h) => h(App),
});
Finally a small Vue.js application is created to fetch and display the retrieved quote. In Vue.js the code can be written in a Single File Component (SFC) .vue file. This can make it easier to see all the HTML, JavaScript and styles associated with the component in one file. The component can then be re-used and imported into the larger application as needed.
<div class="full-area h-100">
<div class="d-flex justify-content-center">
<img alt="Earth with tree" height="200px">
</div>
<div class="d-flex justify-content-center"><h1>Environment Random Quote</h1>
</div>
<div class="quote"><div class="d-flex flex-row justify-content-center">{{ quoteText }}>
</div><div class="d-flex flex-row justify-content-center">- ><;{{ quoteAuthor }}</i>
</div></div>
<div class="d-flex flex-row justify-content-center">Next</div></div><pre>
import 'bootstrap/dist/css/bootstrap.min.css';
import environmentImage from '../img/environment.png';
export default {
name: 'App',
data() {
return {
quoteText: '',
quoteAuthor: '',
environmentImage,
};
},
created() {
this.getQuote();
},
methods: {
getQuote() {
const url = '/random-quote';
fetch(url)
.then(response => response.json())
.then((json) => {
this.quoteText = json.text;
this.quoteAuthor = json.author;
});
},
},
};
.button {
background-color: #4CAF50;
color: white;
padding: 15px 35px;
display: inline-block;
font-size: 30px;
margin: 4px 2px;
cursor: pointer;
}
.full-area {
background-color: #D3D3D3;
padding-top: 2%;
}
.quote {
margin-top: 3%;
margin-bottom: 2%;
border-style: solid;
background-color: white;
font-size: 30px;
}
Result
In Fig. 5 the terminal setup during development is shown. One terminal tab is used for the Python server side to start Django (python manage.py runserver) which runs on port 8000, and a second terminal tab is used for starting the webpack dev server (npm run start) which is running independently on port 3000.
The end result is shown by navigating to http://127.0.0.1:8000 in the browser. The bundled files are retrieved from the webpack-dev-server running on port 3000. With any change in the source files, the new content is automatically refreshed.
Considerations for deployment to production
Once the application is developed it is ready for deployment to production. For development, the application is running on two different ports. This is to allow the application to refresh when changes are made to the client-side files. But for production, there is now no need to watch whether the files are changing as this would be inefficient. For production, the files are built one time using the “npm run build” command. This places the compiled files into a directory where Django knows to find them.
In production, Django has a command to gather static files into a single directory so you can serve them easily with a command called “collectstatic“. Django copies from the defined static directories into the the STATIC_ROOT:
STATICFILES_DIRS = ( os.path.join(BASE_DIR, 'frontend'), ) STATIC_ROOT = os.path.join(BASE_DIR, 'static')
The “npm run build” must be run before collecting the static files. I use the deployment script below to ensure that the happens:
echo "Build the production bundle" npm run build echo "Collect all the static files for Django" venv/bin/python apps/manage.py collectstatic --noinput --clear --settings=quotesdisplay.production_settings
These steps must be done in this order to ensure that the generated bundle files are ready for the collectstatic command. After this the restart Django command is done.
Summary
In this post, the architecture needed to integrate the very useful “webpack” into your Django application was shown. This approach makes use of the django-webpack-loader library to allow webpack to be configured as needed.
The simple architecture shown keeps all of the application code (server and client-side) in one repository. This is useful when it comes to make any changes to the API, as both the client-side code can be deployed simultaneously. Additionally as both the server and the client are served from the same domain, there is no need to worry about CORS issues. This architecture has been tested over time and found to be robust. Although it can be tricky and time-consuming, once the Django and webpack architectures are setup, minimal maintenance is required and you can focus on both server and client side development with confidence.
The “Environment Random Quote” application was used to put these concepts into an example. Many further features can be explored once the webpack environment is setup, such as adding Eslint, Babel, bundle analyser, etc. For this reason, webpack can be very useful to your Django application and allow use of the very latest and greatest JavaScript advances.
References
