As fast as possible with django CMS (django CMS Page Speed)
How we improved the speed of our django CMS sites, and what you can do to achieve the same benefits.
On the Aldryn cloud platform for Django, we recently implemented some steps to improve the performance of Django sites. In this article we’ll discuss those steps in more technical detail, and show you some of the things you can do in your own projects to secure similar speed gains, whether your projects are hosted on Aldryn or elsewhere.
What’s the problem with slow sites?
When a web visitor visits a URL, the first thing they do is… wait. We’ve become accustomed to spending a significant proportion of our lives waiting for computer systems to respond.
If there were only a few websites in the world, and they were all offering something unique, it wouldn’t matter so much: people would simply have to wait (and that’s what they did).
Now, however, there are millions upon millions of websites competing with each other for visitors’ attention, and it turns out that one of the best advantages a site can have in this endless competition for visitors is making them wait less.
The sites winning the competition often turn out to be the ones that make their visitors wait less than their rivals do, and so everyone, in turn, is getting better at making visitors not wait.
Just to add some heat to the competition, search engines also prefer sites that don’t make visitors wait. A faster site gets bumped up the results rankings, because being fast is considered to be part of what makes information good.
In short, if your site isn’t as fast as it could be, then even before it enters the competition with all the other sites your visitors use, it’s at a disadvantage.
It’s not so easy
It’s easier said than done to speed up a site, especially a complex one built with extensive backend and frontend application components, and dependent on all the interactions between them.
This is why we build as many speed-improving techniques as possible into the django CMS projects on Aldryn, so that users get these advantages by default.
Even so, sometimes you simply have to do some of this work yourself, for example if you’re deploying django CMS sites using your own preferred build tools, or hosting them on your own servers.
In this article we’ll look at the steps we’ve taken for Aldryn projects, and discuss how you can do something similar for your own django CMS projects.
First, you must capture your numbers
You can’t act without having something to act on: when it comes to web performance, you need numbers that tell you where your pages are spending their time, and how much. If you don’t rely on numbers, you could waste a lot of time of your own trying to address perceived problems that aren’t actually worth the effort.
It’s worth noting that your aim is not to have fast numbers. Your aim should be to have fast pages. Web developers can get sucked into chasing improved numbers for the sake of it. The numbers are merely indicative - but they are still the best place to start.
There are various tools out there to help you with this; two of the most widely used are Pingdom and Google PageSpeed.
What the numbers told us
We ran these tools on the Aldryn django CMS Explorer theme project (a site, complete with content, that allows users to explore django CMS without setting it up themselves).
Pingdom’s score was 83%. PageSpeed’s was in the 60s. As important as the scores they returned, these services also gave hints about how to improve matters.
The problems and how we dealt with them
Server response time
PageSpeed warned us that our server was slow to respond. That is, the time between the browser’s request and getting data back was too long. So, we spent some time looking into that, and with the aid of Django Debug Toolbar, decided that django CMS’s in-application caching could be improved, and that some key components were simply inefficient.
In-application caching
Caching is a complex business, always a balancing act between trying to use the cache to serve data quickly, and ensuring that we don’t serve stale data. The answer isn’t just more caching, but better caching.
The latest django CMS 3.3 represents the work we’ve done to improve caching; in particular, we’ve introduced an additional layer of “fragment caching” which substantially reduces the amount of content that must be re-rendered when a single plugin needs updating. During this effort, we’ve also added the ability for individual plugins to determine their own cache expirations. This change permits site developers to set the global content caching to much higher values but still get fresh content in specific areas. For more details, please see our release post for django CMS 3.3.0.
And, for advanced use-cases, plugins can now also set VARY headers. This enables the CMS to manage multiple caches when the plugins’ contents change for certain audiences.
The next version of django CMS will take some even bigger steps forward. We’ve been working on caching that targets specific subsystems (such as plugins and the menu) in intelligent ways, so that more content can be cached, and cached longer, without the risk of serving stale content.
Efficiency
We’ve also been looking very critically at some of the more time-consuming functions in the CMS. Many components begin their lives in a fairly simple form, and grow more complex over time. As functionality, abstraction and edge-case checking are added, there’s a tendency for the complexity to cause (and conceal) inefficiencies. We discovered a few of these; we'll expose some of the gory details in a forthcoming article.
What you can do to improve server response times
One of the first and easiest things to do is simply to ensure that you are using the latest version of django CMS. We are making huge efforts to improve performance, and we’re determined that each new version will be faster.
If you’re working with custom code of your own, use tools like Django Debug Toolbar. Django’s own documentation contains some excellent advice on how to improve the performance of your code. It’s a never-ending task to keep performance as high as possible, but it should be part of your overall development process.
Taking advantage of browser caching
Browsers are quite good at caching content. After all, they too are in a speed competition with their rivals.
However, they can only cache content when they are confident that it’s safe to cache, so to take full advantage of their caching powers, a website needs to provide the information that lets them do that.
One thing you can rely on is django CMS itself; it aims to the right things, and we’re committed to improving its behaviour in future development. Keeping your django CMS up-to-date will never hurt.
Elsewhere, you may need to take more responsibility for the way your site does things. One of the keys to good caching is to make correct use of HTTP headers, that tell the browser what can safely be cached and for how long. By hashing filenames you can avoid having to make compromises on cache duration and can cache every single asset indefinitely.
How to use HTTP headers
This information is provided in HTTP headers and defined either by your web server configuration (e.g. Nginx) or set dynamically on a per-request basis by your views, a middleware, or, in the case of the CMS, by the CMS itself.
What you can do to make use of browser caching
When using django CMS, headers for dynamic content will be set correctly out of the box. You have to make sure that your web server sets the correct values for static and media files, based on how dynamic their content is.
For example, django-filer creates a unique filename for each upload, it is thus safe to set the headers of those files to never expire.
Django’s own caching documentation should be your reference for headers that you need to set on your own applications’ content.
How to make use of filename hashing
The idea here is to let the browser cache CSS and other files indefinitely, so that it never has to load it again. If the file is updated, it should be given a new name, by adding a hash of its content to it, so that as far as the browser is concerned it’s not a new version of the same file, but a new file altogether.
We do this in the Aldryn Advanced Bootstrap 3 boilerplate and the Explorer theme - use those, and you’ll enjoy filename hashing out-of-the-box. If you want to do it in your own projects, there are numerous backend and frontend tools that will help. On the backend, you can make use of Django’s built-in ManifestStaticFilesStorage (on Aldryn you can configure it with a single click), while on the frontend we suggest using webpack to do it; see our pull request for an example of our implementation.
Blocking content
Blocking content is material on a page that holds up everything else. When a browser encounters an instruction to load a resource that will affect the rendering of the page, such as CSS styles or JavaScript, rendering (or even the loading of other resources) will be blocked until the this content is loaded.
You can read more about how to deal with blocking content:
What you can do to unblock content rendering
Our Aldryn Explorer project uses unblocking techniques; use it to explore them. It’s based on the Aldryn Advanced Bootstrap 3 boilerplate, which also uses a subset of these techniques to address this issue.
Note that the work to unblock content is very project-specific, so needs to be implemented on a project-by-project basis. This is why the boilerplate only does some fairly minimal unblocking.
Of course, you may have a site already, or perhaps would like to use some other frontend set-up for your site. We’ll discuss below how we did it for the Explorer theme; feel free to adapt this for your own own projects.
The basic principle is to move the script and link elements that load the blocking resources to the very bottom of your HTML templates, so that they are loaded last and can’t block anything else from happening.
This brings its own problems however, especially if your page relies on the JavaScript to display content, in which case the page can be affected by a disconcerting flicker effect.
To avoid this, the developer can do a number of things. One is to reserve space using width and height declarations (you should do this anyway).
A more sophisticated technique is to hide content until the JavaScript is ready. What we do is avoid loading any CSS in the HTML <head>; instead, first we have is a simple:
<style>body { visibility: hidden; }</style>This ensures that the content can remain hidden until we’re ready to trigger its appearance.
Then, we load the CSS files proper using JavaScript, and re-enable the visibility when all is ready to be displayed.
First, in case JavaScript is disabled, or fails to load:
<noscript id="deferred-styles">
<link rel="stylesheet" href="{% static 'css/base.css' %}">
</noscript>This needs to go at the end of the HTML document, before the other JS calls.
Then:
<script>
var loadDeferredStyles = function() {
var addStylesNode = document.getElementById('deferred-styles');
var replacement = document.createElement('div');
replacement.innerHTML = addStylesNode.textContent;
document.body.appendChild(replacement)
addStylesNode.parentElement.removeChild(addStylesNode);
};
var raf = requestAnimationFrame || mozRequestAnimationFrame ||
webkitRequestAnimationFrame || msRequestAnimationFrame;
if (raf) raf(function() { window.setTimeout(loadDeferredStyles, 0); });
else window.addEventListener('load', loadDeferredStyles);
</script>The final result is a page that progressively loads what it needs to, without flicker.
You can go much further with these techniques. For example, you could use:
<script src="{% static 'js/libs/jquery.min.js' %}" async></script>
to enable JavaScript to execute asynchronously. However, you need to ensure that your JS will work when loaded this way, and that likely means more work to achieve that. We don’t yet do this in the Explorer theme, but it’s an improvement we have in our plans.
Another complication: desktop and mobile sites need to behave differently. For a mobile site, we don’t use the simple visibility: hidden mentioned earlier. This is because mobile and desktop browsers display material very differently, so instead we do:
<style>
@media (max-width: 600px) {
body { visibility: visible; }
.feature-visual img { width: 100%; }
.sr-only, .skip-links, .navbar-collapse { display: none; }
}
</style>The implementation details of these techniques can be found in a pull request on the django CMS Explorer theme repository.
There’ll be more improvements in the future - it’s an ongoing process of development, and you will find that it’s the same for your sites too. In particular, there are some very sophisticated above-the-fold techniques for unblocking content that we will be exploring.
Compression
Everything that is transmitted over the network can be compressed using Gzip algorithms; these take a very little computing time to decompress in the browser, but can save significantly on network times.
On the Aldryn platform, this is available to all project out-of-the-box and you don’t need to do anything to enable it.
On your own stack, check how your web server can help you. Nginx can transparently compress response bodies based on the content-type, while other servers (e.g. uWSGI) require a gzipped version to be provided in the first place.
Minification
HTML, CSS and JavaScript files can all be minified for deployment - reformatted to eliminate all redundant whitespace and carriage returns. It makes them unreadable for humans, but computers don’t mind, and (especially in the case of JavaScript) can make them much smaller.
It makes much less difference when you’re also using compression, but if you’re not, it’s a useful advantage.
What you can do to minify files and HTML
We do this by default in the Explorer theme, using a minify.py Python script. Feel free to use that. You can also get frontend tools such as gulp to do it for you.
Consolidation
Finally, rather than having CSS and JavaScript split across multiple files, it’s better to consolidate all styles into a single file, and do the same for JavaScript - this can also significantly lowers the burden on the network, by reducing the number of requests that need to be made.
How to consolidate files
The Explorer theme does this automatically, and the Advanced Bootstrap 3 boilerplate does some too.
Frontend tools will help; for example we use libsass to process our CSS and one of its functions is file consolidation.
Using content delivery networks
Content Delivery Networks (CDNs) are services that take the load off your server and the network between it and the browser. They are dedicated systems optimised for file delivery, and can serve them from nodes much closer to the web visitor than your application servers might be.
On Aldryn we are implementing first-class support to integrate CDNs for media files, static assets, and also dynamic content delivery.
When maintaining your own servers, you can pick any CDN and put it in front of your website. The configuration varies from service to service, but mostly revolves around modifying your DNS settings to point to the CDN servers instead of your own server directly.
Image optimisation
An image can say a thousand words, but if your images run into several hundreds of thousands of bytes, they pose their own problems.
Image optimisation is no longer a matter simply of resizing and choosing a preferred level of compression. More can be done to lower file sizes without compromising quality.
We run images through a series of optimisations using the a wide range of tools (such as PNGCrush, pngquant, Zopfli, JpegOptim, gifsicle,...). Images that are served at runtime can be optimized automatically thanks to the integration provided by Django Easy Thumbnails, and we provide a Django management command to do the same for static image files.
What you can do
If you’re using Aldryn together with the Aldryn django CMS addons, the integration is already done for you; you just have to execute the aldryn_optimize_static_images management command on your repository to optimize the static images.
If not, you will have to install the required libraries yourself and configure Easy Thumbnails to optimize the generated thumbnails (this is how we do it). For static images, you can write your own optimization pipeline or just use a desktop application to do it (on Mac OS X we suggest ImageOptim).
Keep on keeping on getting faster
As we noted, better performance is not just a moving target but also a competition with constantly improving rivals - it’s not something that you just need to do once and forget.
However, we’re doing our part to help you, by building the drive to speedier websites into django CMS itself, and we’re going to continue sharing best practices and and help to ensure that your own sites stay up to speed.
We’ve done a great deal of work in django CMS, on Aldryn, in the Explorer theme project and in the Advanced Bootstrap 3 boilerplate to implement the improvements we’ve achieved so far, but as we’ve noted above, in many of these areas we still have plans for more. The new PageSpeed and Pingdom scores are very high, but our aim is not just high scores, it’s continuous improvement.
In other words, stay tuned, and expect further performance gains.
In the meantime:
- update to the latest version of django CMS to enjoy its speed benefits
- try an Aldryn project (you can use a free account) for the additional speed benefits of the platform
- see the Aldryn Advanced Bootstrap 3 boilerplate for an excellent template for your own projects, with some more techniques built-in
- check out an Explorer project of your own on Aldryn for the project-specific techniques you can also use
