Monday 13 February 2012

Streaming JavaScript is still Great! Use It!

So back in January 2009 I was advocating splitting your codebase up into different functions, where you can stream in different logic bit by bit as required. This was due to support explosion of smart phones which didn't have the bandwidth or processing power desktop computers had.

It's been over 3 years since supporting this pattern, and I've honestly had very little to complain about, but I figured I'd highlight two of the negatives I've come across over the years.
  • Caching - Anything you stream will not be cached, so you'll have to cache the JavaScript yourself.
  • Bugs - As this isn't a practiced design pattern in the world of web programming, browser vendors don't always check for it before pushing out an update.

This one's a piece of cake. Simply use LocalStorage to store your cached JavaScript file.
LocalStorage gives you around 2mb no questions asked storage to store binary data, which is more than enough to store small javascript files.

Here we walkthrough the process of caching a JavaScript file.

 var jsScriptData = false;  
 // If we have localStorage  
 if( window.localStorage )  
   // See if we have the data  
      if( window.localStorage.getItem( "jsData" ) )  
           jsScriptData = window.localStorage.getItem( "jsData" );  
     // If not download the file normally  
     jsScriptData = DownloadFile( "" );  
     // Then save to localStorage  
     window.localStorage["jsData"] = jsScriptData;  
   // If we don't have localStorage download the file normally  
   jsScriptData = DownloadFile( "" );  
 // Create a new script element  
 var newScript = document.createElement( 'script' );  
 // Fill in the script element with more source code  
 newScript.text = jsScriptData;  
 // Add our script to the document  
 document.body.appendChild( newScript );  

Of course more smarter checks can be put in, for example, checking for an update every week, or having a flag in the main JavaScript file to flag when to stream in the update.

Next issue to discuss is bugs. As of writing Mozilla recently updated Firefox to 10.0.1, which is great, updates are always good. However, for some reason stopped working on the OSX skew.

On Windows it was fine, but the OSX just hung on streaming in more JavaScript files. Now on the iGrapher project we load 4 files.
  • startup - 21kb file which loads in the main view UI and downloading engine.
  • main - 98kb file which handles the graphing and stock market downloading logic.
  • extras - 150kb file which ads additional tools and menus to the graphing component.
  • news - 33kb file which handles the news stories.
What's shown in the screenshot above is the webapp hung getting the extras file. Now I didn't put in any time to debug the issue as I'm more focusing on the updated codebase. So unfortunately I can't deduce what exactly in the code causes it, but I found that by combining the startup and main files together into one file. The extras and news files stream in fine.

Mysterious, but I thought it'd be useful to highlight that yes, bugs occur, when they perhaps shouldn't.


I come across bloated websites all the time which rely on the browser to load in lots JavaScript files at the same time. This delivers a poor and very staggered start up experience. It's poor on desktop computers let alone the exploding smartphone/tablet market. We think about streaming in content all the time, why stop there? Let's start thinking about streaming in processing and fix this poor user experience we've accepted as the norm.


  1. GWT does this (code point splitting) for you automatically. It also implements 'perfect caching'.

    It is a bad idea to store scripts in local storage. I for one refuse all requests from browsers to allow a site to write into local storage.

  2. I too think it's a bad bad idea to use local storage for caching Javascript files. Browser bugs are just the tip of the ice berg of problems you're entering into with that solution. Like the previous commenter said, people don't necessarily allow the browser to save anything into the local storage, older browsers don't support it and so on.

    In my honest opinion a much better way is to divide this into two parts: first, combine all your Javascript files needed in the first page load into one file and use YUI encoder or Google Closure or whatever to minimize the data size. Second, just use plain old AJAX to load additional files like you suggested. A nice thing in this is that the Javascript files *will* be cached as you'd expect. In fact in order for you to disable the caching you need to change the URL by appending something random as a not needed GET-parameter (such as timestamp). Please check for example the jQuery documentation:

    So, all in all I agree that you should use dynamic loading of Javascript files where needed to improve performance but I disagree in that you should use local storage for caching the files. Why when the browser does provide you with everything you need already?