a weblog by Ed Eliot

Pressure is nothing more than the shadow of great opportunity. - Michael Johnson

Automatic merging and versioning of CSS/JS files with PHP

3 years, 8 months ago

Update

I've created an updated version of the script described in this post which supports JSMin compression.

Introduction

Most sites include a number of CSS and JavaScript files. Whilst developing it's usually easier to manage them as separate files but on a live site it makes sense to merge files to reduce the number of HTTP requests the browser has to make. For JavaScript this is particularly important as browsers block rendering whilst downloading. It's also important to version your files to ensure that browsers download the latest copies when you've made changes.

I hate maintaining this stuff manually so I've written a PHP script which takes care of merging files on the fly whilst also versioning the merged file automatically as the various component files change. The file is merged on first request and cached. Subsequent requests are served the cached version. The script also sets HTTP headers to ensure the user's browser maintains each version in its own local cache therefore preventing repeated requests to the server. Finally an archive of the merged files is maintained to ensure that requests for old versions return the relevant CSS/JavaScript rather than the latest which might not match the user's cached HTML.

Using the script

Step 1: Start by setting the correct mime type for the files you want to merge.

  1. define('FILE_TYPE', 'text/javascript');

Step 2: Modify the $aFiles array to include the paths to the files you want to merge. These should be relative to the server document root.

  1. $aFiles = array(
  2. 'js/yahoo.js',
  3. 'js/event.js',
  4. 'js/connection.js',
  5. 'js/blog-search.js'
  6. );

Step 3: Set the location the script should write the archive files to. When first run it will automatically create the folder you specify if it doesn't already exist. For this to work you'll need to make sure that the parent directory, in this case "js", is owned (or is writable) by the user your web server runs as.

  1. define('ARCHIVE_FOLDER', 'js/archive');

Step 4: When called directly the script returns the merged code which you reference from your HTML source. For JavaScript files your HTML source should look something like this:

  1. <script type="text/javascript" src="/js/site_<?php require('combine.php'); ?>.js"></script>

When included via require the script returns the latest version number rather than the source. When rendered it will look like this:

  1. <script type="text/javascript" src="/js/site_1166088093.js"></script>

I've used a .htaccess file containing the following mod_rewrite rules to map this filename to the script.

  1. RewriteEngine On
  2. RewriteBase /
  3. RewriteRule js/site_([0-9]+).js js/combine.php?version=$1 [L]

If your host doesn't support .htaccess files you can rewrite your code to:

  1. <script type="text/javascript" src="/js/combine.php?version=<?php require('combine.php'); ?>"></script>

That's it for the set up. When you make changes to your source files the script will now take care of updating both the code served and the corresponding filename in the HTML source.

Caveats

If you subsequently add files to the script which have older last-modified dates than those already included they won't trigger a new version. I could have added code to support this but it would have significantly increased the complexity of the script. To trigger a new version simply touch or re-save one of the files.

Thanks

Thanks to Mike Davies for suggesting the archive functionality and for generally acting as a great sounding board.

Further Information

The following links were really helpful whilst putting together this script.

Feedback

The script is still very much work in progress. I'd really like to hear what you think of it - suggestions for improvements, problems etc.

Download

Download the script here

Comments

  • I really like this idea. Perhaps a future version could automatically run the js files through JSMin after combining them, so as a developer you only have to work with non-minimised code.

    Stuart - 14th December 2006 #

  • Ed -- Great work on this. Thanks for sharing it. This goes right in hand with what Tenni Theurer talks about in her 80/20 Rule on Performance article on YUIBlog the other day. Regards, -Eric

    Eric Miraglia - 14th December 2006 #

  • Nice ideas! Great! Fnx!

    Vitaly - 14th December 2006 #

  • You might be interested in the work we did on a related technique last summer:

    http://bestpractical.typepad.com/worst_impractical/.../smart_caching_f.html

    Jesse Vincent - 14th December 2006 #

  • Stuart and I discussed the idea of integrating JS Min this afternoon. I think it probably makes most sense to try and modify the PHP version to allow it to be called as an include rather than via the command line. I'll try and get something working over the next few days.

    Eric - thanks for your feedback and also taking the time to add a comment to the performance article on YUI blog pointing back to my post.

    Jesse - thanks for the link. I haven't had time to fully digest yet - I'll be taking a closer look.

    Ed Eliot - 14th December 2006 #

  • This looks nice, Ill have to test it out later. I did something similar for a previous project to reduce requests to the server. Slightly different approach - but still achieving the same goal.

    Nice work.

    Nate K - 18th December 2006 #

  • Thanks for the script. This is exactly what I was looking for to reduce the number of javascript files from having to be downloaded. Thanks again. :)

    Keef - 31st January 2007 #

  • Very cool, thanks!

    Just as a wee sugar on top, you could replace this part: // files to merge $aFiles = array( 'js/yahoo.js', 'js/event.js', 'js/connection.js', 'js/blog-search.js' );

    by: $aFiles = glob('js/*.js');

    Dieter Raber - 1st February 2007 #

  • Dieter - I considered doing that but there could be situations where the order that the JS or CSS files are merged in is important. If you merge with glob you can't be sure of this.

    Ed Eliot - 1st February 2007 #

  • If you have PHP prior to 4.3, :( then replace line 68 with

    $sCode .= implode('',file("$sDocRoot/$sFile"));

    file_get_contents() is 4.3.0 or later.

    thanks for this nifty script!

    Elaine - 21st February 2007 #

  • thats good ,, it have usefull script that contain some good ideas.. i think it will help me.. continue...

    hans - 26th February 2007 #

  • Thank you! Looking forward to giving this a try.

    Jen - 21st April 2007 #

  • Good stuff. It seems like, in a high performance situation, checking the files' last-modified dates on every page request might be an unnecessary performance hit. What about creating a cron job to determine the last-modified date periodically and putting into APC cache or something similar? That way the script can just check the cached value rather than calculating it every time.

    Mike D. - 23rd May 2007 #

  • I've created a WP plugin that optimizes CSS fully automatically. You might want to check it out. It can be found in this article I wrote about front end optimisation.

    Marco - 29th May 2007 #

  • Modern browsers multithreaded, I do not see this problem

    Jack - 28th November 2007 #

  • Just as a wee sugar on top, you could replace this part: // files to merge $aFiles = array( 'js/yahoo.js', 'js/event.js', 'js/connection.js', 'js/blog-search.js' );

    by: $aFiles = glob('js/*.js');

    +1

    oda kapisi - 18th January 2008 #

  • How can i use it with html?

    vim - 28th January 2008 #

  • Great script. I'm very impressed. It's given me some great ideas on how to break up my CSS and JS into logical chunks when developing but still have a larger streamlined version in production.

    The only problem I found was that explicitly setting Content-Length causes delays when the web server is using gzip, compress, deflate, etc. The browser is expecting the uncompressed length, and sits and waits for more data even though the server delivered it already in a compressed form. Watching response times in FireBug, it would always take 6+ seconds to deliver one combined JS file. Removing the Content-Length got it back down into the millisecond range.

    Thanks again.

    Steve - 13th March 2008 #

  • My suggestions: a. I want to run a server side code (perl in my case) only when the script is not compressed:

    ErrorDocument 404 /cache

    b. I want to make (ideally) only one request per page 1 for css, one for js): /assets/cache/css/one.css,two.css,three.css (I found this interesting technique, the joining line in my thought, here: http://rakaz.nl/item/make_your_pages_load_faster_by_combining_and_compressing_javascript_and_css_files)

    In this way I can run server side code only when necessary (Apache gives 404), and when the script runs: - it splits by comma - if not already done, it compresses each single file and store it under "cache" - it put all together the compressed files into a compressed big file containing commas: next request will not have to run the script!

    What are your impressions? Expires condition are missing yet, I promise I'll work harder ;-)

    Mauro - 26th May 2008 # 

  • <Location ~ /assets/cache/(js|css)>
    ErrorDocument 404 /cache
</Location>

    http://rakaz.nl/item/make_your_pages_load_faster_by_combining_and_compressing_javascript_and_css_files

    Mauro - 26th May 2008 #

  • "Modern browsers multithreaded, I do not see this problem"

    Yes, but modern browsers still only use one thread for downloading JS-files and halt page rendering or downloading of other assets while doing so :)

    Roy - 7th June 2008 #

  • this helped me in how to break up my CSS and JS into logical chunks..thanks very much , have a nice day....Nice site

    elena - 3rd August 2008 #

  • Nice idea. Tried it and it didn't generate anything and no javascript worked. This will be a pretty cool script when its up and running.

    cheers.

    Derrick - 20th September 2008 #

  • It's working great, thanks :) No JS errors

    Tommix - 1st November 2008 #

  • wrote something like this without the rewrite rules... have a php file that compresses my array of src files into outfiles (js/) using YUIcompressor then just wrote a php function for the inclusions at the template level.

    just uses filemtime() in php to append ?xxxxxx to the end of the file based on it's latest compilation time.

    ... many ways to skin a cat, like your approach though.

    chad - 29th December 2008 #

  • Thanks for this wonderfull piece of code!

    Do you plan to publish more complex version which overcomes 'older last-modified dates than those already included' problem?

    What about different array of files to merge depending on website section? I`d like to merge i.e:

    global.js, homepage.js - for homepage and global.js, archives.js - for archives page

    Currently this is impossible, assuming that global.js is the file which i change more often.

    kminek - 8th April 2009 #

  • We just incorporated this into our web site and it works like a charm. Although we use multiple stylesheets for a single system so I had to modify it to accommodate my needs. I've posted it here for anyone else who might need help as well.

    I've actually modified it so it caches multiple stylesheets based on the $_GET query string. If you want it to work differently, simply change the functionality within the function. It's a very simple process

    1) Add this snippet to the bottom of the file http://pastebin.com/f7f5e25cc

    2) Replace all $iETag.cache with ".getParsedQueryString()."$iETag.cache. Be sure you include the escaping for the function.

    3) Change $aFiles to be defined as $aFiles = explode(",", $_GET["files"]);

    Then all you have to do is pass your URL as combine.php?files=/lib/css/all.css,/lib/css/media/picture.css

    That will combine them into a single cache.

    Ben Kuhl - 13th May 2009 #

  • Sorry, above code was just modified....

    Use this function rather than the pastebin listed above. http://pastebin.com/f43344c68

    We use a global index.php for our entire site, and thus are caching based off of the query string. Here's our ONLY css include on the page:

    $css is a comma delimited list of paths to css files, $cssQueryString is the current page's query string ignoring numerical values

    IE - index.php?cmd=article&subsec=news&id=1 $cssQueryString is &cmd=article&subsec=news

    Ben Kuhl - 13th May 2009 #

  • 1) Add this snippet to the bottom of the file http://pastebin.com/f7f5e25cc

    2) Replace all $iETag.cache with ".getParsedQueryString()."$iETag.cache. Be sure you include the escaping for the function.

    3) Change $aFiles to be defined as $aFiles = explode(",", $_GET["files"]);

    Then all you have to do is pass your URL as combine.php?files=/lib/css/all.css,/lib/css/media/picture.css

    That will combine them into a single cache.

    çilingir - 17th July 2009 #

  • Hi all,

    I am trying to pass two parameters to the php file as shown below.

    Venkatesh - 17th March 2010 #

  • Hi all,

    I am trying to pass two parameters to the php file as shown below.

    Venkatesh - 17th March 2010 #

  • Hi all,

    I am trying to pass two parameters to the php file as shown below.

    <script type=text/javascript src=/js/combine.php?version=<?php require('combine.php'); ?>&ref_page=one></script> ref_page is the page from where it is calling. based on ref_page it is going to select the javascript files for combining them using below code but it is throwing the below error

    if($_GET['ref_page']=='one'){ $aFiles = array('js/js1.js','js/js2.js'); }elseif($_GET['ref_page']=='two'){ $aFiles = array('js/js3.js','js/js4.js'); }

    error:Failed to load source for: http://localhost/http-js-consolidator/combine.php?version=1268308057ref_page=one

    Venkatesh - 17th March 2010 #

  • I really liked your solution. Inspired by it I created a program called Squish: http://code.google.com/p/php-squish/

    It uses the file hash for the version number. It also lets you specify includes inside of a source file so you can specify dependencies.

    Dave - 20th April 2010 #

  • Hi, This is a very useful script I have been using across quite a few sites. Any ideas how to use this with WordPress? The require function creates an error here

    Viki Pandit - 14th August 2010 #

  • # wasn't paying enough attention to them (a classic case of not RTFM). I wish all projects did it this well. # Being able to query models from the python command line shell. It also doubles as a fairly quick (and I guess scriptable) way to add test data to your database.

    sohbet siteleri - 19th August 2010 #

  • Aşk herşeye değer seninle anldım ..

    izmir chat - 20th August 2010 #

  • thanks admin hi5'de çıkalım good blog

    hi5 - 29th August 2010 #

Help make this post better

Notes: Standard BBCode for links, bold, italic and code are supported. rel="nofollow" is added to all links. Your email address will never be displayed on the site.

Back to index