Tag: ColdFusion

Posted on

Optimizing site speed: a case study

One of our clients runs an eCommerce site selling workout videos on behalf of George St.Pierre, the UFC Fighter, called GSPRushfit. The videos sell well, but they’re quite rightly always looking at ways to increase sales. A few weeks ago, we ran a study to see how fast the site loaded, and how that affected conversion rates (sales). I wrote a post about how we measure that a couple of weeks ago. The result of this was that we could see that page loaded reasonably well, but not fantastically. Across all pages, the average load speed was 3.2 seconds. What was eye-opening was that pages that loaded in 1.5 seconds or less converted at about twice the rate of pages loading 1.5-5 seconds. There was  a further dip between 5-10 seconds. So with this data in-hand, I started to look for ways to increase page load speed. I came up with a laundry list of things to do. Most of these are suggested by YSlow:

  1. Remove inline JS/CSS: We didn’t have a lot, but there was some unnecessary inline scripting. These were moved into the existing CSS & JS files. I think I added about 50 lines of code. Not a lot, but helpful. There’s still some inline JS & CSS that’s being written dynamically by coldfusion, but all the ‘static’ code was moved into one.
  2. Minify Files: This doesn’t do a lot, but does compress files slightly. I think I was able to reduce our JS file by 30% & our CSS by 15%. Not a lot, but still helpful. I use an app called Smaller, which I’m a big fan of. While YSlow suggests you combine files, I chose not to – the reduction in requests didn’t offset the problems for us in doing this.
  3. Reduce Image Size. The site is fairly graphically intensive – large background images & lots of alpha-transparency PNGs. When I started, the homepage was loading just under 1.2MB in images, either as CSS backgrounds or inline. Without (to my eye) any noticeable loss of quality I was able to re-cut those same images to about 700KB in size.
  4. Use a CDN: The site loads video, which we call from a CDN. But the static assets (CSS, JS, Images) weren’t being pulled. This was originally because the CDN doesn’t support being called over SSL. But it only took a little scripting to have every image load from the CDN while not on SSL, from the local server while over SSL. This, as you’d expect, greatly improved the response time – by about 0.2 seconds on average.
  5. Query Caching: this one is likely a no-brainer, but the effects were stunning. All the content is query-driven, generated by our CMS. But it doesn’t change terribly often. So I started caching all the queries. This alone dropped our page load time by nearly a full second on some pages. And to maintain the usefulness of a CMS, I wrote an update to clear specific queries in the cache when new content was published.
  6. GZip: Again, likely something I should have already been doing, but to be honest, I had no idea how to accomplish this on IIS. So I figured that out and requested that the server gzip static assets (JS, CSS & HTML files).
  7. Far-future expires headers. Because very few of our images change frequently, I set an expiry date of 1 year in the future. I likewise set a cache-control variable of the same time frame. Which, should, in theory, reduce requests and allow clients revisiting to just use their local cache more. I of course added a programmatic way to clear that cache as well for when we change content or edit the JS or whatever.
  8. Clean up markup: While I was doing all the rest, I also cleaned up the markup somewhat – not a lot, but again, we were aiming to eliminate every extraneous byte.

So, as you can see, we sort of threw the kitchen sink at it to see what stuck. In retrospect, I wish I had made these updates one at a time to measure what sort of an impact (if any) each one had. There’s only a couple I can see clear before & after differences, which were mentioned above. So for everyone out there, which of these were the most effective?

  1. Caching Dynamic Content: Even on a CMS-driven site, most content doesn’t change constantly. But if you can eliminate trips to the DB server to make a call, that’s time saved. Even if you cache a recordset for a few minutes or even just a few seconds, on a high-traffic site you can see some real impressive gains. We cache queries on this site for 10 days – but can selectively update specific queries if a user makes an edit in the CMS – sort of a best of both worlds right now. This somewhat depends on having a powerful server – but hosting hardware & memory are pretty cheap these days. There’s no reason not to make use of it.
  2. Crushing images: When building the site, I did my best to optimize file size as I exported out of Photoshop. but with a few hours in Fireworks I was able to essentially cut the size of the images in half with no real visual impact. A hat-tip to Dave Shea for the suggestion of using Fireworks.
  3. Pushing Content to a CDN: this is the head-smacking no-brainer that I don’t know why wasn’t already part of our standard workflow on all sites. As I wrote above, we gained about 0.2 seconds by doing this – which doesn’t sound like a lot, but it’s noticeable in practice.

The nice thing about this exercise was that it shook up how I built sites, how our internal CMS runs and how I configure both our IIS and Apache servers to all run slightly more efficiently. I suspect that I could eke out a few more milliseconds by playing more with the server settings itself, but I’m satisfied for now with how this worked out.

Posted on

Coldfusion Java.lang.ThreadDeath

Of  late, we’ve stared seeing a raft of errors of type “java.lang.theadDeath” on our Coldfusion-based servers. In every case, these errors were generated by the CFSchedule User Agent, meaning that scheduled tasks were triggering them. Of more interest, they were, with 1 exception, being thrown by a scheduled CFPOP call to handle email newsletter bounce-backs by our eNewsletter delivery system. The scary part is that, more often than not, our server would start to hang shortly after getting a bunch of these and we’d have to restart the service. Not ideal on servers that require very uptime on account of some exceedingly busy eCommerce sites.

After casting about fruitlessly on Google & Bing for some answers to this, I buckled down to investigate the errors more closely myself. And I believe I have figured it out:

Each of our websites has a Cron handler: We add in any number of scripts we wish to call, and interval between & whether it is allowed to run concurrently with anything or not. The central ColdFusion scheduler service then simply calls this handler for each site at regular intervals (generally, we call the handler about every minute. The tasks themselves range from every 2 minutes to monthly). This handler runs multi-threaded – that is to say that, if a task is allowed to run concurrently with other tasks, it’ll spawn a thread for each task, run it, then tie up the threads afterwards.

So here’s where I think this issue arrives from: According to the very scant documentation and blogs I could find, this error-type will be handed off when a thread is killed off by the server after being queued for too long, awaiting execution, or for the loser in race conditions. We had our server to allow for 10 concurrent threads. Which is pretty small, but seemed ok. My guess is the connection to the mail server when popping messages runs threaded itself. So we were spawning massive number of threads. Further to that, our allowed timeout on mail server connections was longer than our allowed request time or thread-queue time. So threads were constantly timing out, or being killed off because they were queued for too long – which would then spit that error.

Given that we have oodles of RAM, I’ve both upped the number of concurrent threads we allow, as well as reduced the mail-server connection timeout to be less than both our request-timeout and queue-timeout. Testing throughout today appears to have solved this, but I’ll have to watch & see over the next few if indeed this is now solved.

Posted on

FCKeditor & Firefox 3.6

With the release of Firefox 3.6, those of you who interact with FCKeditor will note that it ceases to work, giving you instead a textarea. This is because in the browser sniffer, FCK ignores all browsers made in 2010 or later – such as Firefox 3.6. I can’t speak for any other versions, but in the Cold Fusion version, there’s a really simple fix:

  1. Locate fckutils.cfm, located in the root folder in the install.
  2. On line 47, edit the line that reads
    stResult = reFind( “gecko/(200[3-9][0-1][0-9][0-3][0-9])”, sAgent, 1, true );
  3. Edit the reFind to read:
    stResult = reFind( “gecko/(20[0-9][0-9][0-1][0-9][0-3][0-9])”, sAgent, 1, true );

This will give you another 90 years. Which should be plenty of time to find a replacement, particularly as the new CKeditor is now out, and doesn’t have this issue.

Posted on

Using fonts with CFImage

One of the more useful bits of ColdFusion (version 8 and above) is the ability to dynamically write text as an image. As you can imagine, there are nearly endless uses for this. However, one of the more curious things is that it is not that easy to determine which fonts are available for you to use on the system on the fly. You can do it in the Administrative Console, but not programatically. . Worse, if you specify either a non-existent font or one that is not usable, ColdFusion throws a really ugly error. In the Pencilneck CMS, we have a number of bits of code that allow both ourselves & our users to generate images from text on the fly, and so it has become important to know what fonts are available. To that end, I wrote a pair of scripts today that now handle this for me, which I’m going to share below. One important note: In order for this script to work, you must have administrative access to ColdFusion, as it uses one of the Admin API components. The first script simply loads a list of all the valid, usable fontnames in a list (my apologies in advance for this not being properly indented. I’ve not yet found a good way of writing code with WordPress):

<cffunction name=”getServerFonts” access=”public” output=”false” returntype=”string”>
<cfargument name=”loginPW” type=”string” required=”true” />
<cfset var x = structNew() />
<cfscript>
adminObj = createObject(“component”,”cfide.adminapi.administrator”);
// this shoud be the CF admin log in
adminObj.login(arguments.loginPW);
myAdmin = createObject(“component”,”cfide.adminapi.runtime”);
x.fontList = myAdmin.getfonts();
</cfscript>
<cfset x.basicFontList = “” />
<cfloop collection=”#x.fontlist.systemfonts#” item=”x.i”>
<cfloop collection=”#x.fontlist.systemfonts[x.i]#” item=”x.j”>
<!— we disallow the “Adobe Built-in” fonts because CF can’t use them. This list is inclusive, not exclusive simply by my own preference —>
<cfif ListFindNoCase(“TRUETYPE,TRUETYPE-COLLECTION,OPENTYPE”,x.fontlist.systemFonts[x.i][x.j].fonttype)>
<cfset x.basicfontList =ListAppend(x.basicfontList,x.j) />
</cfif>
</cfloop>
</cfloop>

<cfloop collection=”#x.fontlist.userfonts#” item=”x.i”>
<cfloop collection=”#x.fontlist.userfonts[x.i]#” item=”x.j”>
<cfif ListFindNoCase(“TRUETYPE,TRUETYPE-COLLECTION,OPENTYPE”,x.fontlist.userFonts[x.i][x.j].fonttype)>
<cfset x.basicfontList =ListAppend(x.basicfontList,x.j) />
</cfif>
</cfloop>
</cfloop>

<cfreturn listSort(x.basicfontlist,”TextNoCase”) />
</cffunction>

So I like to run this script once on init, and load this list of fonts up into server scope so it’s available to all my applications. Once that exists, it becomes trivial to check for the existence of a font, which I do with the following script:

<cffunction name=”isSystemFont” output=”false” access=”public” returntype=”boolean”>
<cfargument name=”fontName” type=”string” required=”true” />
<cfset var x = structNew() />

<cfif NOT isDefined(‘server.fontList’)>
<!— do this in case the server.fontList hasn’t been initialized yet. If you load your fonts into a different scope, edit this —>
<cfset server.fontlist = getServerFonts() />
</cfif>
<cfif ListFindNoCase(server.fontList,arguments.fontName)>
<cfreturn true />
<cfelse>
<cfreturn false />
</cfif>
</cffunction>

And that’s it. I can now safely test to see if a font is available on a server or not, and then have conditionals by preference, etc. or whatnot. Hopefully, this’ll be useful for you too in your adventures in coding.

Posted on

Better redirection

For sites powered by the Pencilneck CMS, we’ve for ages and ages had what we called “Friendly URLs”. These are common across CMS’s, blogs, etc (some ironically, I don’t use them on Tannock.net because I’m too lazy to reconfigure my config & htaccess files). Back in the day, we did this by physically writing a folder to the server. Which was not ideal. So we then switched to redirected using ColdFusions onMissingTemplate() handler. In this instance, we have the IIS server trigger a 404 error, which calls a custom CFM file, which does not, in fact, exist. This triggers the sites onMissingTemplate() handler to actually read the URL and load up the appropriate page from the DB. This works well, but there’s a fair amount of overhead. Vs. the physical stub option, it added between 400-600 ms – not a huge amount, but enough to cause a noticeable overhead on busy sites.

But a while back for a larger project, we installed the Helicon ISAPI rewrite module for IIS. I’ve been using this for a variety of things on sites, such as locking down directories for documents, redirecting particular scripts elsewhere, etc. But I hadn’t set it up to handle our friendly URL system.

This past weekend, I finally spent the time to test out the difference – and it’s impressive. Despite running exactly the same script as the current OnMissingTemplate() handler calls, it runs much faster – on average, 80-100 ms (vs. 400-600 previously) – a significant decrease – I’m guessing because there’s overhead from IIS to generate the 404, hand that off to the custom 404 page, which doesn’t exist, so hands it off to the onMissingTemplate() handler which then calls this script. Now, the the ISAPI rewrite condition kicks in and it calls the script directly.

And now all of our sites are slightly more responsive, which is better for everyone!

Posted on

Installing root Certificates for ColdFusion

This another one of those post-this-here-so-I-don’t-have-to-google-forever items.

In May of this year, Verisign introduced a new intermediate root certificate. This september, Paypal renewed their own, forcing API users to be using this new root certificate on their end for all API calls. So that’s all fine and dandy, the solution is pretty easy:

  1. Go to Verisign & download the new root certificates
  2. For me, the critical one is the ‘Class 3 Public Primary Certification Authority – G2’ – to be used with the Paypal Payflow API
  3. Install it. For installation to IIS, this is straightforward – simply click on the .cer file and follow the instructions. However, for Coldfusion, this process is a little more involved:

All of the following instructions can be found on Talkingtree.com – I’m rewriting them here for clarity and how I had to do it:

  1. Grab the newly downloaded .cer file and copy it to your {CFROOT}/runtime/jre/lib/security folder (or the equivalent location if you’re using a different JRE).
  2. Run the Keytool function to install it. Keytool exists in {CFROOT}/runtime/jre/bin, using the following path:
    keytool -import -keystore cacerts -alias {UniqueName} -file {filename.cer}
    the  Uniquename can be anything – I recommend something that references what the cert is for. The filename from above is “Class 3 Public Primary Certification Authority – G2.cer”
  3. The keytool will ask for a password. By default, in CFMX7, the password is “changeit”
  4. restart CF server

Happy connecting!

UPDATE 2009-10-23: Sarah Kelly has a great post about why doing this is important, beyond just the how, so read that too.

Posted on

tips for running ColdFusion on Mac os X (Leopard)

My home computer has been a Mac for some time now, although I still use a PC at work. This means that I’ve been switching back and forth between dev environments, and have learned a wack of things about getting ColdFusion 8 set up on a Mac. Some of these are more general “dev on a mac” tips, but apply, so here goes:

  1. Run ColdFusion from the command line: 
    cd /Applications/JRun4
    sudo start cfuson

    Leave the terminal window open, and you’ve got a great debugger and view into what’s going on behind the scenes. Also – always run ColdFusion as root, or you’ll get the occasional odd error, to do with permissioning.

  2. Set up Virtual hosts: this is a general local-dev tip, but I find it even more helpful in the Mac/Apache environment than in the PC environment. You’ll need to edit your hosts file (found in /private/etc), and add in an entry per site, in the following format (where [sitename] is the name of your site): 
    127.0.0.1 [sitename].local

    which will allow you to browse to http://[sitename].local/ to view your site – much nicer than http://localhost/[sitename]/.

  3. Configure Apache to support Virtual hosts: To this to allow yourself to create aliases, custom 404 errors, etc. This is done in the following way:
    1. Edit httpd.conf. You’re looking for line (somewhere near the bottom, that looks like: 
      # Virtual hosts
      #Include /private/etc/apache2/extra/httpd-vhosts.conf

      Edit the second line to remove the #, so it looks like:

      # Virtual hosts
      Include /private/etc/apache2/extra/httpd-vhosts.conf
    2. Edit the httpd-vhosts.conf file. In there, you’ll find a sample virtual host. Delete, or comment that out, then add the following line: 
      Include /private/etc/apache2/vhosts/*.conf

      Then save and close the file.

    3. Create a new folder in /private/etc/apache2 called “vhosts”, then create a new file called [sitename].conf.
    4. Open [sitename].conf, and put in the following: 
      <VirtualHost *:80>
          ServerAdmin webmaster@[sitename].local
          DocumentRoot "/Path/to/[sitename]"
          ServerName [sitename].local
          ErrorLog "/private/var/log/apache2/[sitename]-error_log"
          CustomLog "/private/var/log/apache2/[sitename]-access_log" common
          ErrorDocument 404 "/404Custom.cfm"
          DirectoryIndex index.cfm index.html index.htm
         <Directory /Path/to/[sitename]>
          Options Indexes FollowSymLinks
          AllowOverride All
          Order allow,deny
          Allow from all
          </Directory>
      Alias /cfide /Library/WebServer/Documents/CFIDE
      </VirtualHost>
    5. If you need to add other aliases, do so in there. Then save the file.
    6. restart apache, with the following command: 
      sudo apachectl restart
    7. You’re done! You should now have a happily running ColdFusion site on your local box.
  4. Set up Custom 404 handlers/stubless folders: This is all about getting mod_rewrite working. To do this, open httpd.conf and ensure that the mod_rewrite module is NOT commented out. Then, in your site folder, create a .htaccess file, with the following in it: 
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule (.*) /404Custom.cfm?%{REQUEST_URI}

    You can now use your 404Custom.cfm script to redirect and handle those 404s however you like.

  5. Avoid Lazy coding: If you were like me, you likely had some code that referenced cgi.path_info to figure out where a script was. This won’t work on Mac OS X. Fortunately, the cgi.path_translated works just fine – only remember that this includes the file. Likewise, if you’re directory crawling, *nix paths use “/” to separate folders, whereas windows paths use “” it’s super easy to store a variable for if/when you’re doing file & directory management work, and will make your code that much more portable. Plus, you’ll likely avoid the 8-odd hours of frustration I had when I first switched, and didn’t realize that I had left some windows-specific pathing in some old code.
  6. Tell me how to get cfchart working: I don’t know what the issue is here, and I’m sure it’s something I’ve done, but I cannot seem to get flash-based cfcharts to run on Mac OS X – if you have, please tell me, it would make me very happy.

Happy Coding!

%d bloggers like this: