Empirical Testing of Custom Tag vs. CFC for Output

This is one of the drier posts I’ve ever written. And I’ve written some dullards. If you start to fall asleep, look at the pretty tables.

It is recommended to use CFC’s for “business logic” encaspulation, and leave UI encapsulation to custom tags. This is based on what we’ve read out on the Adobe Blogs about the subject. There’s one problem with that I dislike custom tags. I have no reason for it. I could argue that I prefer a few large files as opposed to many small files. This is probably why I jumped onto CFC’s so quickly.

Consequently, I’ve been doing UI encapsulation in CFC’s. Specifically, I put all of my reused UI components (headers, footer, navigation menus) in a single CFC named “interface.cfc.” I then push that to the application scope, and call it from there. I will totally own up to the fact that this is my personal preference.

However at a code review, my coworkers and I got into a discussion of the pro’s and con’s of using such a memory resident ‘output’ CFC . The discussion of to Custom Tag or CFC came up again. At the end of the discussion, I was willing to leave it at “It is a stylistic concern and not a performance based concern. ” But frankly, that was based on what I knew about CFC execution and gut feelings, but certainly not hard facts. So I decided to do some performance tests.

I created a custom tag that outputted the letter ‘H’ with no markup. I created a CFC with a method named test, that outputted the letter ‘H’ with no markup. I then created an index page that would loop through both calls the same number of iterations, and record the length of time it took to loop through the calling method at that iteration count. I tried three different methods:

  1. Straight Custom Tag Call
  2. Invoke on an CFC directly
  3. Invoke on an CFC that had been placed into memory as an object.

My hypothesis was that method number 3 would be the fastest performer, but that it wouldn’t be an order of magnitude thing, just a percentage thing.

Figure 1. Results of CFC comparisons on Development servers at 10 Iterations
Methodology Average Time (ms)
CFC Invoke from Memory 23
Custom Tag 33
CFC Direct Invoke 68

My first test upheld the hypothesis, but the differences there were pretty small so I fooled around with the number of iterations until I found one that ran quickly, but still showed clear patterns. The number I came up with was 1000. Below that it all method were too fast to see any difference.

Figure 2. Results of CFC comparisons on Development servers at 1000 Iterations
Methodology Average Time (ms)
CFC Invoke from Memory 2313
Custom Tag 4272
CFC Direct Invoke 5542

That continued to uphold my hypothesis. However, I was running it the code on a development server. There are differences between the performance of development and production servers. Debugging information gathered on our development servers tends to slow down CFC method execution. So I ran that same iteration test on a production-configured instance of ColdFusion that sits on our development server. It has debugging turned off. Other than that there is no difference, same hardware, same memory allocation. Here are those results.

Figure 3. Results of CFC comparisons on Production servers at 1000 Iterations
Methodology Average Time (ms)
CFC Invoke from Memory 39
Custom Tag 1262
CFC Direct Invoke 2649

That’s not a typo. On a production-configured machine CFC invocation from memory will run in about 3% of the time it will take Custom Tag execution to occur even for output components.

I figured I’d share these results and see if I’m on to something, or did I miss something major in my methodology. Does this change anyone’s mind on using CFC for output?

Source Files: If you would like to reproduce my experiment, please feel free to give it a try: Source Files

11 thoughts on “Empirical Testing of Custom Tag vs. CFC for Output

  1. I would love to see what would happen if the cfc and custom tag were more like a header include. Or some other part of your interface.cfc, rather than just h. Would the ratios hold?

    Like

  2. I’m not a big fan of testing like this (although I used to do it a lot a few years ago) as I feel it doesn’t take into account other factors that could potentially be far more important – like code maintainability and ease of updating. I just don’t put much faith in the “Run it a 1000 times and take an average” method of testing speed.

    That being said – it does make sense in that calling a custom tag will typically result in overhead from the scope creation (caller, variables, this) that a created CFC would be able to skip (it just creates arguments and the local scope for the CFC method execution).

    I’m probably rambing a bit – but I’d probably say use the method that works best for you and your team.

    Like

  3. Admittedly, it’s been a long time since i’ve done any serious Cold Fusion programming, so please excuse the question if it’s simple.

    What’s the trade-offs/difference between the CFC/custom tag method a cfinclude or reusable interface content?

    Like

  4. Ray: Obviously more goes into coding than performance. I wouldn’t say otherwise. But, my curiosity was mainly about performance. Do you have a method for testing performance that you rely on?

    Like

  5. Joe: Encapsulation technique like CFC functions and custom tags are in general better for this type of operation because they allow for variable passing between the calling document and the encapsulated code while maintaining a separate set of scopes for the encapsulated code. Cfincluding a template will encapsulate it, but you can’t pass in variables. (However you can set them before you call the included file.) For example, let say I had an encapsulated navigation bar, and I wanted it to behave differently on different pages. With cfinclude, I would have to set a variable somewhere before I called the included code to indicate which page it was on, or use the cgi.scope to determine it. With other encapulation techniques like custom tags and CFC’s you explicitly pass them. Does that make all make sense?

    Like

  6. Interesting. I ran your tests on my Mac and, in “development” mode, I saw the in-memory CFC invocation be anything from 10-50% faster on repeated runs. In “production” mode (i.e., with trusted cache enabled), the times were much closer between in-memory CFC invocation and custom tags. My in-memory CFC invocation tests typically ran in around 70 (prod) to 100ms (dev). My custom tag invocation tests typically ran in around 70 (dev) to 200ms (dev). Therefore I’m suspicious of your custom tag timings on your systems since your production system is an order of magnitude slower than my laptop (whereas the in-memory CFC invocation is in the same ballpark).

    FWIW, the direct CFC invocation was way slower on my system than on yours, both with and without trusted cache.

    Definitely an interesting observation tho’!

    My main argument for not using CFCs for output (and preferring either custom tags or simple CFML pages) is that you can’t really have a graphic designer / web production person editing your CFCs, whereas they can easily edit custom tags and CFML pages in their regular WYSIWYG HTML editor. That may not be important in your environment but it’s important in mine 🙂

    Like

  7. Er, that first piece isn’t very clear:

    … anything from 10-50% faster … [than the custom tag invocation]

    And then an error:

    My custom tag invocation tests typically ran in around 70 (dev [I meant prod of course]) to 200ms (dev).

    Like

  8. Sean: First off, thanks for giving me the first practical explanation as to why CFC’s are less preferable than custom tags for output. I’ve never heard that reason before, and while it doesn’t apply to my environment, I can see why it would have an impact on yours. As for your results. I can see I left the trusted cache off on my production instance. I will re-run after tweaking that, and post my results in a day or so.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s