I will be speaking about Squidhead to the Nashville CFUG Thursday January 29th, 2009. I’ll be speaking over Adobe Connect.
I don’t start working for Adobe until Monday February 2nd. So I won’t be revealing anything secret or anything, but I will set aside a few minutes if anyone wants to bring anything to my attention.
One of the biggest complains I hear from management types about ColdFusion is that they can’t hire good ColdFusion developers. I think that this occurs because people often overlook one of ColdFusion most accepted benefits. ColdFusion is exceedingly easy to learn. Which leads me to my biggest piece of ColdFusion hiring advice – don’t look for good ColdFusion developers, look for good web developers, if they know ColdFusion, great, if not, teach them, or let them learn it.
The argument I hear back on this point is “we don’t have the time of resources to train someone; we need them to hit the ground running.” I think this is penny wise pound foolish. Based on anecdotal evidence only, I would contend that it takes an average web programmer about 1 or 2 months to learn ColdFusion. (This assumes they know HTML, CSS, SQL and another server side language.) I’ve seen some ColdFusion job hunts take upwards of 9 months. You can’t afford the month to train an employee, but you can be without them for 9 months? That doesn’t make a lot of sense to me.
Add to it that for the cost of 1 or 2 months, you get a ColdFusion developer who knows another language. Developers who know more languages tend to be better. ColdFusion programmers that know Ruby for example are usually better Object Oriented ColdFusion developers through the knowledge they picked up in Ruby.
Now, I’m not arguing that you shouldn’t hire a ColdFusion developer if you can find one. I happen to know a few that are looking for work. I also don’t mean to suggest that finding a good web developer is necessarily that much easier than finding a good ColdFusion developer. Finding “good” people is never easy, but I am arguing that you need to increase your chances of finding someone to fill your position. You can do this by opening up your search criteria, and letting one of the major selling points of ColdFusion actually work for you.
What do you think, does this agree with what you’ve seen in the job market of late?
Wow that took less time than I thought… So yeah, I’m joining Adobe. Specifically I’m joining Adobe’s Platform Evangelism group. I’ll be working under Kevin Hoyt with the team that includes Ryan Stewart, Lee Brimelow and Danny Dura amongst others and ultimately headed by Ben Forta. So it will take all of my composure to not, you know, break down into an Adobe fanboy in my first staff meeting.
What does that mean? It means that I will be working with the rest of the team to spread excitement about the Adobe Platform Products:
I’ll be promoting the entire platform, but considering my experience to date, I imagine that I’ll start with a slight focus on ColdFusion and AIR.
However in addition to that focus I will have an overriding goal:
Get Adobe Platform Technologies taught in the classrooms of Higher Ed.
It’s a big goal, and not a trivial challenge. I see a lot of different paths to achieving it. I can’t wait to work with all of you to accomplish it.
And as my first act of Evangelism I will remind you once again that both ColdFusion and Flex Builder are available free to Higher Education. All you have to do is go to one of their respective “freeriatools” sites, fill out a form, and upload a picture of your Academic ID.
I’m working with the ConstantContact API webservice, and ran into a slight problem. The API uses Digest Authentication, and ColdFusion does not support Digest Authentication in cfhttp. I found this out by reading in the live docs that:
The cfhttp tag does not support NTLM or Digest Authentication.
The ConstantContact forums pointed me to a custom tag cfx_http5. But I hate CFX tags. I like to place as few dependencies on the ColdFusion Administrator as possible when I code. CFX tags also tend not to make the trip the first time in server migrations or emergency moves. It might be my own prejudice but, no, there will be no custom tags.
Which leaves me to implementing some Java client, as luck would have it there was ample documentation on doing this. I just needed to use the open source Apache HTTP client. To work with them, I had to add them to the class path, or add an entry in the ColdFusion Administrator class path list. That would have bought me nothing, as I didn’t want to go through the administrator.
This left me looking for a way to dynamically call jar files. If only there was some way to dynamically call jar files in ColdFusion… preferably written by someone with a marsupial pouch…. Ohh wait, Mark Mandel wrote the Open Source project JavaLoader, which allows just that. Hmm, and Mark is a mammal from Australia. Things are looking good.
So I was able to build my http client with Digest Authentication. I added some code that formatted the response like a cfhttp struct, so that if there were other advantages to this method I could just swap out this for cfhttp in a pinch. The code is below in the extended section of this blog post.
I was trying to do 3 things with this post:
I hope I’ve done these three things for you.
var paths = arrayNew(1);
var rootPath = GetDirectoryFromPath(GetCurrentTemplatePath());
paths[1] = rootPath & "/jars/commons-httpclient-3.1.jar";
paths[2] = rootPath & "/jars/commons-codec-1.3.jar";
paths[3] = rootPath & "/jars/commons-logging-1.1.1.jar";
//create the loader
variables.loader = createObject("component", "javaloader.JavaLoader").init(paths);
var result = "";
var credentials = loader.create("org.apache.commons.httpclient.UsernamePasswordCredentials");
var HttpClient = loader.create("org.apache.commons.httpclient.HttpClient").init();
var httpGet = loader.create("org.apache.commons.httpclient.methods.GetMethod");
var AuthScope = loader.create("org.apache.commons.httpclient.auth.AuthScope");
var jURL = createObject("java", "java.net.URL").init(arguments.url);
if (len(arguments.username) and len(arguments.password) gt 0){
AuthScope.init(jURL.getHost(), arguments.port, arguments.realm);
credentials.init(arguments.Username, arguments.password);
httpClient.getState().setCredentials(authScope, credentials);
}
httpGet.init(arguments.url);
httpClient.executeMethod(httpGet);
result = convertHttpClientResponseToCFHTTPFormat(httpGet);
httpGet.releaseConnection();
return result;
var result = structNew();
var responseheader = structNew();
responseheader['Status_Code'] = httpGet.getStatusCode();
responseheader['Explanation'] = httpGet.getStatusText();
responseheader['Http_Version'] = httpGet.getEffectiveVersion().toString();
header = httpGet.getStatusLine().toString();
headers = httpGet.getResponseHeaders();
for (i=1; i lte ArrayLen(headers); i=i+1){
responseheader[getToken(headers[i], 1, ":")] = getToken(headers[i], 2, ":");
header = listAppend(header, headers[i], " ");
}
result['Charset'] = httpGet.getResponseCharSet();
result['ErrorDetail'] = "";
result['Filecontent'] = httpGet.getResponseBodyAsString();
result['Header'] = header;
if (structKeyExists(responseheader, 'Content-Type')){
result['Mimetype'] = GetToken(responseheader['Content-Type'], 1, ";");
}
result['Responseheader'] = responseheader;
result['Statuscode'] = responseheader['Status_Code'] & " " & responseheader['Explanation'];
if( not structKeyExists(responseheader, 'Content-Type') OR
FindNoCase("text", responseheader['Content-Type']) OR
FindNoCase("message", responseheader['Content-Type']) OR
FindNoCase("application/octet-stream", responseheader['Content-Type'])
){
result['Text'] = "YES";
}
else{
result['Text'] = "NO";
}
return result;
Spurred on by Kevin Hoyt, who commented on my last post I have some predictions for technology in higher education as a whole.
First, a quick background. The economy sucks. This is good news and bad news for higher ed. Enrollment and applications will be up. Bad news, is that most institutions arenapost entirely funded by tuition. Money comes into schools from state governements and the federal government in addition to money from their endowments and donations. All of those sources will be down in the next two years at least. Also in the picture is that fact that because of institutional culture of tenure, and the compliance issues that come from taking federal money, most higher ed institutions cannot cut budgets by trimming employees from the books. Good news, you’ve still got your job, bad news, remember Ed who quit, you’ve got his too.
Cost savings will be sought elsewhere. This is going to lead to a contraction of commitments. Upgrades will be postponed. Technology investments (big virtual machine hosting hardware, SAN’s) will not be pursued.
So what will all this lead to…
Hosting
Up until now, a penchant for open source technology, and educational discounts have made schools prefer to host their own services a little longer than the rest of technological community. However, that was before the financial craptaculon. External hosting will be considered for applications which would never have been outsourced before. It’s already started for student email, IM, web sites, and storage. It will accelerate and be targeted for smaller burdens like staff email, intranet sites, and public websites.
Virtual Classrooms
Virtual classrooms where schools could sell an unlimited number of seats to an audience for a set cost has been a holy grail for a few years now. With cut travel budgets hitting continuing education dollars, and the rise in applications coupled with the pressure to bring in revenue, they will have no choice to squeeze every dollar they can. Expect a lot of activity here.
Development Language free for all
Because of the rise of hosting for previously unhosted solutions, there will be no lock in for particular platforms or languages. Want to try Ruby or make the switch to .NET? No worries, you don’t have to build an environment, just rent it. It provides opportunities for highly productive languages and solutions to re-argue their case to the higher education world.
Students
Students are going to be looking to either stay out of the job market for as long as possible, or get trained in skills that will more likely get them hired. Any schools out there have Flex or Ajax content in their program, congrats, your students are still marketable. Expect more focus on vocational skills, like specific languages to be the rage.
With the new year fast approaching my mind turns to what’s coming up in the next year.
Centaur
Centaur is slated to be released in 2009 as ColdFusion 9. I’m confident it will be. ORM and the IDE will be the most talked about features. Although ORM is talked about, it will be slow to move in to the mainstream of CF development. Considering the time it took to get CFC’s to be widely adopted, I think ORM will be what’s hot in 2010 but bleeding edge in 2009.
The IDE will be a big seller, and I mean seller. I think the IDE will be a for purchase and stand-alone product. I think there will be a lot of people developing on Bolt and Centaur for production ColdFusion 8 applications for some time to come.
ColdFusion Development
Apple will continue to gain market share among CF developers. Ubuntu boxes will also proliferate as Vista users with regrets can’t justify new hardware purchases but tire of Vista’s drawbacks.
Web Development in General
Rails and Merb joining together will cause the Ruby development community to get a major boost. The meme that “rails is cool, but not ready for enterprise” will really get challenged.
Technology in General
There will be an increase in problems due to companies putting off equipment upgrades in an attempt to cut costs. I think companies will continue to push more hardware outsourcing regardless if they experience a cost-cutting driven failure or not.
That’s all I have for now, if I think of more I’ll add…
For reasons I cannot entirely explain, I was trying to call dynamic methods in a cfc in cfscript. (The cfc code was calling a method within itself) I tested a few syntactically wrong ways of getting it done. It the underlying fact that functions are another type of variable in ColdFusion hit me. I should be able to assign dynamic variable to a new variable, and call that as a function. It worked.
<cfscript>
function test(input){
return input;
}
functionName = “test”;
args.input = “Yo!”;
tempfunc = variables[functionName];
output = tempfunc(argumentCollection = args);
writeOutput(output);
</cfscript>
I feel like someone must have done this before, but could find nothing on it. So I figured I would blog it.
I added a huge new feature to Squidhead, which I think deserves some mention. I’ve added another that’s not as cool, but still worth mentioning. I think I’ll try and get it published out to Squidhead users in the next few days.
Feature: Paged and Sortable List views
Basically this feature adds a paging view to the list view, and that view is sortable by each column in the shown in the view. The compelling part of this feature is that it isn’t doing the sorting and paging in ColdFusion, it writes a stored procedure that can present a sorted query of a particular page size. Because this happens in the database it’s blazingly fast.
Writing these types of stored procedures is really annoying. SQL 92 specification prevents variables in the LIMIT, TOP, and ORDER BY clauses. In order to pull down paged lists of records from the database requires adding an extra temporary column that stores row count and only including particular ranges. Then to do order you have to do a switch case block for every order by possibility. It’s the annoying, repetitive code that was designed just so someone could write a code generator to automatically write it.
Feature: List View Filter
The list view filter feature allows you to configure what columns get displayed in the list view of your generated applications. One of the first things that I do when I go to turn a generated application into a real application is to start manually limiting which columns get shown as inevitably there is a text field that needs to go. This can now be done by simply altering the table.xml that is generated by Squidhead, and setting the excludefromlistview property to true.
Northwind and Sakila support
This isn’t really a feature per see, but I’ve started testing against Northwind for MSSQL and Sakila for MySQL. Both of these are sample databases provided by the vendors. They expose a wide range of database features for each one. By doing testing against them, I found a number of errors and bugs in the Squidhead code that have been rectified. Going forward, Squidhead features will not be released unless they pass these tests.
Like I said earlier, I’d like to get these features out in the wild in the next few days.
Today the Knowledge@Wharton tech team put into the wild something I’ve been working on for some time: a new platform for Knowledge@Wharton and India Knowledge@Wharton. The new platform consists of the following:
Windows 2008 Load Balanced Cluster
We built a two node cluster using Windows 2008 64 bit Enterprise Version. One node is a VMware instance, and one node is a blade server. I like this configuration as I only have to worry about a machine warrantee on one node, but I have the backup of a hardware-based node if something goes wrong with our VMware installation. Not that such an event is likely; I would prefer not to tempt fate.
I’ve mentioned it before that we don’t use Load Balancing so much for load as for availability. By having dual node clusters for our production environments we buy ourselves zero downtime patch cycles. We did have a little trouble getting NLB on Windows 2008 working, but we did get it fixed after talking to Microsoft support.
The upgrade went really smoothly. I’m used to using cnames to handle this sort of move, but due to SSL considerations knowledge.wharton.upenn,edu has an A record. The easiest way to make the change was to add the new nodes to the existing Windows 2003 cluster, then remove the windows 2003 nodes. It worked like a charm, and I think it will be my new procedure as it was shockingly easy.
Core Services Code Base and ColdFusion 8
In looking to upgrade Knowledge and India Knowledge to ColdFusion 8 I had to touch a lot of the code. Not so much because there was a problem with it, but because we wanted to take advantage of new features. In the course of doing that I discovered that the main Knowledge site and India contained a lot of duplicated code between them. I was able to centralize it and then add new features to both sites. There are two main features that I added to the central code base: cached queries and search driven folksonomy.
Caching the queries was pretty trivial. I rolled my own instead of using an existing caching framework or native ColdFusion caching. I wanted an easy to flush cache system that didn’t need to be too complex. Because of the highly normalized nature of the database, I couldn’t get a tremendous performance boost through indexing; caching however has proven to be the correct solution by a long shot. It makes sense; we have a lot of frequently read, rarely written data here. I’m just surprised at the overall boost to the site we accomplished with one fix.
“Search driven folksonomy” is a cool idea that my boss Dave had back in 2006. It was running for awhile then got deactivated for some reason and I just re-implemented it. Basically, instead of having people manually tag articles, instead use our search referral keywords to tag articles automatically, then when an article hits some sort of critical number of hits for a keyword to an article that keyword becomes a tag on that article. We’ve enabled the collection piece for now and will enable tag display once we tweak the model a bit after getting some real data.
Layout
I can’t take credit for the look and feel. This was done by Dave and a co-worker, Sanjay. They worked on pushing Knowledge to a more current centered layout, along with a few other tweaks to accommodate advertizing without compromising the editorial content.
The one thing I contributed here was a custom tag that converted an article to an array of
tags. Then the article custom tag was able to wrap around other custom tags and display them in the flow of the article at set positions in the array, or the next pre-determined location in the array, or at the end. It made for a very flexible way to showcase link suggestion or article tools within the flow of the article thereby freeing up space for the aforementioned ads.
Development and Publishing System
This was the hardest to tackle part of the whole thing. Because I was asking people to change the way they worked. But Dave and Sanjay were open to it, especially since I promised that it would make their lives much easier after a little bit of pain.
The old model consisted of doing development on a shared development server with no source control. Changes were manually pushed to production. Occasional copies were made of the code. Communication about changes were ad-hoc and not necessarily as frequent as the changes.
The new model pushes development to local installs of ColdFusion. Source control is handled through Subversion hosted on Unfuddle.com. Communication about changes occurs on every update, thanks to Unfuddle’s notification system. The shared development server gets automatically updated from the trunk on every svn commit via svn commit hooks. Then to move the code around I have one click ANT tasks that handle updating development from Subversion, updating staging from development, updating production from stage, and a unified task that can does all of the updating in sequence (subversion to dev to stage to production in one click with a warning that you should only do this if you are sure about it.) All of this is to accommodate all of the various publishing needs we have. I then wrote ColdFusion that calls the ANT tasks, and an AIR application that calls the ColdFusion. This gives us a one-click publishing tool that we can run from a browser or a desktop application.
We replaced one node of the cluster yesterday, fixed a few bugs, then replaced the other node today – all in all, a very smooth upgrade. I’m extremely happy. It’s a lot to accomplish in 3 months. Mostly, after years of working on very backend systems which never get touched by users, it’s extremely gratifying to work on something that I can show off.
I got an opportunity to volunteer for the Obama campaign and donated some custom web application building to the effort. In case you’re turned off by politics, this post isn’t going to be about politics, it’s more about the environment and technical challenges that I experienced. Finally, I want to make it clear that I am only claiming a teeny, tiny part in the effort. The campaign was won by a lot of people working a lot harder for a lot longer, they deserve a lot of respect, even if you don’t agree with them.
About three weeks before Election Day a call for volunteers came my way from a co-worker. The Voter Protection division of the Obama campaign in Pennsylvania needed someone who had experience working with databases.
After talking with them for awhile we distilled down their problems. They had about 6000 volunteer lawyers willing to work 9000 polling places to protect voters from various threats (some of it malfeasance, but more normally a failure of someone to grasp the full set of election law as it pertains to a particular voter.) They were making these assignments by combining information from a central web application with information collected in the field. They were using copies of an Excel spreadsheet they made every night for in-the-field collection. The guy I was working for had to make that spreadsheet every night, and it was obvious that he needed a better way, but he couldn’t just drop what he was doing and knuckle down and do it.
I, having no other responsibilities to the campaign, could.
In the end, from an application standpoint it was pretty basic: 2 tables, 1 linking table for the many to many relationship (multiple volunteers could be at multiple poling locations.) I pointed Squidhead in fkcrazy (foreign key crazy) mode and it did all the work for me. I had to do some custom tweaking of the app, but I tried to do as little of that as possible because I knew that there would be schema changes. I just had that sense. Then over the next few days I responded to various and sundry schema changes (I told me so).
It was a great experience, mostly because as a small database driven application with many schema changes and not a lot of custom interface work, it was the perfect use case for Squidhead. It reminded me just how useful code generation can be.
I finished up work for it in time for the last two weeks of the campaign. The local volunteer staff would use my application to make assignments. Then before Election Day they would upload their work to the central campaign to integrate with the incident tracking system they had.
Over the next few days, I didn’t think about it much, I the usage stats, and got a note from my host which noticed the spike in traffic (but coolly forgave the overage – YoHost Rocks!). All in all I just watched the final days of election coverage, and hoped all was well.
Around 11:30 on Election night I got a text message inviting me to the local victory party. I figured I should go, because with all of the hype, emotion, and passion of this past election season it seemed like an awesome opportunity. I got there; met up with the guy I worked for; and got introduced around. It went something like this:
Hey, I’d like you to meet Terry Ryan.
Hi, Terry. (Polite, but unexcited)
He’s the guy who ran the Numtopia application
Followed by me getting hugged by a total stranger.
I, uncomfortable, dismissed some of the praise because the app was pretty ugly, only to get some very informed feedback:
I have never been quite so thoroughly thanked and appreciated for my work ever. It was an awesome feeling. I am so gratified that I could use my skills to do something for the cause.
Now, I saw the “application for voter protection from Chicago” and I have to say it was very beautiful, with a whole lot of cool features.
I don’t want to knock those guys, as dealing with thousands of records on the State level and hundreds of thousands of records on the National level are two different things. Additionally, dealing with the demands of 1 office as opposed to 50 also changes the game. But I will say that in 3 days, 1 volunteer developer using ColdFusion replaced an application built over weeks by a team of paid developers using PHP. (I don’t know the specifics.)
I took a few lessons from my experience:
The constraint of a drop-dead-deadline can be incredibly freeing. I didn’t have time to think about things trying to come up with elegant solutions. Some of my solutions were using query of queries in a way I would never recommend. But I needed to get it done because November 4th was unavoidable.
Compelling User Interfaces aren’t always the answer. Squidhead creates usable and accessible but ugly UI components from the get go. They’re meant to be styled by custom CSS. That they were usable was the users’ only concern. They didn’t care that it wouldn’t win a design award. Their major concerns were speed and predictability.
Squidhead is better than I think it is. I’ve been down on Squidhead for awhile as it doesn’t have the user base that Transfer, Reactor, or Illudium has. Nathan Mische been telling me that the stuff he wants to add will help get the word out, and I think he’s right. However, it did stuff that I didn’t originally think it could do, or thought it would be hard to make it do. It worked flawlessly. I just need to get the word about how powerful the foreign key introspection is.
Working for the campaign was an awesome opportunity. It was pretty fulfilling compared to other volunteer political work I’ve done. But what’s more, it has pushed me to do a bit more, and have a little more confidence about the work I can produce. Expect to hear some new features coming out about Squidhead in the next few weeks.
tpryan.blog 