Re-associate Microsoft SQL Login with a Database User

I’ve had to search for this several times in the last few weeks, (ever since Mark Drew pulled me into the dark side that is local development.) I figured if I blog it, I’ll always have the answer at hand.

If you are restoring a Microsoft SQL database from a backup onto a new server, and you need to re-associate the server login name to the database username, you need to call the “sp_change_users_login” stored procedure. That will allow you to reconnect them.

ColdFusion 8 Monitoring Heisenberg Errors

I ran into my first inexplicable crash that I eventually traced back to the ColdFusion Server Monitor. Now first off, this isn’t a problem or bug with the Server Monitor. This is to be expected. The server Monitor adds overhead to requests, and if you have an intense process, it’s going to generate a lot of monitoring data. It’s possible that you might reach its limit.

I just wanted to let people know what a crash caused by the monitoring service looks like, because it doesn’t give you a message that “You have left the monitoring service on in production!”

I had a long running complicated process crashing on my local workstation. It did work on our communal development server. So it wasn’t just the process itself. I thought maybe it was that my laptop wasn’t a server class machine. But actually, the virtual machine that we are testing on wasn’t tremendously more powerful.

The browser session would error out with a message that said:

500

Java heap space

java.lang.OutOfMemoryError: Java heap space

After digging in the JRun logs for awhile I found this:

javax.servlet.ServletException: ROOT CAUSE:

java.lang.OutOfMemoryError: Java heap space

at coldfusion.monitor.event.MonitoringServletFilter. doFilter(MonitoringServletFilter.java:70)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:284)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

java.lang.OutOfMemoryError: GC overhead limit exceeded

Of course I didn’t bother actually reading this error until just now when I copied and pasted it. It clearly indicates that the problem is in the Monitoring Servlet Filter. In any case, after much trial and error, I turned off memory tracking and then turned off profiling. Once I turned off profiling the error went away.

Using Microsoft SQL in Ant

I had to do a repetitive database task yet again the other day, and I stumbled onto the fact that Ant has full blown support for SQL. The only challenge is getting it to work with the Microsoft SQL we use here. It’s not exactly self evident, mostly because I’m not a Java programmer, so I figured I would share the information.

First download the Microsoft SQL JDBC driver from Microsoft:

Download SQL Server 2005 JDBC Driver 1.1

Once that’s done, execute it, and place the packaged contents somewhere on your computer.

Then fire up Eclipse. You’ll have to add the JDBC driver to the classpath for ANT:

  • Go to “Window”
  • Choose “Preferences”
  • Expand “Ant”
  • Choose “Runtime”
  • Select “Ant Home Entries” This will cause buttons on right to be enabled
  • Press “Add External Jar”
  • Navigate to where you put the SQL JDBC driver
  • Add sqljdbc.jar
  • Hit Apply and hit “Ok”

Now that we have a SQL Driver on our machine, all that’s left to do is to write the Ant tasks that will use it.


<sql
driver=“com.microsoft.sqlserver.jdbc.SQLServerDriver”

url=“jdbc:sqlserver://${dbserver}:1433”

userid=“${dbusername}”
password=“${dbpassword}”

print=“TRUE”>

SELECT CURRENT_TIMESTAMP

</sql>

Obviously, you’ll want to use your application’s database password to do that. Also, I included the SQL directly, but you can also call a .sql file with statements in it.

There are a few gotcha’s:

  • “Go” intermittently causes issues. Use a semicolon to separate statements
  • Use [database name] doesn’t seem to work
  • Putting Database passwords in an .xml file will expose them.
    • Rename build.xml to build.ant.
    • Make sure you don’t expose .ant files to your webserver.

What can you do with this? Well I have a couple of ideas:

  • Delete rows created by tests.
  • Run Consistency checkers
  • In conjunction with an export script, synchronize schemas and stored procedures between two databases.

You may also want to look at the documentation for the SQL Ant Task.

Unplanned Outage

You may or may not have noticed a little unplanned downtime. The webserver that Numtopia.com is on died last Thursday. My hosting provider got ColdFusion up by Saturday. But I don’t run BlogCFC like all the other cool kids, I run Movable Type which needs Perl and a special MySQL Perl driver. Getting my hosting provider to re-install it (it was there pre-crash) was an exercise in patience. It took almost a full week for them to do so.

Now I’ve been oscillating between impatience (I want this up) and understanding (this isn’t a business, and I’ve had to deal with people with custom configs post crash.) I’ve decided that for ColdFusion hosting, for $12 a month; 1 week of downtime for my ability to post to my blog over the course of 2 years, is probably acceptable.

ColdFusion 8 Makes It Easier

ColdFusion 8 is now out as a public release candidate, and do I have a lot to say.

First off, I think the ColdFusion team did a great job at adding features across the board that would make ColdFusion a much more competitive product. But I think more interesting is looking at what the ColdFusion team is making easier for you in this release.

ColdFusion has, in my opinion, always been about taking the operations that everyone has to do to build web applications and make them easier. Early versions were focused on making talking to databases easier. ColdFusion MX 6 was about porting the codebase over to Java. You could argue that it was about making ColdFusion easier to work cross platform. That’s a stretch, so if you prefer we can say it was a rebuilding release. ColdFusion MX 7 was about making interfacing easier. Between <cfreport> and <cfdocument> ColdFusion gained new ways of output data. Flash Forms was intended to make writing form based interfaces easier. Gateways make ColdFusion easier to interact with in non-traditional, non-browser based ways.

So what does this version of ColdFusion make easier? I’ve been searching for an answer to that for a few hours now and I think I have a few answers.

First, from the original Scorpio Labs Page:

  1. ColdFusion 8 will allow ColdFusion developers to create richer, more engaging web applications
  2. ColdFusion 8 will provide powerful new insight into server performance and simplify application troubleshooting
  3. ColdFusion 8 will give developers broader platform support and enterprise integration

Here’s my translation:

  1. ColdFusion 8 makes Web 2.0 Easier
  2. ColdFusion 8 makes Administration Easier
  3. ColdFusion 8 makes selling ColdFusion to the Enterprise easier

ColdFusion 8 makes Web 2.0 easier

Cheesy, unliked, overused and unfortunately for me, necessary, the Web 2.0 moniker is unfortunately here to stay. It’s about many things. But I’ve always liked it, because I’ve always looked at it not from a technology perspective, (It’s Ajax, or Flex) but from an authority focus. Web 2.0 is about user content, not “authority” based content. It’s about users choosing how to consume their applications, not site owners dictating how to consume it.

One could make the argument, that there are only two types of typical, Web 2.0 user data that ColdFusion has to date, not dealt well with. They are Video and Images. Video is still shaking out, and still isn’t addressed by ColdFusion. (Adobe as a whole has direction on this.) But at least on the image side of things, ColdFusion 8 makes it easier to handle with <cfimage> and the suite of image functions.

It has always been easy to do Flex development against ColdFusion, but now <cfajaxproxy> and the JSON functions are making it easier to do Ajax development against ColdFusion. This further separates the choice of ColdFusion from the front end used to access it. This is a good thing, it expands a bit on what they did with 7, but also allows “ColdFusion and Ajax” to be said a lot together in press releases and technical news articles. This is good for us as a community.

ColdFusion 8 does not abandon the interface side of things though. There are a bunch of new display management tags including <cflayoutarea>, <cfpod>, <cfmenu>, <cftooltip>, <cflayout>, <cfmenuitem>, <cfwindow>. This shows that ColdFusion is still trying to make it easier to do things that everyone has to do. This time, they are clearly going after the interactivity of “Ajaxy” sites. Now, in the past, in ColdFusion, these types of interfaces have not always been usable on a forward facing application. This is no longer the case. Not only do they look completely modern and professional by themselves. But if you don’t like them, they can be styled by CSS.

Lastly on the Web 2.0 side of things, there is a <cffeed> tag. This makes both creating and consuming RSS or Atom XML feeds much, much easier. Not much to say about it, this is something that all of us have to do now, ColdFusion makes it easier.

ColdFusion 8 makes Administration Easier

The new ColdFusion Monitor tool is perhaps the best improvement to my job ColdFusion has ever made. The ability to track down an errant template is simply stunning. The ability to kill a running template is beyond stunning, it’s ginormous. You can actually see the stack trace of a running template if it’s causing a problem. You can even see every variable in every conceivable scope in a request. This is truly stunning.

Even though we central administrators love exercising iron-fisted control over our shared servers, it’s really nice for Adobe to open up limited capabilities of the administrator to distributed users. I do wish they had made it easier to tie it into a third party directory tool like Active Directory, but I’ll take what I can get.

Finally the administrator now has more options for tweaking performance. You can set limits on number of simultaneous page requests, CFC calls, Flash Remoting calls, and web service requests. All of these tweaks do make it easier for administrators to control their environments.

ColdFusion 8 makes Selling as an Enterprise Solution easier.

I’ve never been one to demand Java like features in ColdFusion. However there are a few areas, where it would be nice if more Java like behaviors were exposed. One case I’m talking about is file processing. ColdFusion 8 now lets you parse files line by line, instead of forcing you to load an entire file into memory at once. But that doesn’t even mention the addition of native Zip tools, which now exist. Granted you could have always dropped down into Java to handle this stuff, but the more you force that to be a solution for people, the more entry-level programmers that run into walls.

But, ColdFusion 8 doesn’t just play with Java better. Adobe’s been hanging out in Redmond. First ColdFusion 8 can call .Net objects like it can Java classes or CFC’s. I think a little cooler, is the fact that ColdFusion 8 can talk with Exchange. Granted, I’m a little sad to be out of an expertise, but happy that everyone can now play with Microsoft’s groupware offering. I’ll probably be saying even more on this feature in the upcoming weeks. But make no mistake about it, this is hands down the easiest way interact with Exchange other than Outlook. It’s really, seriously, that good.

There’s a lot of touting of the performance enhancements to ColdFusion 8. But in addition to just running faster, ColdFusion 8 allows developers to have more control over the performance of their applications through the addition of <cfthread>. This technology will allow developers optimize sequential behaviors like emailing, or rebuilding static files by running them asynchronously. Up until now, the only clean option for asynchronous operations was the Asynchronous CFML Gateway. It does the job, but is a little unwieldy to use if you aren’t an administrator on the machine. <Cfthread> doesn’t replace the Asynchronous CFML Gateway, it just allows another option in the continuum of asynchronous operations.

All in all, developers can now make faster performing applications, on a faster platform, which can talk to many more systems now.

Conclusion

This is just the tip of the iceberg. Every one of these one sentence blurbs represents a feature that deserves its own blog post. (They also probably represent years of developer’s time.) Additionally there are a ton of features, functions and tags I didn’t mention. This is by far the best release of ColdFusion yet.

Squidhead and CFUnit

I published a change to Squidhead earlier today that included an option for Squidhead to write its own unit tests today. These unit tests are compatible with and require CFUnit. Here’s a little bit about the reasons behind this.

I originally started fooling around with unit testing because after drinking Pragmatic Programmer Kool-Aid, and attending cf.Objective, it was the next thing on my list. I figured I would be pulling them into Squidhead, as it’s pretty easy to tell if Squidhead worked (the application is either written or not,) but it’s hard to tell if the created application is working. So I started to write unit tests for my DAO Objects figuring it would be an easy place to start. Once I got one working, I realized it was relatively rule based and could stand to be generated itself.

What it does is:

  • Create a mock struct of the record.
  • Create a mock struct of the record slightly different from the first.
  • Create a record from the mock record.
  • Read it back and compares it to the source.
  • Update it according to the different mock record.
  • Read it back and compares it to the source.
  • Delete the record
  • Read again to make sure the item is gone.

All that is wrapped in a <cftransaction> making sure the thing never happened. Additionally it reads in and honors foreign key constraints. It’s got some weaknesses: it doesn’t like binaries or images; its smalldatetime check is only accurate to the hour; and there are probably more bugs lurking.

Now once I got them working, they were in, but what’s the point for users of the application? One should expect that the parts work.

So to make this worthwhile I added some logic that will run any test that is in the test directory structure. So if you’re using Squidhead and looking to dip your feet in the waters of CFunit or unit testing in general here’s your chance.

If you’re wondering why CFUnit, and not CFCUnit, it’s because as far as I can tell CFCUnit requires Mach II, and I figured one requirement was better than two.

So if anyone has any suggestion on how to test gateways or dao’s I’d be curious. Otherwise check it out.

svnAuth.cfc

I figured I’d drop a line about a RIAForge project I just posted tonight.

It’s pretty basic, but I figured there might be someone out there who would need the same thing. SvnAuth.cfc allows for programmatic interaction with Subversion authz files from ColdFusion. I know this might not be cool with the philosophy behind SVN. But I’m not really looking to prevent people from updating other people’s projects, but rather preventing sensitive information (like datasource passwords) from being seen by people other than the developers.

Anyway, if you’re in the small market for programmatic control of an SVN auth file from ColdFusion , check out svnAuth.cfc.

Philly CFUG and Scorpio

Adam Lehman and Tim Buntel graced us with their presence at the Philly CFUG meeting tonight.

A couple things I really liked:

  • Tim directly addressed the fear of the CF community that ColdFusion doesn’t have a future. Basically “Look at everything we’re positioning for the future! ColdFusion is the technology that we’re using to drive them all!”
  • Adam has really refined the examples he uses to explain each feature.
  • Adam also really did a great job of answering earlier criticisms of <cfexchange> issues. (Not just the ones I leveled at him.)
  • In general, I’m really impressed by the ability the CF road team shows to alter their presentation in response to the community. They’re not just reading the same presentation to you. Well they are reading from the same presentation, but they’ve definitely changed the talk based on issues brought up by their audience.

figured I would take down the entire list of features discussed to date. Why? So I don’t accidently mention a hypothetical to-date-unnamed feature, after the jump:

  • Ajax
    • <cfajaxproxy>
    • Json Functions
    • Ajax Wizard
    • Ajax UI Components
      • Yahoo UI Library
      • <cfgrid>
      • <cftree>
      • <cfselect>
        • Dual Select Lists
        • Auto Suggest Combo Boxes
      • Calendar and date pickers
      • Rich TextAreas
      • <cfwindow>
      • <cflayout>
  • Image Manipulation
  • Improved File IO
  • Thread Manipulation
  • <cfdocument> Improvement
  • Reporting Enhancements
  • CFC enhancements
    • <cfinterface>
    • onMissingMethod
  • Database Enhancements
    • Database driver level logging
    • Nested <cftransaction>
    • <cfdbinfo>
    • <cfstoredproc> can be cached
    • <cfqueryparam> can be cached
  • Debugging
    • Eclipse based debugger
  • Argument Collections for various tags
  • Implicit array creation
  • Implicit structure creation
  • Javascript Operators
  • SFTP support
  • FIPS Compliancy
  • CFC Serialization
  • .Net consumption
  • <cfexchange>
  • On Demand Presenations
    • <cfpresentation>
    • <cfpresentationslide>
    • <cfpresenter>
  • <cffeed>
    • RSS
    • Atom
  • PDF Manipulation
    • <cfpdf>
    • <pdfform>
      • <cfpdfformparam>
      • <cfdpfsubform>
  • Server Monitoring
    • Monitoring and Profile have a minimal effect on server resources
    • Memory tracking currently causes 18% processor utilization overhead.
    • Customized Alert actions based on CFC
  • Per Application Setting
    • Application Mapping
    • Custom Tag Paths
  • Platform Support
    • Vista
    • Apple Mac OS X on Intel
    • 64-bit Support (Solaris)
    • JDK 1.5 & 1.6
    • JBoss 5.x
    • VMware
    • MS Virtual Server
  • ColdFusion 8 is 5 to 8 times faster on object creation

Squidhead – 3 Big Additions

Over the last few days, between the inspiration I got from cf.Objective(), and some positive internal feedback regarding Squidhead, I was inspired to make some cool additions (Or what I consider cool editions.) I figured I would share.

Business Layer

Dave Konopka advocated for a layer between the DAO and UI for awhile, as I explain in my last Squidhead update. Well he finally got it. Basically it’s the place where you want to put all of your logic for dealing with the objects as they will be interacted with by the UI.

XML Structure Cache

I wanted to do a few things to overcome the fact that the database is does not contain enough metadata to handle certain things. But I didn’t want people to have to start with XML to start working with Squidhead. Solution: an XML version of the variable structure that Squidhead uses to generate all of the procedures, functions, cfc’s, custom tags, and cfml pages for an application.

So what happens is that at some point you freeze changes on the database, then you set Squidhead to use the disk cache instead of the database. It will then allow you to change a couple of things in how the application works.

Currently you can only change three things (and have the change do something useful):

  • DisplayName on database columns that will show as labels on all UI components instead of the name. (“First Name” vs f_name).
  • OrderBy on tables which will alter the order of the default List elements
  • ForeignKeyLabel the column which will be substituted for the primary key, in all tables that link to this table via foreign keys.

Which leads me to…

Foreign Key Support

Squidhead now understands foreign keys. Here’s how it works:

Form UI – On forms that interact with tables with foreign keys, Squidhead will produce a select box of the aforementioned “foreign key labels.”

List and Read UI- These components now call separate stored procedures that properly join the requisite tables to show the appropriate “foreign key label” in lieu of the actual value.

I haven’t done anything with children objects or anything yet. Basically the change was to make the UI a little more useful.

My next pass of updates will be documentation related, as I’ve fallen behind.

Anyway give a try when you get a chance. Get it at RiaForge.