Four steps for selecting your development technologies

Author: Justin James

One my friends is thinking about starting a programming project, and he asked me "how do I pick my technologies?" with an eye towards performance as well as getting the project done. This person has been writing code on and off for a while, but he hasn't worked on any substantial projects. He isn't wedded to any particular technology, and he has the luxury of a clean slate for this project. While he does have a preference for deploying to Linux due to his experience with it, this is not a final decision.

In this column, I share the advice I gave him. I am not going to recommend any particular technologies, but rather show you my approach for making technology and architectural decisions.

Step 1: Loosely design your application

Agile methodologies have become very popular, and some folks believe that means you don't try to think beyond a two week timeline. But there is a lot of value in preparing a loose design of your project. All you need to do is use a tool that has flowchart capabilities (such as Visio) to provide a high level overview of the logic. You do not need to get bogged down in details like "validate that this field contains at least five characters" or data layouts. But you do need a general idea of what parts of the application will be handling what responsibilities.

Some examples of things that should appear in this diagram include:

• Any major batch processing tasks.
• Where data is stored (files, databases, "the cloud," etc.) and which components retrieve it and expose it to the rest of the application.
• Where significant processing occurs (in the database, in a business logic layer, a Web service, the client, and so on) and what it does.

Step 2: Identify resource usage and characteristics

Once your diagram is complete, we can use it to identify resource usage. The resources that you want to be aware of are CPU, RAM, drive space, and bandwidth. Are you transferring a large amount of data to or from an external Web service? That is going to be a bandwidth resource on the connection between your application and that service. Perhaps you are doing an intense calculation within your database — that would be CPU use within the database. And so on.

Step 3: Determine performance critical areas

Once you know where the resources will be used, you have found your potential performance bottlenecks. These are the places where your technology choices will have the biggest impact. If it turns out that you are storing very little in a database, you have more options for the database. Perhaps you will be performing CPU heavy algorithms in the business logic layer, which points out that you will need a language and platform that supports high-speed calculations. This is a chart I've made which will help you see how this affects your decision making.

	CPU	RAM	Disk space	Bandwidth
Database	DB must be a high performance system like PostgreSQL or Oracle.	All major databases should be able to work with lots of RAM.	Skip the low-end DBs with size limits, like SQL Server Express.	Locate your application in the same server room as the database.
Business layer	Language must be fast, and may need excellent multithreading support.	N/A	Locate the application in the same server room as the disks.	Locate your application in the same server room as the database.
API that you expose to clients	Language must be fast, and may need excellent multithreading support.	N/A	Reconsider your architecture.	Reconsider your strategy.
Client-side software	Language must be fast, and may need excellent multithreading support.	Carefully consider your target market and their client capabilities.	Carefully consider your target market and their client capabilities.	Ensure that target market has the bandwidth (don't sell to consumers in rural areas, for example).
Third-party service	Pick vendor carefully.	N/A	Choose a vendor with low cost storage.	Reconsider your strategy.

Step 4: Scale your needs

Another thing that you can learn from your diagram is where your application needs to scale. If the bulk of your processing needs occur in the client piece of the application, your server architecture can be much more modest, for example.

You will also be able to see what kind of scaling you need. Most databases have clustering capabilities, so if you have a choice, it is often easier and better to push things that need to scale (especially if they require a shared state between requests) into the database where scaling is already handled, or to consider technologies higher up in the stack that also have clustering or scaling built in.

Conclusion

By starting your development process with a lightweight sketch of the application's logic, you will be on the right path to select the best technologies for your needs. There are lots of non-technical considerations (such as your budget, experience in particular technologies, and so on), but you need to start somewhere, and this decision making process will help you narrow down your choices and highlight any problem areas before they come up.

I'd love to hear from you in the comments section below to get your experiences with these kinds of issues.

J.Ja

Fixing a hang while opening ASPX files in Visual Studio

Author: Justin James

The strangest bug I've seen all year was with Visual Studio, a program that I ordinarily find to be quite stable and reliable, at least in versions 2008 and 2010. The problem that I faced was that, every time I opened an ASPX file from a particular solution, Visual Studio would hang and give the dreaded "Visual Studio is busy" error message. Usually when you see this error, it is a one-time event, and it really means "Visual Studio is dead." In this case, though, if I left it alone for 5 to 90 minutes, the program would suddenly come back to life.

Digging deeper into the problem, I discovered all sorts of clues, but nothing that lead to a solution. These are some of the things that were happening:

• It would hang when adding an empty ASPX to the solution.
• It would hang if you opened an ASPX in that solution directly from Windows Explorer.
• It would hang if I copied the solution's contents outside of my TFS working directories.
• It did not hang with any other projects.
• It hung on Visual Studio 2010 on another person's machine, but not on a third developer's machine (the one who had originally been working with the project).
• When it was hung, CPU usage was 0%, RAM usage was not changing, and there was no network traffic occurring, so it did not look like an ordinary deadlock.
• The problem was random; sometimes you could open an ASPX file just fine, and other times the hang would occur.
• The problem would not occur if I copied one of the "problem ASPX" files outside of the solution's file structure and opened it.

I ruled out TFS issues, problems specifically with a reference to a particular Web Part, a library, or the Master Page, or issues specific to those ASPX files, since the problem would come up even with a blank ASPX file.

After a few days of being completely paralyzed by this problem (the project had been handed off to me, and we had a ton of things for me to do), I eventually opened a ticket with Microsoft. It pains me to do this because I'm admitting that I can't solve the problem. In this case, there were so many different reasons to see this message, and none of them quite fit my situation, that it was simply quicker to engage Microsoft. And given what my time is worth to my employer and what a Microsoft ticket costs, the ticket is usually a bargain compared to wasting my time.

The Microsoft technician I worked with was quickly stumped. We went through all of the scenarios where it would occur and where it wouldn't. Eventually, he worked with me to generate a dump file. The very next day, he found the problem!

Visual Studio's Design mode requires that the components on a page be initialized so that they can be rendered properly. Some of the components on some of the pages were trying to access a database that was not available from my machine (or the other machine with the problems) during their initialization. When these components underwent initialization, Visual Studio would hang trying to connect to the database, and it would hang for a very long time instead of having a timeout around that initialization. In addition, Visual Studio will pre-initialize components, and that is why it was hanging even when working with an empty ASPX file.

Lesson learned

I am passing on this information in case any TechRepublic readers encounter this bug. If opening ASPX files hangs Visual Studio (versions 2008 or 2010), look carefully at the initialization code of the components in your project because this could very well be the culprit.

I hope this helps someone else before they lose days of work to this issue.

J.Ja

Five tips for solving software problems

Date: October 18th, 2010
Author: Chip Camden

When you're trying to troubleshoot a problem with your software — especially if the client isn't being at all helpful — these tactics will help you zero in on what's going wrong.

---

Do you ever have trouble getting information out of your clients? Even when they could give you that information with just a little effort?

"It's broken," the client said.

"Broken?" I responded. "In what way?"

"It's not doing what it's supposed to do."

"Could you describe for me what it's supposed to do that it isn't?"

"You know — it compiles fine, but when you run it, it just dies."

"Dies how?"

"It gets an error and quits."

"What's the error message?"

"I didn't write it down."

Naturally, the problem doesn't reproduce on my test system. So I have to keep the client involved at least enough to give me access to their system. The client, however, doesn't want to be involved. They just want it fixed. Without saying so, they're probably thinking "Doesn't this software quack ever test this stuff? We're paying him the big bucks just to have this blow up in our users' faces and then paying more to have him fix it!" They're not feeling at all like helping me, but I really need some information if I'm going to help them, because when I try it on their system it doesn't reproduce there, either.

Most of my clients are software developers, and it puzzles me how frequently they plop huge haystacks of code in my lap and ask me to find the needle. I often try to teach them good problem-solving techniques — you know, teach a man to fish — but I'm amazed at the resistance I sometimes encounter. Oh well, more billable hours for Yours Truly.

Here are some general problem-solving techniques I use. Many of these apply to all sorts of problems, not just software bugs.

Note: These tips are based on an entry in our IT Consultant blog.

1: Make it smaller

Distill the problem down to the minimum amount of code required to reproduce it. Eliminate anything extraneous. Why go to all that trouble? One of the easiest ways to find the needle in the haystack is to get rid of most of the haystack. Sometimes, the problem goes away when you cut something out, which should give you an idea where it's hiding. Besides, if you get into an iterative debugging cycle, you'll be able to cycle much faster with less code and fewer steps.

2: Question your assumptions

I had a client call me the other day to report a problem in which static data was supposedly being modified by a return statement. Ah, I know what you're thinking. But there were no objects involved in this code, so no destructors were being called. To believe him would mean that there was a horrible bug in the runtime environment for the language he was using, which didn't seem at all likely to me. So I asked him how he examined the data before and after the return. The data was stored in a library module for which he did not have debug symbols, so he added calls in his code to a standard routine to query the data. But he didn't realize that this routine (in the way it was called) also had the unfortunate side effect of modifying the data, creating an instant Heisenbug. He had been moving these calls around, trying to isolate the statement that changed the data, when the debugging statement itself was doing just as much damage.

3: Beware of false causation

How many times have you heard, "The only thing that changed is X, so the problem must be related to X." No, no, no, no, no, no, NO! More than half the time, something else changed that they forgot about — or didn't even know about. When the "obvious" cause turns out to be a red herring, or even before, I always echo the famous words of Sgt. Schulz: "I know nothing… NOTHING!" Okay, so it's intended more in the spirit of Socrates. But no cause should be assumed until proven. That doesn't mean you shouldn't check out your hunches first, though. We have intuition for a reason.

4: Start at the result and work backwards

I often see programmers start a debug session and then step through routine after routine, examining variables along the way, hoping to stumble across the moment when things go wrong. Usually that doesn't work at all, because problems have a knack for resulting from seemingly innocuous beginnings. It may seem counterproductive because code doesn't execute backwards, but it's more efficient to start at the moment of the failure's epiphany (an error message, for instance) and examine what's wrong at that instant. Then go back to the code that led up to that point to see where it went wrong, backing up routine by routine until you find the culprit. Ideally, debuggers should be made able to step backwards, but even if you have to restart your debug session a hundred times, you'll save time over trying to perceive the cause from the top.

5: Refactor

Sometimes, the complexity of the situation is part of the problem. Maybe the definition of what the "plugh" function does is not entirely consistent, and that's what's leading to a failure. By simplifying and clarifying the design, those inconsistencies often reveal themselves. But use this judiciously — once you get started down that road, you might not be able to stop for a long, long time.