Here we go:

App Sketchbook by Square Position, LLC (USA), @appsketchbook on Twitter, sold via PayPal for USD 12.99, in both a perforated and non-perforated version.

iPhone Application Sketch Book originally by Kapsoft (USA) but recently sold to Apress Publishing, sold on Amazon for USD 9.99 (while supplies last!).

Notepod by Inventive Labs (Australia), @notepods on Twitter, sold via PayPal for USD 17.95.

The Developer Sketchbook for iPhone Apps by Electric Butterfly (USA), @ebutterfly on Twitter, sold on Amazon for USD 19.99.

Finally, the UI Stencils Sketch Pad (for USD 7.95) with its iPhone Stencil Kit (for USD 17.95) by Design Commission (@designcom on Twitter).

I’ve ordered some of these items, and I’ve seen some others in the hands of some colleagues, and I can tell you that they are really handy and useful. I look forward to using them soon! For more information, here’s a recent article on TUAW comparing some of the items featured in this article.

Happy iPhone app designing!

Update, 2009-11-12: The iPhone Application Sketch Book (recently bought by Apress) has been revamped and now features a plastic stencil too!

He actually didn’t know you could remove all compiler warnings; he thought Objective-C was the land of compiler warnings. This situation, I think, is far from exceptional, and due mostly to cultural and technical reasons.

It is my opinion, that removing compiler warnings is basic project hygiene, like writing unit tests, or using the Clang Static Analyzer. I will explain in this post some techniques I use to remove warnings in my Objective-C code.

First of all, why does the Objective-C compiler (or compilers in general) output “warnings”? Many developers are puzzled the first time they encounter them, since even if the compiler complained, the application usually runs anyway without (perceptible) problems.

Warnings are used to signal specific issues in the source code which could potentially lead to crashes or misbehavior under some circumstances, but which should not (pay attention to the verb “should”) block the normal compilation and (hopefully) execution of your code (otherwise, it would be a compiler error).

It’s the way used by your compiler to say:

Hey, I’m not sure, but there’s something fishy in here.

Not removing warnings, as I said above, is a problem that originates both in the programming background of the developer, and specific technical issues.

Culturally speaking, many other programming environments either do not have compilers at all (at least not “visible” ones, like Ruby or PHP) or simply do not spit warnings for anything else than deprecated methods (like C# or Java); this situation has made many developers new to the iPhone platform to blatantly ignore them.

Technically, given the fact that Objective-C is the “other” object-oriented superset of C, and that it behaves as a coin with both a static and a dynamic side, compiler warnings convey a great amount of precious information that must never be ignored.

In this sense, Objective-C has a lot in common with C++. Ignoring warnings in C++ is strongly discouraged, and Scott Meyers explains this in chapter 9 of his book “Effective C++”, stating that (third edition, page 263):

Take compiler warnings seriously, and strive to compile warning-free at the maximum warning level supported by your compilers

In the case of Objective-C, this can be done by setting GCC_TREAT_WARNINGS_AS_ERRORS (-Werror) to true in your build settings.

Steve McConnell takes this advice to another level of importance in his classic book “Code Complete” (second edition, page 557):

Set your compiler’s warning level to the highest, pickiest level possible, and fix the errors it reports. It’s sloppy to ignore compiler errors. It’s even sloppier to turn off the warnings so that you can’t even see them. Children sometimes think that if they close their eyes and can’t see you, they’ve made you go away (…). Assume that the people who wrote the compiler know a great deal more about your language than you do. If they’re warning you about something, it usually means you have an opportunity to learn something new about your language.

To give a concrete example of the importance of warnings, many of us have had to migrate applications developed for iPhone OS 2.x to the 3.0 operating system, mostly because failure to run on the new version of the OS was ground for removal from the App Store. That moment of truth, the rebuild of the Xcode project, unveiled a plethora of compiler warnings, most due to deprecated methods, like the tableView:accessoryTypeForRowWithIndexPath: method of the UITableViewDelegate protocol, or the initWithFrame:reuseIdentifier: method of the UITableViewCell class (which, incidentally, are properly marked as such in the documentation, too).

Compiler warnings in Objective-C have a multitude of reasons:

• Using deprecated symbols;
• Calling method names not declared in included headers;
• Calling methods belonging to implicit protocols;
• Using some ambiguous commands which might be intentional but are syntactically valid anyway;
• Forgetting to return a result in methods not returning “void”;
• Forgetting to #import the header file of a class declared as a forward “@class”;
• Downcasting values and pointers implicitly.

Many solutions exist for these problems, and I do not claim to know them all; I’ll just describe some of them, and hopefully some of the readers of this post will add others in the comments below.

1) Make implicit protocols explicit

This is a really simple one, and it’s also good for documentation and code clarity reasons. Get all the references of delegate methods you use in the code, and group the methods into their own header file. Import the header file whenever you need, and make classes explicitly implement them:

[source:c] //… @interface NewClass : NSObject { //… [/source]

Of course, take advantage of Objective-C’s @required and @optional keywords in your protocol declarations; they are used by the compiler to verify (or not) the existence of delegate methods in your class implementation. This way, you’ll surely remove some warnings.

2) Do not use “id” as the data type for delegate fields Using id as datatype for delegate fields

I personally use the following declaration for delegate fields:

[source:c] //… NSObject *delegate; //… [/source]

instead of simply

[source:c] //… id delegate; //… [/source]

This is because I always check on delegates before calling their methods. Call me paranoid, but this is what a good delegate call looks to me:

[source:c] if([delegate respondsToSelector:@selector(someObj:doesThis:)]) { [delegate someObj:self doesThis:@"123"]; } [/source]

Using NSObject instead of id in the delegate declaration avoid yet another warning. Update, 2009-07-17: This is because using id raises a warning that “respondsToSelector:” is not defined in SomeProtocol. The solution for this is making SomeProtocol inherit from the NSObject protocol (I always forget this double life of the NSObject symbol):

[source:c] @protocol SomeProtocol //… @end [/source]

This way, you can use id<SomeProtocol> variables without problem.

3) Create categories for private methods

Objective-C uses the @private, @public and @protected identifiers only for instance fields, but otherwise methods can only be marked as instance (“-”) or static (“+”), but all methods specified in the header file are public by default.

If you need to specify private methods, do that in your implementation file, creating what’s called a “category” of your own class, which basically “extends” the class with new methods:

[source:c]

import “NewClass.h”

@implementation NewClass (Private) // your methods here @end

@implementation NewClass // … @end [/source]

This way you can define private methods, not exposing them in the public interface file. And you remove some more warnings.

4) Turn implicit type conversions and casts into explicit ones:

This one is inspired by McConnell’s Code Complete (second edition, page 293):

[source:c] int i; float y, x; y = x + (float)i [/source]

Even if the compiler could work out the “y = x + i” expression without problem, the code above will remove yet another warning, and will make your code more obvious and easier to read, since it clearly states your intentions.

5.1) Support earlier OS versions via runtime checks

If you have to write iPhone applications compatible with both the 3.0 and 2.x versions, this sample from Apple https://developer.apple.com/iphone/library/samplecode/MailComposer/listing7.html provides instructions on how to do it.

“MailComposer runs on earlier and later releases of the iPhone OS and uses new APIs introduced in iPhone SDK 3.0. See below for steps that describe how to target earlier 0S versions while building with newly released APIs.”

It is worth noting that this sample application compiles without a single warning!

5.2) Support earlier OS versions via #defines

Use #ifdef IPHONE_OS_3.0 and IPHONE_OS_2.2.1 if you can (or must) provide different binaries for each supported platform. This might be the case for in-house applications, but again, it might help removing some warnings too.

6) Use @class in the @interface, #import on the @implementation

Whenever you use a class on a header file, to avoid cross-references, use the @class keyword to reference it, but do not forget to #import its header file in the implementation!

[source:c] @class AnotherClass

@interface SomeClass : NSObject { @private AnotherClass *field; //… } [/source]

and then

[source:c]

import “AnotherClass.h”

@implementation SomeClass

• (id)init { if (self = [super init]) { field = [[AnotherClass alloc] init]; } return self; }

@end [/source]

The problem is, if you do not #import the file, you get a warning…

The advantage of this technique is not obvious in small projects, but it is strongly recommended anyway; it prevents header files from cross-referencing each other, and it reduces build times in projects anyway. It is the Objective-C analog to the technique of using “class” statements in C++ header files, instead of #include.

For more information: Matt Gallagher also wrote about this issue, and I strongly recommend his article too! Read also this article from Apple about the issue of compiler warnings which covers all the options available on GCC in great detail.

Finally, all of this boils down to the fact that iPhone programming is not as easy as web development, and that iPhone applications, for many reasons, require patience and attention. Removing warnings from your code is just one of many steps to have great iPhone applications, running smoothly without problems.

Update, 2009-07-17: Regarding the “id vs. NSObject” issue, I’ve used NSObject because you get a warning when calling respondsToSelector: on a variable of type id. Now, after reading all the comments I’ve gone back to Xcode and found out that you can define SomeProtocol as implementing the NSObject protocol itself, and this solves the problem. Thanks everyone for the heads-up!

(…) Gore has been a team-based, flat lattice organization that fosters personal initiative. There are no traditional organizational charts, no chains of command, nor predetermined channels of communication. Instead, we communicate directly with each other and are accountable to fellow members of our multi-disciplined teams. We encourage hands-on innovation, involving those closest to a project in decision making. Teams organize around opportunities and leaders emerge.

What you must avoid:

Corporate IT needs the ability to run their own internal infrastructure projects without needing permission from the business.(…) Most people in corporate IT want to do a good job. Most people in corporate IT are capable of doing a good job. Every developer I know is driven crazy by crappy code; they want nothing more than to get in there and make it better, stronger, faster. Most of the time, however, they are actively stymied from doing a good job because they can’t act on their own, best instincts. Sorry pal, but that’s not in the business plan.

What you need:

The Free Electron is the single most productive engineer that you’re ever going to meet. I have not even provided a definition and I’m guessing a person has already popped into your mind that fits the bill. A Free Electron can do anything when it comes to code. They can write a complete application from scratch, learn a language in a weekend, and, most importantly, they can dive into a tremendous pile of spaghetti code, make sense of it, and actually getting it working. You can build an entire businesses around a Free Electron. They’re that good.

Base recipe for quality:

Here’s a theory of software quality for you: software must be nurtured. The existence of bugs isn’t mysterious to any honest programmer. They are the product of neglect. (…) Programmers have complete control over the quality of their code and, when working on code they care about, tend to produce things that work. The secret is to care for the programmers, so that they take good care of the software. Large organizations generally drive their software bees into combat formations, rather than nurturing colonies. Standardization is the mantra. Everyone will use one language, one set of libraries, and three ostentatious software “patterns.” In such forced conditions the feeling of code ownership vanishes and quality goes out with it. (…) The alternative is obvious: break systems down into their smallest practical units, and let small groups of programmers determine implementations from top to bottom. Necessary interfaces between modules are hammered out according to the best available standards; unnecessary interfaces are not built. Coders get to work in their favorite technology, or learn a new one.

The dot-com boom of the 90′s spawned a brand new generation of coders and software developers, including me, by the way. While before that time the term of “software developer” might have been reserved to system programmers fluent in C, COBOL, C++ or other languages, right now the vast majority of developers I know spend their time writing web applications, either public or in a private intranet, in J2EE, ASP.NET, Rails, PHP, you name it.

I have said before that writing web applications should be taken as seriously as writing desktop systems. Call me names if you want, but I’m a big fan of Joel’s Test.

However, after all this years, after the Chaos reports, after Peopleware, after the Mythical Man Month, people still treat quality as an afterthought. And also complain about how much software sucks, how expensive it is, and how late it arrives, by the way. Now that the iPhone SDK is widely available, that the App Store is selling more apps that we could have had imagined 6 months ago, many web agencies want to jump to native iPhone development contracts, which are hype and nice and pricey and whatnot. Which is only going to make things worse.

The dirty little secret in this story is this: iPhone development looks more like developing applications for a desktop operating system, and less, much less than web development. And I’m frightened to see some small shops (and even bigger ones), who never attained a real level of professionalism or quality in their software tasks, starting projects and realizing, later, when they are over budget and behind schedule, that this kind of applications requires a different mindset.

Let’s repeat something that you should know by now: web apps are easy to maintain. To begin with, you just have one instance running of it, and as Paul Graham said, you can write them in any language you want, you can release bug fixes with your application running, monitor performance issues live, change its appearance quite easily and for everyone at once, etc. This is probably the single biggest advantage of web apps. With AJAX we even got the possibility of going beyond in terms of user interface, responsiveness, perceived speed, you name it, and today the number of famous web apps is outstanding.

However, web apps run into this terrific (and terrifying) sandbox called the browser. They (normally) cannot access your file system, they cannot open other applications and interact with them – well, with some custom URL schemes like lastfm: mailto: or skype: you might have the ability to do some things, but certainly not much more than activating the application in some limited way as a help for the user. You cannot access the user’s hardware directly – well, again, you can access the webcam through Flash, or you can trigger the window.print() method to have the native print dialog pop up, but that’s more or less the maximum you can do.

In the iPhone, there is a similar situation. You choose to create a native iPhone application over a web one when you require one or many of these things in your application:

• GPS geographical data;
• Accelerometer information;
• Photo camera or library;
• 3D graphics;
• Complex animations;
• Address Book entries;
• Sound recording and playback;
• etc…!

I’ve talked about the dichotomy between iPhone native vs. web apps in my speech during the iPhone conference where I used this graphic, which might help those having to decide whether they should do a native or a web application:

The tradeoff, basically, consists in knowing that having the ability to access these features, means that you are giving up your capacity to roll out quick upgrades to your code. You have to depend on Apple’s own review process timings for that, which, as far as I’ve seen so far, cannot be predicted. And finally, you depend on the user’s final will to update your application!

Similarly, you also lose the ability to create these applications using your common, well-known, garbage-collected programming language, but you’ll have to deal with Objective-C exclusively, together with its weird syntax, possible memory leaks, eventual out-of-memory notifications, lazy loading techniques, compiler warnings and errors, and seeing the MVC design pattern even in your wildest dreams at night. Regarding the syntax, I admit that I love it, but it took me a while to get used to it, when I started playing with it back in 2003.

Given these conditions, when you start an iPhone project you will have (let me be very clear about this, so I’ll repeat) you will have to dust out your good old desktop application programming techniques (or learn if you don’t have them), read Code Complete again (which I’m doing right now) and take all the advice that you can from C and C++ programmers who have been working in this kind of stuff for the past 20 years.

I’ve even done my Master’s degree project in C++ because of this: the mindset required for iPhone apps is not the same as for your day-to-day web application, and is closer to that of a desktop application. I’ve been doing web apps for 12 years now, and desktop apps for 5 years (mostly in C#, Objective-C and C++); that’s my ground for stating this. You require a spec, you require tests, you require testers, you require daily builds, you require bug databases, you require quiet working conditions.

You require at least an 11 in Joel’s Test to know that you’re doing fine, but not only that: you need to know about pointers, you need to know C, you need to know that your code runs in an fragile environment with EXC_BAD ACCESS (SIGBUS) and low-memory warnings and stuff like that happening when you expect it the least.

Unfortunately, from what I’ve seen so far (at least here in the Lake Geneva region) many companies are spending much more than they intended to for rolling out their iPhone apps; these are real quotes from people I’ve dealt with in the past 6 months:

• “Oh, we do not write specs, we prefer to modify the code as we have ideas!”;
• “Oh, we do not write tests, we do not have the time for that!”;
• (in general) “Oh, we do not work like that here. We do what the client asks and that’s all.”;
• “We will not follow Apple’s iPhone Human Design Guidelines at all; we have our own. Users must double-click on buttons, so they feel more comfortable while using our application”;
• “We want our [PUT YOUR FAVORITE UIKIT CLASS NAME HERE] to have [SOME IMPOSSIBLE FEATURE WHICH DEFIES GRAVITY]. Easy, right?”
• “We have to remove this XYZ feature, right now!”, which means rewriting half of the code and leads to this: “What? So much? For such a small change?”
• “Why have you spent all that time ‘fixing memory leaks’? I won’t pay for that. Please concentrate in the new features we’ve asked. By the way, what are memory leaks?”
• “Here’s the styles document you asked, with the colors and fonts for the application” and the guy provides me with… a CSS stylesheet. Which references a font not available on the iPhone, by the way, and with colors in hex format.
• “We want an animation at startup, like the Chanel application” (everyone wants to release the next Chanel iPhone app these days) “here’s an [animated GIF / Flash movie / PowerPoint slides] with the animation for you.”
• “Our team has slightly modified the code when you weren’t there, but it does not work any more, could you fix it please?”

And of course, the all-time winners: the support tickets stating that “everything is slow” without further indication, or that “the application hangs”, without any details in it.

OK, I confess, I’m a bit of an extremist here; many of the quotes above are valid in some contexts. But not those of the small-to-medium sized iPhone projects I’ve been involved in lately, with the urgency of releasing the code as fast as possible, just for the sake of being there, in the App Store, right now.

There’s a long road ahead. The problem, again, is not the technology itself, but the people involved in these projects (me, for example :). There’s a bit of what Joel mentioned a while ago, about the perils of Java schools, which actually only has to do with developers, but there’s also the issue about teaching the clients the limits of the platform, and about creating a strong software engineering body of knowledge in companies which usually did not need it previously.

We’ve been doing web apps for so long that we’ve forgotten how to sit down, take a deep breath, fix that goddamn memory leak, and realize that, as Brooks and Joel said, good software takes time.

I had just joined this company a couple of days ago, and the management asked me to take the project in charge. Not an easy task, particularly because it was my first experience of this kind. The client was pushing to get the software it had paid for (it was a desktop reporting application for the Police department), and had not got any kind of preview yet. So the first thing I did is to pick up my copy of “Leading a Software Development Team” book and read chapter 2, “I’m taking over the leadership of an existing project // where do I start?” and get a thorough read:

The first thing that you should start to do is to review the situation. This involves more than just absorbing impressions; you need to organize these impressions into a framework. Try to organize your thoughts into the following areas, and in each area try to separate technical issues from personnel ones: - Where is the team now? (…) - Where is it supposed to be getting to? (…) - How does the team currently intend to continue?

(Whitehead, page 17)

Another highly pragmatic resource was Joel Spolsky, and his “Joel Test”:

The neat thing about The Joel Test is that it’s easy to get a quick yes or no to each question. You don’t have to figure out lines-of-code-per-day or average-bugs-per-inflection-point. Give your team 1 point for each “yes” answer.(…) A score of 12 is perfect, 11 is tolerable, but 10 or lower and you’ve got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time.

(Spolsky, 2000)

The “Joel Test” result for this team was 2 when I joined the team (they just had source control and good tools). When I left the company, they were running at 9 (we just did not have candidates writing code during interviews, nor testers, nor hallway usability testing).

For this project I took the following decisions:

• Since the priority for the client was to see results, I asked the developers to concentrate on stabilizing “visible” features, particularly on a visual report editor, that used a complex set of controls, similar to those of a drawing application, to create reports. Doing this, we could have a stabilized preview version that we showed to the client as early as one week after my arrival to the project.
• In agreement with the developers, we set up a daily build procedure, and I also asked them to provide a “client build” every Wednesday, that would be placed in a public directory available to the client. It turns out that the client never downloaded the binaries, but they liked to see the version numbers grow, and the binaries being delivered. Every week, Wednesday was the “public build” day, Thursday was the “bug correction” day, and Friday, Monday and Tuesday were “new features day”. Small stand-up meetings every day allowed us to know what was going on.
• Another important concern from the developers’ side was to have a quiet environment to work. They were constantly interrupted by the (quite nervous) managers to see their progress, and as such, I decided to stand in between both; I asked them to not to interrupt the developers for any reason, and to ask me for updates. I became a “proxy” between both, which reduced the tensions, and brought some peace to the developers.
• I created a fast project plan in our Intranet (there wasn’t any, so tracking the project was next to impossible) by asking the developers about the tasks they needed to do to finish the project, with the estimated time to do them, and setting some milestones. Since the project was in a wiki page, the developers could change the time estimations in case that they felt they had made a mistake; the only condition being to notify me of these changes.
• Using that information, I could create a couple of reports for everyone to see, and bring more visibility to the project:
  1. I wrote a weekly report stating the week’s achievements, the status of the project (number of open bugs, new functionality available, etc).
  2. In the intranet, I set up a couple of graphs and report tables, which were automatically updated every day.
• I did not take any technical decisions about the project; I gave full authority on this matter to the lead developer, who in turn appreciated this trust and took spontaneously the decision of documenting and unit testing the system thoroughly doing extra hours every day. This boosted the morale of the team, and the quality of the application as well. The other two developers contributed to these tasks as well, and the rhythm of releases and their quality increased in a couple of months. It turns out that the architecture of the system was particularly well done, and as such, adding new features was a relatively simple task, once the underlying framework was done; of course, during that time no visible results were available, which made everyone nervous.

Looking backwards, the only technical decision I’ve needed to take during this project was to use the company wiki; there I could add information pages that everyone contributed to, reducing the number of communication channels and reducing the misunderstandings between project team members. I cannot stress how much this helped; it provided a complete dashboard for everyone to refer to.

The most important problems in this project were human and customer related ones. By providing more visibility to the project, and by reducing the signal-to-noise ratio in the communication channels between developers and management, the team was able to provide the customer with a more reliable and full-featured product.

References

Joel Spolsky, “The Joel Test: 12 Steps for Better Code”, Wednesday, August 9th, 2000 [Internet] http://www.joelonsoftware.com/articles/fog0000000043.html (Accessed June 8th, 2007)

Whitehead, R.; “Leading a Software Development Team – A Developer’s Guide to Successfully Leading People & Projects”, Addison-Wesley, 2001, ISBN 0-201-67526-9

When in some situations it is not legally possible to use the title “software engineer” without an engineering degree of some kind (for example, in some states of the USA or some institutions like the IEEE – http://www.ieeeusa.org/policy/positions/titleengineer.html), the term “software developer” is usually applied to people in charge of designing, writing and / or maintaining software-based systems. I will use the terms developer and engineer interchangeably in this discussion, which some people might think is not correct. The discussion about the need of a formal certification process is a relatively new one:

Professional certification in the IT industry is a relatively recent phenomenon. It was begun in the late 1980s by Novell, Inc., an upstart networking vendor from Provo, Utah, in an effort to build market share and manage support costs for its products by building the skill levels of the people who worked with those products. Novell was one of the first companies to recognize the links between education/skills and product success. They knew that they could not build an education infrastructure that would support their worldwide marketing plans with their own resources. However, they also recognized that if they did not provide for skills acquisition for their highly technical products, they could never meet their product revenue goals.

(Shore)

However, no consensus about whether or not certification is needed has been reached yet. This article will highlight some of the problems raised by software engineering certification, which might explain the lack of consensus cited before:

• The first one has to do with the inherent extension of the software engineering field: are all software developers equal?
• The second one has to do with the large number of available certifications: which one to choose? Which ones are “reliable” indicators of expertise, and in which fields?

What is a “Software Engineer”?

In my career, I’ve found self-made people (I’m one of them, actually), real-estate architects, lawyers, mathematicians, economists and even geophysicists writing code for a life. What I’ve seen so far is that the most successful software developers are those who like doing it, no matter which profession they’ve followed. And the opposite is also true: many guys with a computer science degree discover, some time after they start their careers, that they definitely do not like that code thing.

One of the biggest problems with certifications is that there is not such thing as a “single kind” of software developer:

• There are those who write games, and spend most of their time writing in low-level languages for game consoles, optimizing for speed and space, and creating three-dimensional worlds using as little memory as possible…
• There are those who write web-based applications, and spend their time creating 3-tier architectures, talking to a database, using some kind of object-oriented platform, and luckily exposing some data using XML web services, dealing with cross-browser issues, and wondering what is all that fuss about Web 2.0…
• There are those who write operating systems, and work for some embedded software company, or hack Linux kernel device drivers every night, or work for Microsoft or Sun or Apple, and spend most of their time discussing whether microkernels are better than monolithic architectures…
• There are those who have the ill fate of working as a consultant, and spend more time switching from project to project every day, or dealing more with corporate politics, rather than with code…
• There are those who manage projects and spend more time in their mailing list or in Microsoft Project rather than being able to code (and then complain about this in their blogs)…
• There are those who have a software engineering degree, but work for ZDNet writing about industry trends…
• There are those who turn into human resource consultants, and try to keep up to date on the new trends, but feel completely lost given what they learnt in university…
• There are those who do a little bit of all what I’ve mentioned above, and are or not really good at all of them…
• And finally there are those who might fit any of the characteristics above, but would have preferred not to listen their parents and rather open that scuba-diving shop in Honolulu.

Available Certifications

This diversity explains the existence of more common product-specific certifications: you can be certified to use Microsoft technologies (http://www.microsoft.com/learning/mcp/default.mspx), MySQL databases (http://www.mysql.com/certification/), Apple servers (http://www.apple.com/xserve/raid/certifications.html), various IBM products (http://www-03.ibm.com/certify/certs/index.shtml), Java development stacks (http://www.sun.com/training/certification/java/index.xml), Cisco routers (http://forums.cisco.com/eforum/servlet/CCNP?page=main), RedHat Linux installations (https://www.redhat.com/training/certification/) or UML diagrams (http://www.omg.org/uml-certification/index.htm)

However, given that technology companies have interest in having many people taking their certifications, their affordability and low-entry barriers to get them, many of these become much easy to get than they should be, and as a result, they lose credibility, and do not help IT recruiters to filter properly software developers during the selection processes. I’ve heard many complaints of project managers regarding these certifications, and I think it’s a generalized feeling:

“Certified skills pay has not just flat lined, it’s in the negative. This is big news if you’re certified and you’re thinking about getting recertified,” said Foote. “This trend is in the fourth quarter, that pay for certifications is on the wane, while non-certified skills are growing in pay.”(…) Certifications are losing value because employers are looking for more in their workers than the ability to pass an exam; they want business-articulate IT pros.”

(Rothberg, 2006)

Bruce Schneier, a well-known security researcher, has written about security certifications as well, with a mixed feeling:

In the end, certifications are like profiling. They work , but they’re sloppy. Just because someone has a particular certification doesn’t mean that he has the security expertise you’re looking for (in other words, there are false positives). And just because someone doesn’t have a security certification doesn’t mean that he doesn’t have the required security expertise (false negatives). But we use them for the same reason we profile: We don’t have the time, patience, or ability to test for what we’re looking for explicitly.

(Schneier, 2006)

The conclusion of all of this is that the debate is pretty much still open, and that there is not a simple answer to it.

Market Fragmentation

There is an interesting anonymous comment in Schneier’s website as well:

Another thought on certification is they are not all equal. There are Vendor Certs. Microsoft’s MCP/MCSE, CISCO CCNA/CCNP/CCIE Pro: The canidate is likley to know how to work on your specific platform. Con: The canidate is likely to think in only the vendor’s interest. There are Certs to assure knowledge of standard security terminlogy. ISC CISSP Pro: Can talk strategy and evaluate the nine domains to evaluate how the company is doing overall Cons: Most likely could not tell you what the nineth byte of an ip packet means or if OpenSSL is out of date on Red Hat Linux. Topic specific, vendor neutral. SANS GIAC Pro: Vendor neutral. A lot of focus on specific skills in NIDS or Hardening Windows, Incident Handeling, etc. Con: Concentration on open source tools since they are easily available, but it does not seem to impress all employers.

(@nonymou5, in Schneier, 2006, spelling mistakes not corrected)

I think that this comment summarizes pretty well another problem with certifications: there is a great level of fragmentation in today’s market. Every single important technology in the IT world requires a huge investment in time and practice in order to master it, and this translates in a huge complexity for the developers to choose the right certification. All of these without taking into account the large number of IT-related university degrees available, online or not.

Conclusion

The term “software engineer” is sufficiently vague, and the number of “certifications” sufficiently large, as to allow a single “yes or no” answer to whether professionals in the software sector should be certified or not. I personally think that I would rather avoid vendor-specific certifications as far as possible, and choose university-related or problem domain related certifications instead, to keep my career options open, and my mind free of marketing.

References

Rothberg, Deborah; “Another Nail in the IT Certification Coffin”, Novembrer 3rd, 2006, [Internet] http://www.eweek.com/article2/0,1895,2051272,00.asp (Accessed June 3rd, 2007)

Schneier, Bruce; “Security Certifications”, July 20th, 2006, [Internet] http://www.schneier.com/blog/archives/2006/07/security_certif.html (Accessed June 3rd, 2007)

Shore, Julie; “Why Certification? The Applicability of IT Certifications to College and University Curricula”, [Internet] http://www.developer.ibm.com/university/scholars/certification/ebusiness/pdfs/why-certification.pdf (Accessed June 3rd, 2007)

Given the fact that Web Development equals Software Development, this discussion will be of interest to those building the smallest websites, as well as old desktop-intensive apps. It will not be a “Tell me what programming language you use, and I will tell you who you are” type of article, though it may look like one, because that is something you have to figure out all by yourself.

If you take a look at the list of programming languages, any business person would have an instant headache. There are lots of them. With the strangest names. You cannot possibly guess which one to pick from such a list, obviously. So, how do companies choose the languages they use? There are some straightforward methods that I have seen so far, in no particular order:

• Looking at what other companies use (typically Google, Microsoft, Apple or Sun, but it could be 37signals too).
• Following the advice of the CIO, the Lead Architect or some other politically-powered person, which might or might not have read this article ;)
• Looking at what the current pool of programmers in the company know how to use. Rinse, wash, repeat.
• Following hype.
• Because there is a market plenty of available, cheap programmers that I could use for this project.
• Taking into account the characteristics of the languages themselves (static vs. dynamic, etc).
• Following the company’s history of past projects (successful or not).
• Following what your the client suggests (or mandates).

I think that it is a very bad idea to take any of the above methods in isolation, without considering other factors. Doing so is a path to self-destruction in the medium to long term, even if you succeed in the short term.

Just as a small background for people not into programming: you can safely (roughly) group programming languages in a table like this:

In a static language all variable type references are bound at compile time; in a dynamic language, this is done at runtime (which allows you to assign a string or an int to the same variable). In a strongly-typed language, either the compiler or the runtime enforces the operations you can and cannot do on an object (depending on its type, as you may have guessed). In a weakly-typed one, there is no such restriction, and you can perform implicit conversions from one type to the other. And then you have functional ones, but that is another problem, because there are many programming paradigms out there. And then there is the hybrid ones, which I love as Steve Yegge does:

But also there’s, like, the Boo language, the io language, there’s the Scala language, you know, I mean there’s Nice, and Pizza, have you guys heard about these ones? I mean there’s a bunch of good languages out there, right? Some of them are really good dynamically typed languages. Some of them are, you know, strongly [statically] typed. And some are hybrids, which I personally really like.

He did not include Objective-C as “hybrid”, but I think it is. Anyway, so much for the theory, here is the main point of this article:

First of all, I consider programming languages (I know a few of them, and I have my personal picks) just as tools to get things done™. Nothing else. I think of them as hammers or Black & Decker screwdrivers or saws or nail guns.

Second, I believe that specialization is for insects. Getting stuck in a single programming language because of any of the reasons I have enumerated above is just stupid.

So what I want to say is: You need polyglot programmers in your team, like you need a team of people knowledgeable in many human languages in every company doing business at global scale. I would say even more, you need not only people fluent in western languages (like English, French, Spanish and Italian, which is my combination) but also in other languages, with different paradigms behind, like Arab, Hebrew, Hindi or Chinese.

What does that mean in programming terms? You want to have programmers in your team being able to use different languages in different ways; you want to have programmers that learn new languages every so often, just for the sake of it. And most importantly, you want to get rid of programmers that not only get stuck on a single language, but that, even worse, dismiss any other way to do things. Having people that takes a negative look on the learning side of things can bring your whole company to a dead-end. This industry is plenty of integrists, and you do not want that in your company.

For example, JavaScript can be used as a procedural, object-oriented or functional programming language. Does your JavaScripters know how to write functional JavaScript? If not, that is too bad, because they will not be able to fully understand what is going on behind the scenes in Prototype or jQuery then. Of course this will not block them from using those libraries, but they might not understand how to apply some interesting patterns in their own code.

Ask more about your programmers: do they know how to do complex C++ template metaprogramming? Are they aware of some performance problems brought by garbage collectors? Do they follow the latest evolutions of the next version of JavaScript? (even if they do not like them) Which blogs do they follow? Do they know why some people hate PHP? Do they know what a continuation server is? Which programming books have they read lately?

And finally, what is even more important than having chosen a good programming language? Your methodology. Ask yourself (or your team) about these points:

• Do you write unit tests? You might not have testers, which is a bad idea anyway, but that does not block you from testing code yourself (Steve Yegge):

<blockquote>
    And I would say it's a pain in the butt, but I mean... it's a pain in the butt because... a static type-systems researcher will tell you that unit tests are a poor man's type system. The compiler ought to be able to predict these errors and tell you the errors, way in advance of you ever running the program.
</blockquote>

• Do you use defensive programming techniques?
• Do you have a wiki, a project website or at least good documentation in your code? If not, how do your new hires learn about your product? No, reading the code is NOT a good answer.
• Is your chosen programming language portable? You might think that your system will never have to run on Linux or Mac, but why stuck yourself on purpose?

The common factor of all the above items is that you have to manage complexity. If you write code, you are creating complex stuff, and one of the best ways to manage complexity is to create small systems; as Steve Yegge said:

Small systems are not only easier to optimize, they’re possible to optimize. And I mean globally optimize.

And this is why you do not only need polyglot, but also minimalist programmers in your team. Small is beautiful. Paul Graham (of Y Combinator fame) knows that dynamic languages yield small systems:

The right tools can help us avoid this danger. A good programming language should, like oil paint, make it easy to change your mind. Dynamic typing is a win here because you don’t have to commit to specific data representations up front. But the key to flexibility, I think, is to make the language very abstract. The easiest program to change is one that’s very short.

And how can you get small programs? By choosing the right programming language. Which brings us to the beginning of this post! So, here go some tips for all of you looking for the right programming language:

• Prefer strongly-typed, dynamic languages; there are a lot of reasons for that, particularly those exposed by Steve Yegge in his excellent presentation at Stanford:

Yeah, sure, it catches a few trivial errors, but what happens is, when you go from Java to JavaScript or Python, you switch into a different mode of programming, where you look a lot more carefully at your code. And I would argue that a compiler can actually get you into a mode where you just submit this batch job to your compiler, and it comes back and says “Oh, no, you forgot a semicolon”, and you’re like, “Yeah, yeah, yeah.” And you’re not even really thinking about it anymore. Which, unfortunately, means you’re not thinking very carefully about the algorithms either. I would argue that you actually craft better code as a dynamic language programmer in part because you’re forced to. But it winds up being a good thing.

• Some points about his talk:
  • Many of the caveats of dynamic languages are not (so) true anymore (like the lack of decent IDEs or the performance problems);
  • There is a lot of research going on nowadays on the performance of programming languages, and this means that code written in these languages will benefit from many improvements in the near future, for free;
  • And yes, there is the hype factor I have mentioned above; the cool kids are using them:

Which makes them exactly the kind of programmers companies should want to hire. Hence what, for lack of a better name, I’ll call the Python paradox: if a company chooses to write its software in a comparatively esoteric language, they’ll be able to hire better programmers, because they’ll attract only those who cared enough to learn it. And for programmers the paradox is even more pronounced: the language to learn, if you want to get a good job, is a language that people don’t learn merely to get a job.

• Teach yourselves; setup internal workshops to have all your team learn how to do cool stuff with those languages you have read about in DDJ.
• Teach the community around you, and do not be scared of competition; have your team write papers, articles on business or technical journals, publish code as open source projects, and show that you can go beyond.
• Keep an open mind: continuation servers, Comet or multicore processors are the future. Is your team prepared?

Software is a social process. Once you get this in your mind, and get your team working proactively, collaboratively and teaching each other, the choice of a programming language comes in second place.

[source:python] import unittest

loader = unittest.TestLoader() tests = loader.loadTestsFromName(‘path.to.your.tests.package’) for test in tests.tests: print test._tests[0].class.name.replace(“Test”, “”) for method in test._tests: print ” %s” % method._testMethodName.replace(“test“, “”).capitalize().replace(“_”, ” “) [/source]

This would yield something like this: Business     Accounts have at least one entry     Clerks cannot close accounts Security     Users can create new accounts     Anonymous users cannot access private areas ...

Of course, you’ll get better results if you follow Google’s naming conventions for your tests… ;) This is not rspec (nor an alternative to it) but it might be useful to some of you.

Just as a reminder for Django users: you might need to

setenv DJANGO_SETTINGS_MODULE application.settings

or

export DJANGO_SETTINGS_MODULE=application.settings

in order to make the script work properly! At least I had to :)

I thought it was time to group all those entries into a WordPress category of its own: “Quality”. Feel free to check the articles in there, and of course, to leave any comments that you want.

Total Quality Management is one of the founding pillars of modern mass-production economy, of which the software industry is by far the youngest (and most rebel) child. This article will provide a short discussion on some TQM principles and about their applicability to software projects. TQM Principles

Joel Spolsky has written a brilliant introduction to software testing in his famous “Joel on Software” blog:

Software has bugs. CPUs are outrageously finicky. They absolutely refuse to deal with things that they weren’t taught to deal with explicitly, and they tend to refuse in the most childish of ways. When my laptop is away from home, it tends to crash a lot because it can’t find the network printer it’s used to finding. What a baby. It probably comes down to a single line of code somewhere with a teensy tiny almost insignificant bug in it.

(Spolsky, 2000)

The software industry has been a victim of its own success. Software companies have experienced the fastest growing rates in the history of capitalism, and this rush to conquer juicy worldwide markets, and to offer services to literally billions of human beings has, more often than not, been done at the expense of proper quality guidelines.

William Deming proposed 14 key principles that form the basis of what became TQM in the second half of the 20th century:

1. Create constancy of purpose for the improvement of product and service (…)
2. Adopt a new philosophy of cooperation (win-win) (…)
3. Cease dependence on mass inspection to achieve quality. Instead, improve the process and build quality into the product in the first place.
4. End the practice of awarding business on the basis of price tag alone. Instead, minimize total cost in the long run. (…)
5. Improve constantly, and forever, the system of production, service, planning, of any activity. (…)
6. Institute training for skills.
7. Adopt and institute leadership for the management of people, recognizing their different abilities, capabilities, and aspiration. (…)
8. Drive out fear and build trust so that everyone can work more effectively.
9. Break down barriers between departments. Abolish competition and build a win-win system of cooperation within the organization. (…)
10. Eliminate slogans, exhortations, and targets asking for zero defects or new levels of productivity. (…)
11. Eliminate numerical goals, numerical quotas and management by objectives. (…)
12. Remove barriers (…) abolishing the annual rating or merit system that ranks people and creates competition and conflict.
13. Institute a vigorous program of education and self-improvement.
14. Put everybody in the company to work to accomplish the transformation. The transformation is everybody’s job.

(Wikipedia, emphasis added)

In the following section, I will discuss how these principles overlap and intersect, and how they can be applied (and have been applied historically) in software companies.

Focus on the People

From the above list, some commonalities appear for all and each one of the principles: the biggest one in my opinion is the focus on the people. As such, knowing that software is a pure mind product, completely intangible and tremendously flexible, I think more than ever that software is a human & social process. This focus in people is particularly evident in principles 2, 6, 7, 8, 9, 10, 13 and 14.

For example, reducing the causes of internal and external conflict in the organization (principles 12, 9, 2), makes software developers able to collaborate with other departments or companies, creating integrated software suites that become de facto standards in their respective industries. A very extreme example of this phenomenon is Microsoft’s Office System, which was created from separate products created by different teams, such as Word and Excel. The synergy created by the developers working together, not only unifying the UI look & feel or the programming model, but also increasing the interoperability of both systems, created a product that effectively is much more than just two products together.

Moreover, the Peopleware book by DeMarco and Lister showed that historically, the most successful software companies have been those that excelled in creating a human-centric environment:

In 1982, (Mitchell Kapor) founded Lotus Development Corporation, for which he is most noted. While there, he revolutionized corporate workplace culture by making diversity and inclusivity top priorities in his goal for creating an environment that attracted and retained employees. There were many “firsts” for Lotus, including being the first company to sponsor an AIDS Walk event in the mid-80′s and refusing to do business with South Africa due to Apartheid.

(Sterling-Hoffman)

Thanks to a sharp hiring process, a series of innovations in their flagship spreadsheet product, and a progressive corporate culture, Lotus dominated the software landscape of the 80s. Today, Google follows very closely Lotus’ steps (Google, 2007a), and their brilliant results in the last few years seem to confirm this trend. Google applies principle 13 very strongly, allowing their employees to use 20% of their time in their own projects (Google, 2007b). This is resulting in an incredible amount of code, used internally and also released as open-source projects, such as the MacFUSE project (Google Mac Blog, 2007):

Google is a fantastic company to work for. I could cite numerous reasons why. Take the concept of “20 percent time.” Google engineers are encouraged to spend 20 percent of their time pursuing projects they’re passionate about. I started one such exciting project some time back, and I’m pleased to announce that Google is releasing the fruits of this project as an open source contribution to the Macintosh community. That project is MacFUSE, a Mac OS X version of the popular FUSE (File System in User Space) mechanism, which was created for Linux and subsequently ported to FreeBSD.

Conclusion

Deming’s principles are today, more than ever, extremely important in the software industry. With a very high rate of turnover and burnout, software companies are faced with the choice of stop considering their staff as a “resource” but rather as an “asset”. The most successful companies in the field are not only those that are able to cut costs effectively (following the fourth principle) but also those that arrive to empower their staff, increasing creativity, well-being and innovation, which ultimately leads to market leadership and economic success.

References

DeMarco, Tom & Lister, Timothy, “Peopleware – Productive Projects and Teams, 2nd Edition”, 1999, Dorset House Publishing, ISBN 0-932633-43-9

Google, “Top 10 Reasons to Work at Google”, 2007a [Internet] http://www.google.com/jobs/reasons.html (Accessed April 5th, 2007)

Google, “What’s it like to work in Engineering, Operations, & IT?”, 2007b, [Internet] http://www.google.com/support/jobs/bin/static.py?page=about.html (Accessed April 5th, 2007)

Google Mac Blog, “Taming Mac OS X File Systems”, January 11th, 2007, [Internet] http://googlemac.blogspot.com/2007/01/taming-mac-os-x-file-systems.html (Accessed April 5th, 2007)

Lewis, W.; Veerapillai, G.; “Software Testing and Continuous Quality Improvement”, Auerbach Publications, 2005, ISBN 0-8493-2524-2

Spolsky, J.; “Top Five (Wrong) Reasons You Don’t Have Testers”, [Internet] http://www.joelonsoftware.com/articles/fog0000000067.html (Accessed April 5th, 2007)

Sterling-Hoffman, “Opening Doors To Higher Education”, [Internet] http://www.sterlinghoffman.com/newsletter/articles/article140.html (Accessed April 5th, 2007)

Wikipedia, “William Edward Deming”, [Internet] http://en.wikipedia.org/wiki/W._Edwards_Deming (Accessed April 5th, 2007)