Mark Staples

A colleague forwarded to me a pointer to the current issue of the ITMPI journal, on Best Practices in Configuration Management. They make an interesting contrast with the best CM practices in the November issue of the CM Journal on “What Best Practice Is Best?”, and specifically the article CM: The Next Generation of Top 10 Best Practices.

The CM Journal article’s list is as follows:

• 1. Use of Change Packages
• 2. Stream-based Branching Strategy – do not overload branching
• 3. Status flow for all records with Clear In Box Assignments
• 4. Data record Owner and Assignee
• 5. Continuous integration with automated nightly builds from the CM repository
• 6. Dumb numbering
• 7. Main branch per release vs Main Trunk
• 8. Enforce change traceability to Features/Problem Reports
• 9. Automate administration to remove human error
• 10.Tailor your user interface closely to your process
• 11. Org chart integrated with CM tool
• 12. Change control of requirements
• 13. Continuous Automation
• 14. Warm-standby disaster recovery
• 15. Use Live data CRB/CIB meetings
• 16. A Problem is not a problem until it’s in the CM repository
• 17. Use tags and separate variant code into separate files
• 18a. Separate Problems/Issues/Defects from Activities/Features/Tasks
• 18b. Separate customer requests from Engineering problems/features
• 19. Change promotion vs Promotion Branches
• 20. Separate products for shared code

The ITMPI Journal articles lists just 3 best practices:

• Software Configuration Management Audits (This isn’t really covered well by the 20 practices above, but is somewhat related to practices 5 and 13.)
• Risk-Based Configuration Control – Balancing Flexibility with Stability (This is somewhat related to practices 7, 11, 15, 16, 18a/b, and 19.)
• Bidirectional Requirements Traceability (This is somewhat related to practices 1, 2, 3, 4, 8, and 12.)

The ITMPI Journal article doesn’t really cover the practices 6, 9, 10, 14, 17, and 20, partly because the CM journal article practices are at a lower level of detail and cover a few practical/administrative issues outside of the core of CM theory.

Anyway, the practices in both these lists are all “good” for “most” development groups. They aren’t rocket science, but I’d say there are a fair number of development groups out there that barely do any of them. Although version control is ubiquitous in industry, CM is still very poorly understood by most practitioners.

For what it’s worth, I think that for most commercial software development, the most important practice isn’t about sophisticated version control issues or processes – it’s a simple but critical piece of organisational structure and policy.

All Customer Deliveries Go Through the Release Team
Create a release team that’s different to the development team. Ensure all customer software deliveries happen through the release team. Have the most senior manager you can find declare it a firing offense for a developer to ship code or patches directly from their desktop to the customer. Even in “emergency” situations.

More than once I’ve seen or heard of companies that own high-end commercial SCM tools, but make those tools worthless because developers sometimes email patches directly to the customer. (Sometimes in the rush they also forget to check in their changes to version control, which just makes the nightmare worse.) The developers are usually trying to “get the job done”, and be “helpful” to a customer with a problem. Sadly, they end up causing bigger problems for everyone down the line. If you’re not managing releases properly, you don’t know what code the customer has – you degrade your ability to diagnose faults, you won’t be able to ship working patches in future, and you might easily introduce a regression problem by losing the fix itself.

Our paper “An Exploratory Study of Why organisations Do Not Adopt CMMI” has started to be cited in the literature. It’s been in the “hottest 25 downloads” for the journal since it became available 3 quarters ago, so I’m hopeful the number of citations might grow.

On one hand, it’s great to cited! On the other hand, I worry that the results could be mis-interpreted. Specifically, people might be tempted to mis-read the paper as saying “small companies can’t do CMMI”. We don’t say that. So, I thought I’d try to clarify a few points here…

The main question we do provide some initial evidence for is:

• When companies decide not to do CMMI, why do they say “no”?

Here’s some questions we don’t provide evidence for in the paper:

• Do most companies do CMMI?
• Can most companies do CMMI?
• Do most companies think they can do CMMI?

(These are alternate “complementary research questions” – by saying what my main research question is NOT, I hope you get a better understanding about the boundaries of the main question, and how to interpret the results.)

In looking at the main question, we broke down the companies by size to see if we could get hints about why companies gave the reasons they did. Small companies tended to give different kinds of reasons than medium and large companies. Roughly, small companies said they couldn’t do CMMI, whereas medium and large companies said they shouldn’t.

But obviously some small companies can do CMMI! There are fairly credible reports of it in the literature. So, what’s going on?

What’s going on is that my target population is not “all software companies”. My target population is software companies that have decided not to do CMMI. To start my research, I first exclude companies that are doing CMMI, or that want to do it.

It’s interesting to progressively partition the population of companies, and think about where most software process improvement research happens. (This looks better as an animation, so think about these pictures as a sort of flick book in your mind’s eye.)

First, there’s the population of every organisation that might reasonably use CMMI.

Some of them will not have heard of CMMI, but some will have. You could study the differences between these groups as a piece of advertising or diffusion research, but I’m not aware of anyone who’s published research on this for CMMI or any other SPI approach.

Of the organisations that have heard of CMMI, some will have decided not to adopt it, and some will have decided to adopt it. This is the point in the adoption lifecycle that we examine in our paper. It is an extremely rare kind of research in software engineering.

For interest’s sake, let’s drill down some more…

Of the organisations that have tried to adopt CMMI, some will have succeeded in rolling it out, and some will have failed.

Of those who succeeded in adopting CMMI, some organisations will have got some benefit, while others will have had no benefit.

Where do you think most CMMI/SPI research sits?

Most research papers report on how organisations benefitted from using CMMI. In the literature, you almost never hear about organisations who tried and failed to adopt CMMI, nor about organisations who adopted but then failed to benefit from CMMI. (This effect is called selection bias, or publication bias, depending on how it arises.) It’s sort of understandable – companies don’t want to let themselves be seen in a bad light, and researchers may think that success is more glamorous than failure.

Looking at successful cases is necessary, but to really broaden the impact of software engineering research, we need to learn from failures as well!

The recent announcement by Somasegar at Microsoft that F# is being “productized” has now started to be picked up by various news agencies. Being productized means some future version of the F# language and its Visual Studio integration will be officially supported by Microsoft.

However, I see a couple of people are implying that because F# is functional language, it’s not an OO language. Wrong! It’s both.

Somasegar says F# will be appealing to developers in the domains of “financial, scientific and technical computing.” He says that’s because of the correspondence you can achieve between F# and the mathematics that those developers work with in their domains. I say, also because of the speedy execution performance you can get from F#. Strong static typing in F# gives the compiler many more opportunities to be much smarter about optimization. Speed matters when you’re working with massive data sets and data streams. Dynamic languages will always struggle in comparison.

F# also has great potential as a teaching language in Universities, and Somasegar says that’s a target. So, I really hope Microsoft puts out a free Visual Studio Express edition for F# to make sure the barriers to getting started with F# on Windows are as low as they can be. F# can also run on mono on *nix, so really there’s no excuse to not give it a try whatever platform your University uses.

F# is particularly nice move for Microsoft as a company, because it’s such a strong story of home-grown innovation. I’d agree with Somasegar’s opinion that “[F#] is one of the best things that has happened at Microsoft ever since we created Microsoft Research over 15 years ago.”

p.s. Does anyone know how to search for “f#” on google news without finding faux-swear words? It’s f#@&!ng annoying.

“Web services” has been a buzz word for so long you might wonder if there’s any buzz left. I’d known in principle about how web service technology was full of goodness, especially for achieving interoperability, but I’d never really taken the red pill. However, recently I had an epiphany.

I’ve been working in a team using the Confluence wiki to organise some information. Confluence is a page-based wiki, but I’ve been pushing it a little in the direction of being a semantic wiki, by using 2-column tables on some pages, to represent key-value pairs for page attributes and relationships. My problem was that, to Confluence, those tables were just ordinary textual content. There was no way to check that the special tables were well-formed, and there was no way to query the wiki using the page attributes and relationships.

My first plan was to export the wiki to XML and query that, but then I discovered that Confluence has a SOAP API. It looked promising, and it also looked like a good excuse to do some more scripting with F#.

I bumbled around the web looking for simple guide on how to actually use SOAP, and finally came across a page from Robert Pickering’s site that laid it all out for me. For Confluence, it goes like this:

1. Generate a C# file from the Confluence WSDL file:

   wsdl http://<your-confluence-path-here>/rpc/soap-axis/confluenceservice-v1?wsdl

2. Compile it to produce a dll:

   csc /target:library ConfluenceSoapServiceService.cs

(My wsdl.exe is in C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin, and my csc.exe is in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727, but yours might be somewhere else.)

Then you’re set to go. For the purposes of exposition, say you want a page called “PAGE” from a Confluence space called “SPACE”, to which you’ve got anonymous access:

1. In the F# interactive environment, load the dll, e.g. :

   #r "ConfluenceSoapServiceService.dll"

2. Then, create a new type F# object to expose that SOAP API:

   let wiki = new ConfluenceSoapServiceService()

   (Right now, in the interactive environment, I need to call this twice to get it to work, because of some strange F# interactive bug, but Don assures me it works first time when it’s compiled.)

3. The API functions are available as members of that new object, just like they were written natively in F#:

   let page = wiki.getPage("", "SPACE", "PAGE")

4. But it’s not just the API functions – the values too are all just like they were written natively in F#. For example, the page returned above is in the RemotePage type. It has fields for things such as the page content:

   page.content

After that I was away – the standard libraries and seamless .net interoperability provided by F# made most things easy. The hardest part of the exercise was working out how to parse the Confluence wikicode using regexes. (Oh, the pain!)

So, now I’ve seen the light – web services really are ace. I was accessing an API implemented in Java, running on *nix, but to me it looked like it was native to F#, running on Windows. And setting it up took almost no effort. (After I found out how it was done. :-) )

I think my epiphany was mostly about web services, or was it about F#?

Today, version 7.04 (“Feisty Fawn”) of Ubuntu Linux was released. Yay! Earlier this week I installed the Feisty beta release, and rediscovered Linux happiness.

I first started using Linux at home in the mid/late 90s, installing Redhat Linux 4 on a second-hand PC. I faithfully followed the Redhat releases, installing Redhat 6 on a new PC, and eventually upgrading to Redhat 9. Then, Redhat went all “enterprise”, and stopped their free package update management service. My old Redhat 9 box slowly became impossible to maintain. Maybe Redhat thought focussing on the enterprise was good for them – and maybe it did help their stock price for a while. At the time there was no credible transition (e.g. to Fedora), and so as a home user I just felt abandoned. Now I think Redhat’s getting what’s coming to them.

To successfully install Redhat 4, I had to host a private “installation party”, and buy pizzas for the Linux geeks who came around to mess around with OS configuration files on the command line. For a while I thought my Feisty beta installation experience was going to be similar. The desktop install stopped half-way through (ubiquity crashed), and then the alternate install managed to corrupt my software RAID configuration before failing. But finally the alternate installation worked without software RAID. Afterwards I was left in 800×600 screen resolution for a while, until I discovered some advice on the net about using dpkg-reconfigure.

There are still dome problems remaining now I’ve finished the installation. My box won’t poweroff when I shutdown. And, it’s still hard for Windows and Linux boxes to play nicely together on my network. I can share files in one direction but not the other, and it hasn’t “just worked” setting up my Linux box as a network print server for its local printer.

But, these are minor problems that don’t diminish my renewed Linux happiness. The desktop menus are clean, the system administration tools work (apart from the problems noted above), the package update management is fantastic, I like the use of sudo for root-less security, and it’s very neat to be asked to install system features only if/when you need them. Now we’ve upgraded, our scanner finally works, and we’ve been able to install Wesnoth to keep Liam occupied these school holidays.

Our paper on why companies don’t adopt CMMI is now available online. This paper describes the details of some issues I’ve talked about previously to various industry audiences. In this study, we found that some companies (especially small ones) felt there were barriers to adopting CMMI. For example, often they didn’t get around to considering the possible benefits of adopting CMMI, because they perceived that it would just be too costly or time-consuming. The goal of this sort of research is to better understand the needs of software-developing companies, to help researchers develop software process improvement approaches that are more relevant and accessible to companies. There are more details in the paper.

Maybe of note is that the article is currently #20 in the Top 25 Hottest Articles for the journal, even before it’s appeared in print!

Joe Hurd is in town – he’s been working with some other local researchers on how to verify probabilistic computing systems. We were looking for some place to meet and catch up with each other, so I suggested we go to the Linux Conference Australia 2007 Open Day.

It was an interesting afternoon – a whole lot of exhibits, talks, and free food.

There were things like non-standard UI devices (dance mats, Wii Remotes, body/motion-detecting cameras) connected to Linux, Linux on amateur rockets and amateur satellites (“running Debian unstable”), commercial-and-homebrew Linux PVRs, a homebrew Linux-based Segway, and a homebrew Linux-based 3D-printer (“prints 1cc per hour”). Oh, and flashing fridge magnets from Google.

I took along Liam, my grade 1-aged son. I sat him down in front of one of the One Laptop Per Child laptops – there were three of them there at an exhibit. (These are the $100 laptops that you can’t actually buy yourself, even for $300.) We played around, and had someone run us through a demo. I don’t know… Liam happily uses both Windows and Linux at home, but he (and I) were a little stumped by the OLPC user interface. Maybe it was just our previous experience tainting us, but it seems a little strange to put a “non-standard” GUI on a mass-market item. Though I guess if it sells in the volumes they’re hoping, maybe it will end up being a “standard”!

The day I got back from Rio, I was on the plane again to go to Melbourne, for the SEPG Australia conference, where I presented a talk Why Organisation’s (Don’t) Choose CMMI. The talk was a variant of the one I’d done for the ESE Breakfast seminar. For this audience I didn’t need to provide quite so much background on CMMI, and so I expanded a little on the setting and interpretation of the work instead.

The talk included some material from our paper An Exploratory Study of Why Organizations Do Not Adopt CMMI, which has now been accepted for publication, in the Journal of Systems and Software.

On Tuesday 26 September I got back from Rio de Janeiro, after having attended ISERN (a research network), IASESE (a school), and ISESE (a conference). A full week of empirical software engineering goodness, but sadly not much spare time to look around Rio.

ISESE (the International Symposium on Empirical Software Engineering) was the main reason I was there. I presented a paper Evaluating Guidelines for Empirical Software Engineering Studies, which was about how we evaluated a proposed set of guidelines for reporting controlled experiments in software engineering. I was one of 10 (!) authors. We used a perspective-based reading approach to review the guidelines from the perspectives of various kinds of stakeholders (Researchers, Reviewers, Replicators, Meta-Analysts, Practioner/Consultants, Authors). We expressed the needs of the stakeholders as questions about papers that we imagined had been written using the guidelines.

My last full day in Rio was almost too exciting – in the morning Emilia Mendes and I did a tag-team presentation of Maggie Wojcicki’s paper (Maggie had lost her voice on the flight to Rio), in the afternoon both (coincidentally) my paper and Emilia’s won distinguished paper awards at the conference, and then after a few rounds of volleyball on the beach, I got to do a bit of surf lifesaving when a student got caught in a rip. Luckily a local swam out to help with the rescue – I assisted for a while, but by the end of it all I was just rescuing myself. Everyone was OK.

Josef Nedstam and I have finally had our paper Evolving Strategies for Software Architecture and Reuse accepted for publication, in the Journal Software Process: Improvement and Practice. It’s been through a couple of iterations/submissions, but from go-to-whoa it will have taken about 3 years to see this published from when we first wrote it. It’s a nice story, mostly based on data from Josef’s PhD work, about how the best form of architecture-driven reuse for an organization depends on its business situation, and how they both can change over time. It’s an extension/response to Jan Bosch’s view of increasing levels of maturity for reuse (product line development).