Mark Staples

The discussion earlier this year about the death of the blogosphere is surely exaggerated.  OK, my blog was “resting” for most of this year. My excuse is general busy-ness – moving house, moving office, and changing roles at NICTA. But recently I had the enthusiasm and time to write a flurry of blog entries. (I’ve had more time during my newly extended train commute, but notwithstanding that, I can see my blogging enthusiasm comes in bursts…)

Isn’t this how most of the blogosphere works? Most bloggers are amateurs, writing about their family, pets, or hobbies. The glamorous fantasy of blogging driving democratic journalism and incisive public commentary is true, but it’s only ever been true for only a tiny part of the whole.

It takes a certain perverse commitment to blog regularly if you’re not being paid for it. Of course increasingly, some bloggers do get paid for it – either as journalists, company employees or, for an elite influential few, through significant online ad revenue. But the fact that some people are paid for it doesn’t significantly affect the cost or value of blogging for the mass of amateurs.

So Nic Carr’s wrong to say that blogging “outside the bounds of the traditional media is gone” – the blogosphere is not dead. Yes, as the Economist says “Blogging has entered the mainstream” – blogging is now accepted a part of the spectrum of modern media. But non-mainstream blogging hasn’t died. There are still plenty of blogs about family, pets, hobbies, and there are still individuals reporting and providing independent social commentary.

Ironically, at the same time as blogging technology is becoming accepted by the mainstream media, it’s also becoming accepted for other more industrial purposes. Blogging technology is no longer just for blogging – the formats that support blogs (RSS, Atom, etc) are used as a REST representation for representing  time-series content, including mundane things such as home loan product announcements.

It’s certainly not dead, but both socially and technically, blogging is growing and adapting. It used to be the message, now increasingly it’s the medium.

I had to miss the second day of ICSOC, but was back for the morning of the third, and another great keynote, on Web-Scale Computing, from Peter Vosshall – a VP and Distinguished Engineer at Amazon. Amazon needs a highly reliable and scalable infrastructure internally to run its retail business, but has also been selling web services infrastructure to third parties. Peter spoke about EC2 (compute), SQS (messaging), S3 (storage of blobs with metadata), SimpleDB (storage of lightly-structured data with indexed queries), and EBS (storage for EC2 when you need a traditional filesystem or database).

As an example of how companies are using and benefitting from these services, he talked about a company called Animoto.  On their website you can upload a song and some photos, and they automatically build a video montage, matching transitions to beats.  They started with around 5000 customers in total, but after they built a facebook app and got some viral awareness, they shot up to 5000 to 10000 users per hour. They had deployed on EC2 and ramped up to 3500 – 5000 instances.  It looked like a neat story.

The business benefits of using the web services are having a capability for fast incremental infrastructure growth, and turning what would have been fixed capital expenses into variable operating expenses.  (Coincidentally I had also mentioned this latter benefit of web services in a podcasted interview I was in on Monday.)

As well as supplying web services, Amazon’s using them internally too.  Peter briefly reviewed how Amazon started as what looked like a 2-tier+web client-server web-application, but then refactored that incrementally (and painfully over 2002-2003) into a collection of services. They’ve seen reliability benefits – he said they can lose an entire data centre with no impact on the customer experience.  They’ve also had product management benefits – each service maintains its own data and operating responsibility, which lets them each evolve at their own pace. Amazon’s key NFPs are security, incremental scalability, availability (systems fail not by stopping, and failures aren’t independent), performance (not just mean performance but also performance in outlying cases), and cost-effectiveness.

Intriguingly, despite the claimed product management benefits, he said that 70% of development time was spent on “undifferentiated heavy lifting” delivering updated services – dealing with non-functional, administrative service management issues.  So only 30% of their effort is spent improving the customer experience. I think their delivered experience could certainly use some extra work, especially for their non-US customers!

The 6th International Conference on Service Oriented Computing is on in Sydney this week. NICTA is a sponsor, and I managed to score a registration to attend.  Ian Foster opened with an interesting keynote. (Preceded by a 30 minute delay fussing with Mac technology issues!)  He spoke on “Services for Science” – how SOA is being used to support knowledge creation in science. Currently there’s a surprisingly strong growth of online services providing data and analysis, in astronomy and especially in the biomedical field.  He talked about the caGrid network. Ontologies are key there for meta-data of experimental results – Ian commented that the community is very “neat” (not scruffy) in being explicit and standardised in the representation and organisation of their data.

It’s interesting that for representing scientific workflow they’ve dropped BPEL in favour of the workflow notation and supporting infrastructure in Taverna. The workflows are used not only to coordinate data and analyses, but also to communicate methods and in principle to promote reuse. But the caGrid leaders recognise that it’s hard to design for workflow reuse, and hard to achieve reuse in practice.  Ian also discussed experimental use of functional programming techniques to support provenance – to capture computations as a first class entity for scientific audit, review, and mining. He finished with some discussion of scalability and text mining of research publications.

I think there are interesting analogues of some of the issues now being explored in the e-science domain that have already been thrashed out in software engineering. They are quite similar in some ways – in the two fields of practice at an industrial scale, there are teams of knowledge workers working on complex and partly-shared electronic assets. Large scale reuse and variation has been made methodical in Software Product Line Engineering, and provenance issues are very similar to those that are well known in the established discipline of (Software) Configuration Management.

COAG met on Saturday and decided to invest to implement a national approach to conveyancing – the National Electronic Conveyancing System (NECS).  Currently each of Australia’s eight states and territories has its own different system for dealing with the transfer of real estate.  You might not think that’s a big deal – after all, wherever you are, the house you buy is only going to be in one state!  Why does it matter to have a uniform national system?

At an abstract level from the public’s point of view, when you buy a house, there’s just a buyer, a seller, and a central land registry that maintains the “golden truth” about ownership under the standard Torrens system of title.  It’s a little more complicated than that because mortgages for housing loans are also registered with land registries.  So banks and non-bank lenders are normally involved too.  It’s more complicated than that, because there’s a whole raft of other auxiliary entities involved in title exchange, such as title search companies, lawyers, property valuers, and insurers providing related services.  The whole industry (banks, non-bank lenders, and the auxiliary service organisations) operate nationally.  Currently they need to implement and maintain systems to deal with the land registry systems in each of the eight states and territories.  In the past conveyancing has been a manual process, and human processors have been able to deal with the inefficiencies of working with multiple interfaces.

However, access to land registries is starting to move online, to reduce the cost and time of buying real-estate.  When conveyancing becomes automated, there’s a large initial cost borne by everyone in the industry to integrate with the new system(s).  Companies would prefer to pay this initial overhead cost once, not eight times!

NECS is intended to address this problem.  The goal is not to create a single national land registry, but instead to create a single national interface to all of the state and territory land registries. Organisations will be able to integrate with the national interface, and gain access to the land registries in every state.

Our group at NICTA has been working with NECS, looking at issues in the definition and management of business vocabularies, business rules, and business processes.  NICTA’s research philosophy is “use-inspired research” – working on fundamental scientific advances and technology innovations in the context of, and with an understanding of, real-world problems.   The goal is to do research that has more impact, and benefits Australia.  Our work with NECS is an example of all of this.   It’s still early days, but having a deep engagement with conveyancing and e-government has already been important to motivate and direct the research we’re doing.