It was whilst working on an exceptionally complex piece of distributed code today, that I realised that such clues may indeed be all around us; yet without inertia, they appear random.

Clue 1: Dining Philosophers

One of the most famous examples of distributed computing logic can be found in the Dining Philosophers problem, which illustrates the need for cooperation if the issues of starvation and deadlock are to be avoided.

There are obvious parallels between the core concept and the real world, and so compelling are these parallels that it is easy to overlook that the philosophers are eating spaghetti.

Clue 2: The Spaghettified Non-conceivability Ceiling

In massively distributed computing, there are myriad global interrelationships between the complex tendrils of code & data and their mutations through time and space.

These start to feel like a spaghetti of information that cannot be held in a single brain, and the phenomenon is known as the Spaghettified Non-conceivability Ceiling (SNcC) . It is reached when a system becomes so complex that it can no longer be completely understood (and therefore developed) by a single human.

Clue 3: Physical Non-conceivability

For the last few years, scientists worldwide have been working towards Grid computing, and ultimately a Grid-of-Grids which combines all machines into a grand unified system: A system so complex that no single human can behold more than the smallest part of it.

A pure physical manifestation of the Spaghettified Non-conceivability Ceiling (SNcC).

Clue 4: da Vinci and The Second Coming?

It is common in religion to describe how a deity is at one with it’s creation, omni-present and omniscient; and, bringing this article around full circle, it was da Vinci‘s friend (and fellow Italian) Michelangelo who painted the ceiling of the Sistine Chapel where man and deity touch in the “Creation of Adam” fresco; where a small spark can be seen connecting them. Man connecting to deity through an electrified channel.

Putting the pieces together it dawned on me that the Flying Spaghetti Monster may be preparing to take physical form on Earth, manifested as The Grid of all mankind’s knowledge and computation. Present in every home, in evey office, everywhere; seeing and knowing everything, directing us, controlling us.

Are scientists building a deity?

Now that you are aware of these clues, others may reveal themselves to you; so be alert to His mysterious noodly machinations, and be sure to share any divine insight that may be bestowed upon you.

We are now able, using the biggest, fastest supercomputers in the world, to recreate the whole of cosmic history.

The problem here is that Professor Frenk has been literally quoted. Without the context of the surrounding article in Nature magazine, it sounds like the team has recreated all of the Universe’s history in a very big computer.

Unless you’re a cosmologist, or a computing person, or a skeptic, you could be forgiven for thinking that they’ve done exactly that, such is the myth that surrounds computers, and the things that “they” can do with them.

“They” is “we”, and “we” can’t do that.

What Professor Frenk didn’t say (but what I infer when I have my scientist hat on) is:

We have a model that describes certain aspects of cosmic history. It is by no means a complete model, but one which can be used for virtual experiments. It enables us to check certain theories against what we observe in our own Universe. To run these experiments we are using the biggest, fastest supercomputers in the world because if we didn’t, human kind would be extinct before we got the result; even so, the computers are still way too slow and too small.

That’s the science bit. That’s the bit Richard Feynman succinctly described as follows:

If it disagrees with experiment it is wrong.

Frenk (et al.) are running an experiment to see if the outcome agrees with what we can all see when we look at the sky.

For science to be popular it doesn’t have to be smoke, mirrors and awe. It’s all about keeping it simple. Simplicity is the enemy of misconception.

This is a bad thing.

A similar problem was seen in the UK several years ago when British Telecom’s mobile network (BT Cellnet) bought up all the stocks of the Nokia 7110 that the manufacturer could supply. BT then launched an ill advised advertising campaign with the killer slogan “Surf the net, surf the BT Cellnet.”. The upshot of this was that in early 1999 there were tens of thousands of new BT Cellnet customers who were expecting the Wireless Access Protocol (WAP) to give them the kind of internet access on the move which only characters from the Matrix trilogy could aspire to.

The result was wholesale derision from the user community, and the WAP protocol all but died. Crucially it was not the fault of the protocol. The gulf between the marketing and the reality was insurmountable. The advert showed an ice-man-esque digital surfer, surrounded by cyber-data augmenting his very existence. The reality was a phone with a small monochrome screen and a slow processor, which meant that even thought the WML pages of only 1.5K could be loaded relatively quickly, they still took several seconds to render because the phone was just not good enough. During this rendering time, people were dialled up, and therefore paying, and therefore angry at BT Cellnet because the result was slower and less impressive than could have been expected from the perception they’d formed from the marketing. It’s a mistake that O2 (the rebranded BT Cellnet) have been careful not to repeat.

So we arrive in mid 2005 and everybody who wants investment from anybody has to get the word Grid in their pitch: and frankly, right now, and for the next 10 years, anyone who claims their systems offer Grid Computing needs their brain-oil changing.

The basic concepts of what grid computing might need in order to be realized are largely understood, and within this framework, protocols and systems are being proposed, trialled, revised, discarded, extended, etc. However, there are not just technical and conceptual issues to be solved, there are also moral dilemmas such as, “can ‘x’ be trusted not to use The Grid’s immense computing power to mount a cryptographic attack on ‘y’…”.

Grid computing is essentially the most complicated form of computing because it adds several layers on top of every other system already in existence – consequently it will be many years before commercial grids become available.

Even then there’s still no ocean of computing power that can be tapped to solve every problem. One of the Keynote talks at last year’s Cluster Computing and Grid Conference (CCGrid) in Chicago highlighted that to accurately simulate the digestive system of a single ant, at the atomic level, it would require more computing power than the entire planet combined.

So what’s happened so far with Grid Computing is that a few research systems and protocols have grown with more success than others, through both good judgement and good luck. Concurrently, some loosely coupled clustering systems have emerged that provide a near-term win in terms of an apparent abundance of computing capability, but which are not to be confused with Grid Computing, regardless of what their marketing tells you.

Please, don’t believe the hype, and don’t perpetuate the hype.

Grid computing has some very specific and interesting scientific and entertainment uses, but ultimately, there can not, and will not ever, be enough computing power to solve all problems or model all systems. Machines will get bigger and will work together more closely and will solve larger problems, but that is all; for insight into these realities of grid computing, this book provides the technical starting blocks.

The book is accompanied by a website which is still in the process of being set up.

The book is available from Amazon in the following countries:

• UK
• USA
• Germany
• Japan
• France
• Canada

the channel is currently quite sparsely populated but as more people learn & lurk this will no doubt change. all are welcome.

it’s getting closer though. of most interest to me at the moment is the Java Web Services Core which provides WSRF capabilities, and as a corollary to that the Java CoG Kit 2.0 which happily has a fledgling manual already.

note: although technically the 2.0 cog kit – it will be released as version 4.0

This is all about modelling state in web services, which are by default, stateless. WSRF add

OGSI is dead, long live WSRF

The WebServices-ResourceFramework (WSRF) is a refactoring of the capabilities of OGSI, to take account of newer standards such as WebServices-Addressing

WSRF & WSNotification specs from HP

In Globus, a Resource object implements the ResourceProperties interface which provides a getResourcePropertiesSet method. This set, in turn, contains objects that implement ResourceProperty.

This 3.9.2 design doc is pretty helpful in explaining how the WSRF core fralls together.

When they’re written Information Service Documentation for GT4 will be linked from this large table.

Interestingly it says in the latest release notes that that

This release will aggregate Resource Properties from registered resources and publish in service group form.This release does not contain clients for registering or accessing indexes. You can find example code that demonstrates use of the index in the index package at: src/org/globus/mds/index/impl/test1.java

Hmmm the GT4 fact sheet says that “Support for Xindice and aggregated data persistence has been removed.”

This is nice Globus Toolkit 3.9.2 Development Documentation: WS Java Core

the old stuff is broken

Since GT 3.2 the API is mostly new, and the XML schemas in use are all new too (OASIS WSRF and WSN w.d. spec + WS-Addressing from Mar 2004