Expert Texture
The blogged wandering of Robert W. Anderson
Archive for Grid Computing
July 3, 2008 at 4:09 pm · Filed under Grid Computing, Web 2.0
I have found myself talking about cloud services a lot recently. We have been talking about them here — there is an obvious synergy between what we do at Digipede and cloud services. And I’ve been talking about them externally too: at the recent CloudCamp, on the Gillmor Gang, and in all sorts of other interesting contexts.
Note that I refer to cloud services, not to the cloud. I am not interested in defining cloud as a term, because I don’t think it very useful. For those of us in the distributed computing space, cloud is the latest buzzword to compete with the word grid in terms of utter ambiguity. I think the ship has already sailed on this one and I’m not going to try to call it back.
So, everyone is talking about cloud services and much of the conversation centers on understanding them and how they are changing the landscape. Of course, cloud services are not one thing. I find it helpful to think about them as parts of a continuum. This seems useful regardless of the technical level of the people with whom I’m speaking.
The diagram to the right shows this continuum from infrastructure to platform to software. Brief definitions of these parts are:
- Infrastructure includes provisioning of hardware or virtual computers on which one generally has control over the OS; therefore allowing the execution of arbitrary software.
- Platform indicates a higher-level environment for which developers write custom applications. Generally the developer is accepting some restrictions on the type of software they can write in exchange for built-in application scalability.
- Software (as a Service) indicates special-purpose software made available through the Internet.
I have indicated several companies that play at different parts of this stack. This list is not comprehensive nor does it attempt to represent motion across the stack.
One scenario in which I find myself talking about the continuum is when people equate Amazon EC2 with Google App Engine. EC2 is a flexible / scalable virtual hosting platform with provisioning APIs. It allows you to dynamically scale the number of instances of your OS (i.e., Linux). What you do with those instances is up to you. Google App Engine operates at a much higher level in the stack. It is a new software platform with specific APIs. It requires developers to build for this specific platform. yes, they are both in the cloud, but they are very different services.
Another scenario in which the continuum is useful is in thinking about what vendors and new entrants might be up to. The continuum makes one thing even more clear: many vendors that operate higher in the stack are relying on their own internal lower-level infrastructure or platform. This begs some questions: which vendors will expose lower-level interfaces? And of course, which vendors will move up the stack?
- SalesForce is already moving down with their PaaS offering.
- Any chance Google will expose its infrastructure stack? I doubt it, but I do expect them to move down a little.
- Some of the readers of this blog probably know better than I where Amazon and Microsoft are planning to go.
Yet another way it is useful is in comparing vendors inside of a particular category. Maybe I’ll write more on that later.
Is the continuum obvious? Using the definition of obvious from patent law, yes, but I think it a useful paradigm.
Tags: Amazon, cloud, Google, IaaS, Microsoft, PaaS, SaaS, Salesforce
June 9, 2008 at 2:13 pm · Filed under .NET, Grid Computing
Last week Microsoft released the first CTP of the Microsoft Distributed Cache (code-named Velocity). 
I am definitely excited about this release. While Microsoft is not breaking new ground here, the addition of a distributed cache to .NET is a great addition to the platform. Certainly there are competing technologies, but Velocity will be a very simple choice for developers and ISVs because we’ll be able to count on its availability.
This ISV is interested, so we tried it out.
We have many customers who use our Executive pattern to load and cache job-specific data for compute-intensive jobs on the Digipede Network. These data are often fetched through WS calls or directly from SQL databases. Often this is performed in the Executive.Start method. Before Velocity, the code might look like this:
protected override void Start() {
// read the CBOData object from the database.
_cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
}
Including Velocity in this example is really easy. The following snippet adds use of the Velocity cache:
protected override void Start() {
// get cache
CacheFactory factory = new CacheFactory();
Cache cache = factory.GetCache("CBOCache");
// see if our CBOData object is already there
string key = JobTemplate.Parameters["CBODataKey"].Value;
_cboData = (CBOData)cache.Get(key);
// if not, read it from the database.
if (_cboData == null) {
_cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
// store it in the cache for later use
cache.Put(key, _cboData);
}
}
With a few lines of code, we reduce the load on the database server and network and spend more time computing. (I’m making an assumption with this simple code that all Executives don’t start at once, an assumption made obsolete by seeding the cache from a master application).
Of course, this is a simple example, but there are many other use cases. For example,:
- Digipede-enabled applications can share results;
- master applications can load the cache with job-specific data; and,
- others where baking Velocity deeply into the Digipede Network start looking pretty interesting.
I have seen many posts on “must-haves” for a Velocity RTM. I mostly agree with the lists I have seen. I’ll have a list too mostly from the ISV perspective.
Cool stuff.
Tags: .NET, Digipede, ISV, Velocity
May 9, 2008 at 6:18 am · Filed under Grid Computing
Dan has a write up of some of the enhancements added to this release here. He said we should have probably called it 3.0, but it is really more of a 2.5. We’ll be hosting webcasts soon going over the new features.
Thanks to the team for all the hard work in getting this out the door.
Follow http://twitter.com/010111011010111 for Digipede announcements.
Tags: .NET, Digipede, grid
February 26, 2008 at 12:42 pm · Filed under .NET, Grid Computing
Heroes who happen by our booth at the Server 2008, Visual Studio 2008, and SQL Server 2008 launch will get a chance to win an XBOX-360. OK, you don’t have to be hero, but you do have to be spotted wearing a Digipede sticker sporting our mascot, Deatle.
Come on by and see us.
BTW: I won’t be at this event, but I’ll be at the one in SF on March 13th. No Digipede booth or give-away there.
Tags: Deatle, Digipede, Launch, Microsoft, Server 2008, XBOX
February 20, 2008 at 6:39 pm · Filed under .NET, Grid Computing
Last week, PowerShell Architect Jeffrey Snover wrote an excellent post titled the Semantic Gap. He writes about the gap as . . .
. . . 2 worlds:
- The world as we think about it.
- The world as we can manipulate it.
The difference between these two is what is called the semantic gap.
This is a great working definition.
Jeff writes about this specifically regarding PowerShell and instrumentation providers and asks the question,
So why do instrumentation providers close or not close the semantic gap?
Yes, some do, and some don’t. This isn’t just about hierarchy of needs, but also about prioritization. How important to the provider is a narrow semantic gap for product X when used through interface Y?
In the case of X := Digipede Network and Y:= PowerShell, we thought it pretty important.
But how do you decide if narrowing the gap is worth it? Engineering costs aside, understanding what your interface could look like in PowerShell can help you decide. Internally, we answered these questions:
- What would a PowerShell script look like just using your .NET or COM APIs?
- What could it look like with Cmdlets?
- Would these Cmdlets support how we think about the Digipede Network (i.e., small gap?).
I already said the answer to #3 turned out to be yes and in a previous post, I gave an example of the gap in Why a SnapIn for the Command-Line? This example highlights the gap for a common operation on the Digipede Network: get the description of a pool of resources.
If you are thinking about supporting PowerShell in your product, take a look at my post.
I hope this helps you decide.
Tags: .NET, c#, Cmdlet, Digipede, PowerShell
February 4, 2008 at 11:36 am · Filed under .NET, Grid Computing
Today, we announced a program to provide free licenses of the award-winning Digipede Network to Microsoft MVPs. For more details and to request your license, go here.
Thanks to MVP Marco Shaw for the idea.
Tags: Digipede, Microsoft, MVP
January 23, 2008 at 7:10 am · Filed under .NET, Grid Computing
We just received our certification for Windows Server 2008. Or we are about to — it probably isn’t “official” yet. Anyway, congratulations to the Digipede team and thanks to everyone at Microsoft and Veritest who helped us through the process.
Getting the logo was arduous. This has less to do with the technical logo requirements and more to do with the complexities of the process itself. Some of the complexity is inherent in such a process, but much was due to the program itself being a sort of “work in process”. But hey, that’s why we early certifiers got the testing fees waived. I think those who begin the process now will find the test requirements and tools are better written and more robust.
As I said, passing the technical requirements was not arduous for us (we were already very close), but passing the tests did require some minor improvements to the Digipede Network that were motivated by the test:
- Support for User Account Control (UAC).
- More useful logging on the Digipede Server and during installations.
- Improved user messaging and event logging during error conditions between server components and the database.
- Improvements to the Installation Guide including new sections on Custom Actions, installation artifacts, and more.
Some of these changes have already made their way into the shipping product, though others won’t be available until the Digipede Network 2.1 (which, while a minor upgrade, contains many features beyond the improvements mentioned above — I think the feature set will be announced soon).
So now we’re ready for the big launch of Server 2008, Visual Studio 2008, and SQL Server 2008 in Los Angeles on February 27th. If you are going to be there, come see us at the Partner Pavilion. I’m pushing for some kind of Digipede swag — but I’m not in marketing ;).
Tags: Certified, Digipede, Server 2008, sql, VS2008
November 17, 2007 at 6:33 pm · Filed under .NET, Grid Computing, Web 2.0
John has an excellent post wrapping up his trip to SC ‘07 — the bashers’ ball. He is tired of all the Microsoft bashing:
It is amazing to me the level of religious fervor that Microsoft still inspires. The bashers out there can be perfectly calm and reasonable about a wide range of topics – but say the word “Microsoft,” and they turn bright red and irrational. I have watched this phenomenon for years, and still find it inexplicable. Microsoft is a company. That company makes software. Some of their software is very, very good. Some of it is remarkably bad. I don’t understand why some people find it so hard to remain objective (or even civil) when discussing their products and market presence.
Statement of fact. Nothing new exactly, but then he goes on how this relates to our company (emphasis mine) . . .
Many Microsoft bashers think that all of us at Digipede are mouthpieces for the Evil Empire, and that we are just pawns of the Microsoft machine. On the other hand, while we have plenty of fans within Microsoft, there are also some Microsoft employees who think we are difficult annoying troublemakers . . .
Pawns of Microsoft? Please. But maybe we are misunderstood. The other day I found myself convincing Steve Gillmor that I’m not a Microsoft fanboy (I don’t think I succeeded). But John says it well,
In fact, none of us at Digipede love or hate Microsoft – we work with Microsoft. We do so for real-world business reasons that help us change the world for the better while building a great company. We work with other companies too, but Microsoft occupies a special place in the technology landscape, and we work very, very hard to understand how to work with them to our mutual benefit. There are some great people there doing great things, and the bashers only hurt themselves by blinding themselves to these very real contributions.
Yeah, I don’t love or hate Microsoft; however, I do really like Microsoft .NET. Does that make me a pawn of Microsoft? .NET isn’t my religion. I’m not a zealot about it. It doesn’t mean that I think everyone should be using .NET / Windows nor does it require that I go around bashing Apple / Sun / IBM / Google / Linux / Java / PHP / Rails / whatever.
I don’t think Microsoft bashing is a requirement for entry to the HPC and Apple fan clubs either, is it?
As long as I can remember, I have detested the religion of the OS (or programming language, or platform, etc.). Passion for technology is great — it’s a requirement for success in this field. But I’m tired of people using their passion to bash, bash, bash.
Maybe I just don’t get it, but if you’re a basher, please just move along. And if you still think I’m a Microsoft pawn, well, I’m not going to convince you, now am I?
Note to Robert Scoble: you had the temerity to criticize Apple and the zealots came out in force — even called you a Microsoft shill. Welcome back to our club, Robert — though I’m not sure you ever really left.
Tags: .NET, Apple, Digipede, Gillmor, HPC, Microsoft, SC07, Scoble
November 13, 2007 at 11:08 am · Filed under Grid Computing
Two exciting announcements at SC07 came out of Microsoft today. I wouldn’t normally lump them together into one post (because they are only peripherally related), but Microsoft announced them in the same press release here. None of this is news to those who follow Microsoft closely, but it is worth a mention.
- Microsoft announced version 2 of Compute Cluster Server, renamed Microsoft Windows HPC Server 2008. Ever since I saw a preview of this a couple months back, I have been excited about some of the enhancements they’ve made in terms of manageability and interactivity. Mostly, I’m excited about new interop scenarios between the Digipede Network and HPC Server. As some resources become available, I plan to do an internal proof of my idea — I hope I’ll have more on this soon.
- Microsoft announced their Parallel Computing Initiative with the mission of enhancing developer productivity for multicore and distributed systems. Most exciting (to me) is the announcement of their Parallel Extensions to the .NET Framework. These extensions include mechanisms for expressing parallelism inside managed code. One thing that will be built on top of these extensions is P-LINQ. Expect previews of this technology will begin to come out over the next six months. Why am I excited about this? Because the more tools that are made available to .NET developers to express parallelism, the easier it will be for the Digipede Network to fit in and managing varying workloads across a large number of disparate computing resources.
Note to Wagg-Ed: there is no such thing as Visual Studio 2007.
Tags: Digipede, HPC, Microsoft, P-LINQ, PFX, Wagg-Ed
October 30, 2007 at 12:22 pm · Filed under .NET, Grid Computing, Miscellaneous
Today I was in a conference call doing a demonstration of the Digipede Network.
One of the demos includes using Excel as a front-end to a distributed code-behind calculation. A common follow on question was asked (heavily paraphrased),
what about code trapped in Excel workbooks?
The answer is, yes, we can (and have several customers that do) distribute Excel workbooks for distributed execution. Our software moves the workbooks, launches Excel through COM, executes calcs in the workbook, saves them, and brings back the workbook.
The prospect says,
That is both commendable and horrifying at the same time.
Heh.
As is most of the work we’ve done to provide stable, registration-free COM contract for the Digipede COM users.
Tags: .NET, COM, Digipede, Excel, QOTD
Next entries »
Main feed
Email subscriptions
