rwandering.net
The blogged wandering of Robert W. Anderson
March 2, 2011 at 5:31 pm · Filed under .NET, Grid Computing
Read the open letter to insideHPC readers.
Pure mudslinging about conflicts of interest based on things that don’t seem to be true. I won’t mention the mudslinger, but I can tell you that if I bothered to read that other blog — which I don’t — I would unsubscribe.
Build your brand on merit like InsideHPC did, not on publicity stunts.
Tags: Digipede, grid, HPC
July 6, 2010 at 10:41 pm · Filed under .NET, Grid Computing
We recently released the Digipede Network 2.4. Among other things, this release provides support for hosting .NET 4 applications, some new features to improve management and control, and enhanced server-side performance. The entire list and downloads are available on the community site. You can read more about it on the Interwebs:
Those paying close attention might ask "what happened to 2.3?" The answer is Digipede trivia.
- Part of a failed experimental branch? No.
- Is 2.4 actually numbered 2.3.1 under the covers? No. (A minor dig at Windows 6 R2).
The actual reason dates back to the days when .NET 2 was released. Back then, we were ready to release Digipede Network 1.1 with .NET 2 support. To avoid naming confusion with .NET 1.1, we decided to skip the “.1” and went straight to “.2”. Was it in fact less confusing? Probably not materialy.
So, why no 2.3? It is an ever so slight (and obscure) homage to those early days: for .NET 4 we decided to release something that ends in “.4”.
Like I said: trivia.
Tags: .NET4.0, Digipede, grid
May 6, 2009 at 9:23 pm · Filed under .NET, Grid Computing
As I noted recently, I have been working with Giles Thomas and Glenn Jones at Resolver Systems on a sample mixing distributed IronPython objects with Resolver One spreadsheets.
I like those guys. They are smart and do excellent work.
Anyway, they released the sample earlier today. From their site:
As of version 1.5 (which is currently in beta), the world’s coolest spreadsheet can use Digipede Network grid computing to distribute and execute workbooks in parallel. The example on the Exchange is based on the excellent IronPython sample created by Robert W. Anderson of Digipede. The Digipede Network is a brilliant way to get distributed, parallel computation on Windows. It only took a few minor changes to convert Resolver One to run on the Digipede Network and to get the IronPython sample to execute Resolver One workbooks.
Giles gives some more background to the path that got us here on his recent post, Resolver One and Digipede.
The combination of our two products offers a pretty elegant solution. Like I said before,
Try doing that with a spreadsheet or grid that isn’t based on .NET . . .
. . . like Excel and Windows HPC Server. No, don’t. Trust me. It is really hard, complex, and brittle.
Tags: Digipede, Excel, grid, HPC, IronPython, ResolverOne
April 17, 2009 at 9:50 am · Filed under .NET, Grid Computing
We had a call this morning with Giles Thomas and Glenn Jones of Resolver Systems. They demonstrated Resolver One running on the Digipede Network.
They used my IronPython Worker sample and customized the front-end Python code, leaving the C# adapter as-is. With very little coding they had an elegant grid-enabled spreadsheet. Try doing that with a spreadsheet or grid that isn’t based on .NET . . .
Giles said they will have support for this in Resolver One 1.5, coming out in the next couple of weeks.
Very cool.
I’ve just installed Resolver One to take a closer look. Already I’m impressed, but I’ll leave that for a future post.
Tags: Digipede, Excel, IronPython, Python, ResolverOne
April 6, 2009 at 6:31 am · Filed under .NET, Grid Computing
. . . or IronPython-ipede (Part II).
I have been playing with IronPython a little. With the release of Digipede Network 2.2, I am now able to post the sample I wrote. It shows how to distribute IronPython objects on the Digipede Network. You can find it on the Digipede community site. See the posting there for details and download instructions.
The sample uses IronPython 2.0.1 and the included version of the Microsoft Dynamic Language Runtime (DLR). While I focused on IronPython in this sample, it would be pretty easy to expand it to support other DLR-based languages.
Comments welcome. I am specifically interested in feedback on DLR integration and initializing ScriptScope objects for each worker thread. It seems that I should be able to do some of this only once at global scope.
By the way, one thing I like about this sample is that it shows how to keep user code completely de-coupled from the Digipede Network while still taking advantage of our deployment and payload distribution model. This has always been supported by the Digipede Network, and this makes a good example.
Tags: .NET, Digipede, DLR, IronPython, Python
March 3, 2009 at 6:46 am · Filed under .NET
Some time ago I took a quick look at Patrick Smacchia’s NDepend. While I was impressed with it, I never took the time to dive into it. Recently, I took another look to see if it could help inform some refactoring of the Digipede Agent.
If you aren’t familiar with it, NDepend is a static analysis tool that allows you to dive deeply into your code base. Its feature list is truly truly impressive. Here are a few things that stand out for me:
- The Code Query Language (CQL) allowing SQL-like queries of your code base, with a set of pre-canned queries targeting code quality, design, naming conventions, and much more.
- A visual tool with dependency graphs and matrices and more
- Ability to compare between different runs of the same project – at an incredible level.
- Of course, A command-line tool for incorporation into your build process.
I loaded up the Digipede Agent assemblies and – kind of like a kid in a candy store – I found myself heading off in 10 different directions at once. My thought process went kind of like this:
- Wow, look at these matrices and graphs, and all these CQL queries.
- This is really cool to have this level of information available with build integration!
- Wow, there sure are a lot of warnings here.
- Let’s fix them! Wait, let’s prioritize them and customize them and, etc.
(The experience reminds me of when I started using FxCop.)
I highly recommend this software . . .
- To help you make informed refactoring decisions; and
- to add design and code-quality criteria (and enforcement) into your build process.
So, why do I say this is a “must-have” tool? Because code quality is not a nice-to-have. Quality reduces maintenance and support costs and allows you to spend your time and money on more profitable endeavors.
Go buy it.
BTW: I would like to post some of the results I’ve gotten with the Digipede Agent, but I’m not ready to share that yet.
Tags: .NET, Digipede, NDepend
February 11, 2009 at 9:02 pm · Filed under Grid Computing, Miscellaneous
I’ll be at the University of California, Berkeley EECS Annual Research Symposium (BEARS 2009) tomorrow, February 12th.
Looks like it will be an interesting program.
If you are going and want to meet up, email me at robert at digipede dot net.
Tags: BEARS2009, Digipede, Events
February 9, 2009 at 11:03 am · Filed under .NET, Grid Computing
I have been taking a closer look at IronPython for a prospective customer. Never being happy with “shoulds”, I am going to show how to distribute IronPython objects on the Digipede Network.
The first thing I did get our old Python sample running in IronPython. This was the first user-contributed sample (thanks to Sean True). That sample (see it here) uses Python COM libraries to invoke a job with the Digipede Network COM APIs. This didn’t submit objects, just executed a command-line application.
I’m happy to say that the code required very little modification to run under IronPython. The only difference is in the syntax of the “import” commands. Kudos to the IronPython team.
I’ll post the working code once I get a little farther.
Next step: distribute IronPython objects. Fairly straightforward, but I’ll write a reusable C# Executive to load the IronPython class definition.
[tags]Digipede, .NET, IronPython[/tags]
Tags: .NET, Digipede, IronPython
July 17, 2008 at 7:14 pm · Filed under Grid Computing, Web 2.0
Version 2.6 of WordPress came out the other day. From the announcement (WordPress › Blog » WordPress 2.6):
Version 2.6 “Tyner,” named for jazz pianist McCoy Tyner, contains a number of new features that make WordPress a more powerful CMS: you can now track changes to every post and page and easily post from wherever you are on the web, plus there are dozens of incremental improvements to the features introduced in version 2.5.
These feature changes are actually pretty big. Revision tracking? Support for Google Gears? Full support of SSL (finally)? Theme previews? Really cool “Press This” button? Big.
This feels to me like a major release. Probably not as major as the 2.5 release, but still pretty major.
In my book, 2.5 should have been version 3.0 and this one should have been 3.5.
Does the version number matter? Yeah, it does. It isn’t just about marketing. It signals something about the maturity of the product.
Disclaimer: I am not immune to such version number mistakes. After all, the Digipede Network 2.1 should have been version 2.5.
Tags: Digipede, Wordpress
June 9, 2008 at 2:13 pm · Filed under .NET, Grid Computing
Last week Microsoft released the first CTP of the Microsoft Distributed Cache (code-named Velocity).
I am definitely excited about this release. While Microsoft is not breaking new ground here, the addition of a distributed cache to .NET is a great addition to the platform. Certainly there are competing technologies, but Velocity will be a very simple choice for developers and ISVs because we’ll be able to count on its availability.
This ISV is interested, so we tried it out.
We have many customers who use our Executive pattern to load and cache job-specific data for compute-intensive jobs on the Digipede Network. These data are often fetched through WS calls or directly from SQL databases. Often this is performed in the Executive.Start
method. Before Velocity, the code might look like this:
protected override void Start() {
// read the CBOData object from the database.
_cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
}
Including Velocity in this example is really easy. The following snippet adds use of the Velocity cache:
protected override void Start() {
// get cache
CacheFactory factory = new CacheFactory();
Cache cache = factory.GetCache("CBOCache");
// see if our CBOData object is already there
string key = JobTemplate.Parameters["CBODataKey"].Value;
_cboData = (CBOData)cache.Get(key);
// if not, read it from the database.
if (_cboData == null) {
_cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
// store it in the cache for later use
cache.Put(key, _cboData);
}
}
With a few lines of code, we reduce the load on the database server and network and spend more time computing. (I’m making an assumption with this simple code that all Executives don’t start at once, an assumption made obsolete by seeding the cache from a master application).
Of course, this is a simple example, but there are many other use cases. For example,:
- Digipede-enabled applications can share results;
- master applications can load the cache with job-specific data; and,
- others where baking Velocity deeply into the Digipede Network start looking pretty interesting.
I have seen many posts on “must-haves” for a Velocity RTM. I mostly agree with the lists I have seen. I’ll have a list too mostly from the ISV perspective.
Cool stuff.
[tags]Digipede, .NET, Velocity, ISV[/tags]
Tags: .NET, Digipede, ISV, Velocity
Next entries »