THE EXTRA PROJECT
Detailed Project Outline
Executive Summary
The EXTRA project aims to put in place a highly scalable general-purpose GRID processing infrastructure in place to enable several related projects that will require a high degree of computational throughput.
Related projects include:
- WALL: Provisioning multi-channel streaming multimedia
- SOLO: Enabling immersive visual environments
- PIKE: Delivery of three dimensional (3D) virtual environments
- DOME: Developing ultra-scale data projections
From Wikipedia:
Grid Computing is a form of distributed computing whereby a super virtual computer is composed of many networked loosely coupled computers acting together to perform very large tasks. Furthermore, distributed or grid computing, in general, is a special type of parallel computing that relies on complete computers (with onboard CPUs, storage, power supplies, network interfaces, etc.) connected to a network (private, public or the Internet) by a conventional network interface, such as Ethernet. This is in contrast to the traditional notion of a supercomputer, which has many processors connected by a local high-speed computer bus.... (full article)
Applications
- Multimedia
- Computer Graphics
- Computer Music
Science
- Ubuntu Linux
- Blender
- Loki
- Java
The Build:
The Blog:
2012-01-01:
I am in the process of putting together a couple of Blender Clusters at the Multimedia Lab (M-Lab) connected to the School of Design at The Polytechnic University of Hong Kong (POLYU).
Right now we are playing with two clusters.
The small cluster is composed of four machines. It is being constructed and documented to help us understand how to get a basic Blender Cluster going. Why four? Because it is physically small enough to work on without too much walking around and because that's what's available at the moment.
The large cluster is going to be composed of a lot more machines. At the moment, it is composed of eight machines and is being configured as a video wall running MIT's PROCESSING language.
The large cluster project is called WALL:
http://www.graham-leach.com/WALL.php
I have rounded all of the required hardware from extra gear hanging around the lab: Computers, switches, power bars, cabling, etc...so far both clusters are physically implemented and both are running just fine. By this I mean that there were absolutely no problems with the physical installation of the machines except for a couple of very small and easily rectified incidents (mostly bad cables)
The smaller cluster was the focus today. I installed UBUNTU 11 on the constituent machines. I did this by downloading the UBUNTU 11 ISO directly from the UBUNTU website. Then I used a free tool called CDBURNERXP to create a bootable DVD from the .iso file. Yes, the CDBURNERXP tool burns DVD as well as CD. The smaller cluster was the focus today. I installed UBUNTU 11 on the constituent machines. I did this by downloading the UBUNTU 11 ISO directly from the UBUNTU website. Then I used a free tool called CDBURNERXP to create a bootable DVD from the .iso file. Yes, the CDBURNERXP tool burns DVD as well as CD.
The first DVD did not work right. The media was burned at too high a speed and the installation of the first machine failed with a media error. I didn't even try to diagnose the problem, I just burned another DVD at a slower speed. The second DVD was a success.
The UBUNTU 11 install went very smoothly with no real problems whatsoever except for the fact that the machines often needed a hard reset once the installation script had finished. It was a very rare event that any of the machines actually rebooted properly onDuring the install, I didn't bother to do anything but a really, really minimal installation. Once the machines had rebooted and loaded the UBUNTU desktop, I simply started a terminal session and executed the following instruction to update the machine
Code:
sudo apt-get update
Once each of the machines had updated itself, I moved on. Later I may need to revisit the boxes to make sure I can SSH into them, but I'll worry about that later once it's been determined that they are inaccessible. The cluster is so small I just have a monitor + keyboard + mouse bundle that I swap from machine to machine if I want to get at the console.
I personally welcome any hints, tips, tricks, HOWTO (no matter how old) that others who have already walked this path may have to offer to help smooth my way and those who will follow me.
I will also be documenting my experience at www.graham-leach.com, just in case this venue ever ceases to exist.
2012-01-02:
Installed Loki on the cluster. A very straightforward install and easily done. It took about an hour to install all of the necessary software to be able to run a clustered Blender implementation.
The only hangup is that starting up the Loki clients requires console access, but as soon as they are up and running they don't really need console access and can basically run headless from that point onward.
The process was so simple, I'm almost embarrassed to describe the steps required.
Here they are:
- Install UBUNTU (I used a bootable DVD) - Update UBUNTU #sudo apt-get update - Assign STATIC IP (I used the network applet within UBUNTU desktop) - Install SSH SERVER (useful later) #sudo apt-get install ssh-server - Install JAVA RUNTIME ENVIRONMENT #sudo apt-get install default-jre - Install BLENDER #sudo apt-get install blender - Download LOKI I a local copy, click here - Install LOKI #unzip lokiRender_062.zip - Run Loki #java -jar lokiRender_062.jar)
At that point I connected the machines together (all as grunts, one as master) then downloaded some test BLENDER files and started rendering.
I think that if I was asked to do this again I would be able to put a cluster together in a half day or short afternoon.
If anyone was hesitating about getting a BLUSTER together because of a perceived sense of complexity...hesitate no more!
2012-04-09:
The EXTRA project has now been expanded to TWELVE (12) machines, which should give the project a serious amount of firepower to produce computer animations using Blender.
