Linux and clustering

[The Rock]

Anyone here know anything about setting up 2 different machines in a load balanced cluster under Linux?

[Ribbit]

I don't, but go and look up MOSIX and see if that's what you want.

[Marshmallowman]

Are you using a specific distro made for clusering or just a vanilla linux?.

[The Rock]

I haven't done anything yet, and I'm open to suggestions on what distro to use (I don't care which as long as it works. My friend is setting up a dedicated game server on a low-end Athlon; I thought that after my next CPU/RAM/board upgrade, I could build another PC around my old guts and somehow add this to the game server via clustering.

[Marshmallowman]

mosix is worth looking at

http://www.mosix.org/

I was going to say mandrake had clustering distro...but I can't find any details...anyway the mandrake one is called CLIC, but it may be an enerprise thing..

I have not done any clustering...etc but I would be interested in finding out what you turn up..

[The Rock]

I know it won't be easy. From what I've seen on clustering from a big server enterprise level, its VERY complex. On some flavours of Unix, its easier than others. HPUX for example is MUCH easier to cluster with their software (MC ServiceGuard I think?) than Solaris is (with SunCluster software). If setting up a Linux cluster is anywhere near comparable to Solaris clustering, I wouldn't even try it. So I'm guessing that this *might* not be worth the trouble to set up, but more research is in order before I make that call. But what a learning experience it would be. Decisions decsions. And now that I just upgraded to a P4 2.6/800, I have this 1.8G P4 + 512DDR just waiting to have something done with it.

[thop]

Clustering is not really worth it unless you have special custom applications designed around it.

[albatorsk]

Originally posted by thop
Clustering is not really worth it unless you have special custom applications designed around it.

I beg to differ. MOSIX really is "fork and forget", from the users point of view. It's really easy to set up (takes about 5 minutes if you read, understand and follow the on-screen installation instructions), and it's a breeze to work/play with. It even has its own shared filesystem (MFS) for easy migration of tasks which are set up to crunch numbers in/for a separate file.

For example, let's say you have 3 MOSIX-nodes running, and you wish to use mencoder to convert three MPEG1-streams lying on node 1 to AVI/MPEG4 and save them on node 2, it's as easy as running this:

runon 1 mencoder -o /mfs/2/home/thop/avis/movie1.avi -oac mp3lame -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate=1200 /mfs/1/home/thop/mpegs/movie1.mpg
runon 2 mencoder -o /mfs/2/home/thop/avis/movie2.avi -oac mp3lame -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate=1200 /mfs/1/home/thop/mpegs/movie2.mpg
runon 3 mencoder -o /mfs/2/home/thop/avis/movie3.avi -oac mp3lame -ovc lavc -lavcopts vcodec=mpeg4:vhq:vbitrate=1200 /mfs/1/home/thop/mpegs/movie3.mpg

With a minimum of fuss, all three nodes are now being used to their fullest extent. MOSIX can/will (depends on how you set it up) also auto-migrate proceses between nodes, so if you decide to run all three instances of mencoder on a single node, depending on your likings, MOSIX will distribute them evenly across the cluster.

Now, in all fairness, it will not migrate threads, so multi-threaded applications has no use of MOSIX, so it's not the end all be all of clustering, but it's a great tool if you want to connect your computers quickly.

[linuxgeek20]

Originally posted by Marshmallowman
mosix is worth looking at

http://www.mosix.org/

I was going to say mandrake had clustering distro...but I can't find any details...anyway the mandrake one is called CLIC, but it may be an enerprise thing..

I have not done any clustering...etc but I would be interested in finding out what you turn up..

I am in the process of installing CLIC. However, I have been experiencing difficulties (mainly because of hardware incompatibilities).
As you stated, it is very difficult to find any documentation regarding this software; there isn't much info on http://clic.mandrakesoft.com/index-en.html. Does anyone know of any other online resources that may aid in installation?
Also, has anyone had success installing it?

[linuxgeek20]

, does anyone know any clustering software that would be suitable for a High Availability cluster?

[spadnos]

Carrier Grade Linux (if there's ever a distribution that meets the requirements)

- Steve

[The Rock]

Holy dead thread resurrection Batman!

[spadnos]

It was only "mostly dead"

- Steve