General question regarding worker node hardware config

[ptanna]

I was wondering if a worker node with two drives RAIDed (striped) would improve the performance of Qube?  Is there much R/W to the worker node's HD for caching purposes at all?

[jburk]

There's no intrinsic caching performed by Qube, so you won't get any performance benefit out of it.

If you have a large farm (500+ subjobs running simultaneously), you may want to build a faster filesystem for the supervisor and keep the MySQL databases on it.  Just don't try to store the MySQL databases on a network-mounted filesystem, and don't use an iSCSI target.

iSCSI can support high-throughput on large reads (big files, video streams, etc), but the latency and system overhead will kill a busy MySQL server whose databases reside on the other end of an iSCSI target.  MySQL is all about many, many very, very small reads and writes, something network and iSCSI protocols don't handle very well.

However, a 4-drive RAID10 on SAS would do very well at this.  You don't need a lot of storage space, you need more and faster spindles.