The massive exodus to the cloud left many well-proven approaches waiting for a retirement notice. An obvious example is disk-backed filesystems.
Do we need a full-featured filesystem on every virtual node? Hardly. A virtual block device accessible by a virtual instance is much more than a handy imitation of a hard disk. The cloud storage subsystem usually keeps multiple copies of data and automatically recovers from failures. Do we need yet another layer of redundancy inside the instance then? The virtual block device may be backed by a spinning media, a flash memory, or SAN. Does it make sense to make spatial optimisations of the device’s filesystem?
A typical usage scenario for a filesystem is changing too. Often we find all important data kept in a database mapped to a small number of files. The database has its own transactions, journaling, and locks. Implementation of these primitives on the filesystem level only slows things down. A database inside a cloud instance should talk directly to a virtual block device. The standard Erlang databases – mnesia/dets – can not (yet) do this. As an interim solution we have added GooFS.
GooFS is a minimalistic filesystem for a virtual block device for hosting databases inside Erlang on Xen instances. Unlike most modern disk filesystems, GooFS is flat, no directories. Note, that Amazon S3 is flat too.
GooFS allows a small number (254, by default) of files and allocates space to files in sizable (4Mb+) increments. Metadata updates are atomic. Thus, shutting down the instance can not corrupt the filesystem. Overall, GooFS resembles an old-school filesystem for an oversized floppy disk. The detailed description of the layout of GooFS volume can be found here.
GooFS is already included into the version used by the build service. Just configure a block device per instructions and add “-goofs /where/to/mount” to the command line. The virtual block device will be GooFS-formatted upon the first use and you will get a writable directory under /where/to/mount perfectly suitable for keeping database files.
Of course, a GooFS volume can be mounted from outside the instance using 9p protocol.
Tweet comments powered by Disqus