9P2000 protocol Erlang extension v1.0
————————————-

Notation

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, “OPTIONAL” in this document should be interpreted as described in [1]RFC2119.

Introduction

Erlang on Xen makes extensive use of 9p protocol for a multitude of tasks, including code loading, storage access, node monitoring, message passing, etc. In most cases, the standard semantics of the protocol is enough. However, in a few cases limitations of the protocol gets in the way.

Dropped transport connections

9p connections are tightly coupled to the underlying transport (TCP) connections. The loss of TCP connection — a frequent occurence during instance migration — means that all Fids are lost.

Simple operations too chatty

A simple operation, such as writing “0” to a synthetic file, require multiple network roundtrips: walk to file, open Fid, write data, clunk Fid. This makes many administrative tasks noticably slow.

The 9p protocol extension — 9P2000.e — is introduced to address these two issues. Erlang on Xen uses this protocol version for internode communications.

Overview

9P2000.e is the extension of 9P2000 protocol [2]9P2000. It adds several new protocol operations as described below. Semantics of standard protocol operations are left unchanged.

A new operation — session — reestablishes a session upon after reconnecting a transport. All Fids are preserved in the reestablished session.

Also the protocol extension adds a few new operations that act as macro-operations of frequently used sequences.

The server that implements 9P2000.e should fall back gracefully to use 9P2000 protocol by disabling the newly introduced operations.

New messages

size[4] Tsession tag[2] key[8]
size[4] Rsession tag[2] 
size[4] Tsread tag[2] fid[4] nwname[2] nwname*(wname[s])
size[4] Rsread tag[2] count[4] data[count]
size[4] Tswrite tag[2] fid[4] nwname[2] nwname*(wname[s]) count[4] data[count]
size[4] Rswrite tag[2] count[4]

The proposed numeric values for the new commands are as follows:

New operations

session – reestablish a session

size[4] Tsession tag[2] key[8]
size[4] Rsession tag[2] 

When a client performs authentication it may establish a session key. If transport connection is lost and reconnected the client may decide to use the session message to request reestablishing of the session. A successful reply means that the session is reestablished and all connection Fids are still valid.

An error reply means that the session can not be reestablished. The client may decide to continue, treating the connection as completely new.

The session message must be the first message must be the first message that follows a successful version negotiation. The tag of the session message must be set to NOTAG (~0). A rerror message is returned if the session can not be recovered.

9P2000.e protocol uses MUMBLE messages for authentication [3]mumble. A MUMBLE message contains the session key for the new session.

sread – read entire file contents

size[4] Tsread tag[2] fid[4] nwname[2] nwname*(wname[s])
size[4] Rsread tag[2] count[4] data[count]

The operation reads the entire contents of a file. A file is identifier by walking to a series of wnames starting with fid. The operation combines walking to a file, opening it, reading its content, and clunking a fid. The new operation minimizes network roundtrips when reading small files.

swrite – overwrite file contents

size[4] Tswrite tag[2] fid[4] nwname[2] nwname*(wname[s]) count[4] data[count]
size[4] Rswrite tag[2] count[4]

The operation overwrites the file contents with data. The file is created if it does not exist.

Discussion

The session key should be kept secret as knowing it may allow hijacking of a 9p connection.

The Erlang extension does not change the semantics of the standard 9P2000 operations. This should facilitate a graceful fallback to plain 9P2000 protocol if the server does not support the extension.