The application/mbox Media Type
draft-hall-mime-app-mbox-04

INTERNET-DRAFT Eric A. Hall 
 Document: draft-hall-mime-app-mbox-04.txt February 2005 
 Expires: August, 2005 
 Category: Standards-Track 
 
 
 The APPLICATION/MBOX Media-Type 
 
 Status of this Memo 
 
 By submitting this Internet-Draft, I certify that any applicable 
 patent or other IPR claims of which I am aware have been 
 disclosed, and any of which I become aware will be disclosed, in 
 accordance with RFC 3668. 
 
 Internet-Drafts are working documents of the Internet Engineering 
 Task Force (IETF), its areas, and its working groups. Note that 
 other groups may also distribute working documents as Internet-
 Drafts. 
 
 Internet-Drafts are draft documents valid for a maximum of six 
 months and may be updated, replaced, or obsoleted by other 
 documents at any time. It is inappropriate to use Internet-Drafts 
 as reference material or to cite them other than as "work in 
 progress." 
 
 The list of current Internet-Drafts can be accessed at 
 http://www.ietf.org/ietf/1id-abstracts.txt. 
 
 The list of Internet-Draft Shadow Directories can be accessed at 
 http://www.ietf.org/shadow.html. 
 
 
 Copyright Notice 
 
 Copyright (C) The Internet Society (2004). All Rights Reserved. 
 
 
 Abstract 
 
 This memo requests that the application/mbox media-type be 
 authorized for allocation by the IESG, according to the terms 
 specified in RFC 2048 [RFC2048]. This memo also defines a default 
 format for the mbox database, which must be supported by all 
 conformant implementations. 
 
 
 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 
 1. Background and Overview 
 
 UNIX-like operating systems have historically made widespread use 
 of "mbox" database files for a variety of local email purposes. In 
 the common case, mbox files store linear sequences of one or more 
 electronic mail messages, with local email clients treating the 
 database as a logical folder of email messages. mbox databases are 
 also used by a variety of other messaging tools, such as mailing 
 list management programs, archiving and filtering utilities, 
 messaging servers, and other related applications. In recent 
 years, mbox databases have also become common on a large number of 
 non-UNIX computing platforms, for similar kinds of purposes. 
 
 The increased pervasiveness of these files has led to an increased 
 demand for a standardized, network-wide interchange of these files 
 as discrete database objects. In turn, this dictates a need for a 
 media-type definition for mbox files in general, which is the 
 subject and purpose of this memo. 
 
 2. About the mbox Database 
 
 The mbox database format is not documented in an authoritative 
 specification, but instead exists as a well-known output format 
 that is anecdotally documented, or which is only authoritatively 
 documented for a specific platform or tool. 
 
 mbox databases typically contain a linear sequence of electronic 
 mail messages. Each message begins with a separator line that 
 identifies the message sender, and also identifies the date and 
 time at which the message was received by the final recipient 
 (either the last-hop system in the transfer path, or the system 
 which serves as the recipient's mailstore). Each message is 
 typically terminated by an empty line. The end of the database is 
 usually recognized by either the absence of any additional data, 
 or by the presence of an explicit end-of-file marker. 
 
 The structure of the separator lines vary across implementations, 
 but usually contain the exact character sequence of "From", 
 followed by a single Space character (0x20), an email address of 
 some kind, another Space character, a timestamp sequence of some 
 kind, and an end-of-line marker. However, due to the lack of any 
 authoritative specification, each of these attributes are known to 
 vary widely across implementations. For example, the email address 
 can reflect any addressing syntax which has ever been used on any 
 messaging system in all of history (specifically including address 
 
 Hall I-D Expires: August 2005 [page 2] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 forms which are not compatible with Internet messages, as defined 
 by RFC 2822 [RFC2822]). Similarly, the timestamp sequences can 
 also vary according to system output, while the end-of-line 
 sequences will often reflect platform-specific requirements. 
 Different data formats can even appear within a single database as 
 a result of multiple mbox files being concatenated together, or 
 because a single file was accessed by multiple messaging clients 
 which have each used their own syntax for the separator line. 
 
 Message data within mbox databases often reflects site-specific 
 peculiarities. For example, it is entirely possible for the 
 message body or headers in an mbox database to contain untagged 
 eight-bit character data that implicitly reflects a site-specific 
 default language or locale, or for timestamps and email addresses 
 to reflect local defaults, with none of this data being widely 
 portable beyond the local scope. Similarly, message data can also 
 contain unencoded eight-bit binary data, or can use encoding 
 formats which represent a specific platform (E.G., BINHEX or 
 UUENCODE sequences). 
 
 Many implementations are also known to escape message body lines 
 that begin with the character sequence of "From ", so as to 
 prevent confusion with overly-liberal parsers that do not search 
 for full separator lines. In the common case, a leading Greater-
 Than symbol (0x3E) is used for this purpose (with "From " becoming 
 ">From "). However, other implementations are known not to escape 
 such lines unless they are immediately preceded by a blank line or 
 if they also appear to contain an email address and a timestamp. 
 Other implementations are also known to perform secondary escapes 
 against these lines if they are already escaped or quoted, while 
 others ignore these mechanisms altogether. 
 
 A comprehensive description of mbox database files on UNIX-like 
 systems can be found at http://qmail.org./man/man5/mbox.html, 
 which should be treated as mostly authoritative for those 
 variations which are otherwise only documented in anecdotal form. 
 However, readers are advised that many other platforms and tools 
 make use of mbox databases, and that there are many more potential 
 variations that can be encountered in the wild. 
 
 In order to mitigate errors that may arise from such vagaries, 
 this specification defines a "format" parameter to the 
 APPLICATION/MBOX media-type declaration, which can be used to 
 identify the specific kind of mbox database that is being 
 transferred. Furthermore, this specification defines a "default" 
 database format which MUST be supported by implementations that 
 
 Hall I-D Expires: August 2005 [page 3] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 claim to be compliant with this specification, and which is to be 
 used as the implicit format for undeclared APPLICATION/MBOX data 
 objects. Additional format types are to be defined in subsequent 
 specifications. Messaging systems which receive a mbox database 
 with an unknown format parameter value SHOULD treat the data as an 
 opaque binary object, as if the data had been declared as 
 APPLICATION/OCTET-STREAM. 
 
 Refer to Appendix A for a description of the default mbox format. 
 
 Note that RFC 2046 [RFC2046] defines the multipart/digest media-
 type for transferring platform-independent message files. Since 
 that specification defines a set of neutral and strict formatting 
 rules, the multipart/digest media-type already facilitates highly-
 predictable transfer and conversion operations, and as such 
 implementers are strongly encouraged to support and use that 
 media-type where possible. 
 
 3. Prerequisites and Terminology 
 
 Readers of this document are expected to be familiar with the 
 specification for MIME [RFC2045] and MIME-type registrations 
 [RFC2048]. 
 
 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 
 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" 
 in this document are to be interpreted as described in RFC 2119 
 [RFC2119]. 
 
 4. The APPLICATION/MBOX Media-Type Registration 
 
 This section provides the media-type registration application (as 
 per [RFC2048]), which will be submitted to IANA after IESG 
 approval of this specification. 
 
 MIME media type name: application 
 
 MIME subtype name: mbox 
 
 Required parameters: none 
 
 Optional parameters: The "format" parameter identifies the format 
 of the mbox database and the messages contained therein. The 
 default value for the "format" parameter is "default", and refers 
 to the formatting rules defined in Appendix A of this memo. mbox 
 databases that do not have a "format" parameter SHOULD be 
 
 Hall I-D Expires: August 2005 [page 4] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 interpreted as having the implicit "format" value of "default". 
 mbox databases that have an unknown value for the "format" 
 parameter SHOULD be treated as opaque data objects, as if the 
 media-type had been specified as APPLICATION/OCTET-STREAM. 
 Additional values for the format parameter are to be defined in 
 subsequent specifications, and registered with IANA. 
 
 Encoding considerations: If an email client receives an mbox 
 database as a message attachment, and then stores that attachment 
 within a local mbox database, the contents of the two database 
 files may become irreversibly intermingled, such that neither 
 database is no longer independently recognizable. In order to 
 avoid these collisions, messaging systems which support this 
 specification MUST encode an mbox database (or at a minimum, the 
 separator lines) with a non-transparent transfer encoding (such as 
 BASE64 or Quoted-Printable) whenever an APPLICATION/MBOX object is 
 transferred via messaging protocols. Other transfer services are 
 generally encouraged to adopt similar encoding strategies to allow 
 for any subsequent retransmission which might occur, but are not 
 explicitly required to do so. Implementers should also be prepared 
 to encode mbox data locally if non-compliant data is received. 
 
 Security considerations: mbox data is passive, and does not 
 generally represent a unique or new security threat. However, 
 there is risk in sharing any kind of data, in that unintentional 
 information may be exposed, and this risk certainly applies to 
 mbox data as well. 
 
 Interoperability considerations: Due to the lack of a single 
 authoritative specification for mbox databases, there are a large 
 number of variations between database formats (refer to the 
 introduction text for common examples), and it is expected that 
 non-conformant data will be erroneously tagged or exchanged. 
 Although the "default" format specified in this memo does not 
 allow for these kinds of vagaries, prior negotiation or agreement 
 between humans may sometimes be needed. 
 
 Published specification: see Appendix A. 
 
 Applications which use this media type: hundreds of messaging 
 products make use of the mbox database format, in one form or 
 another. 
 
 Magic number(s): mbox database files can be recognized by having a 
 leading character sequence of "From", followed by a single Space 
 character (0x20), followed by additional printable character data 
 
 Hall I-D Expires: August 2005 [page 5] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 (refer to the description in Appendix A for details). However, 
 implementers are cautioned that all such files will not be 
 compliant with all of the formatting rules, so implementers should 
 treat these files with an appropriate amount of circumspection. 
 
 File extension(s): mbox database files sometimes have an ".mbox" 
 extension, but this is not required nor expected. As with magic 
 numbers, implementers should avoid reflexive assumptions about the 
 contents of such files. 
 
 Macintosh File Type Code(s): None are known to be common. 
 
 Person & email address to contact for further information: Eric A. 
 Hall (ehall@ntrg.com) 
 
 Intended usage: COMMON 
 
 5. Security Considerations 
 
 See the discussion in section 4. 
 
 6. IANA Considerations 
 
 Upon IESG approval, IANA would be expected to register the 
 APPLICATION/MBOX media-type in the MIME registry, using the 
 application provided in section 4 above. 
 
 Furthermore, IANA would be expected to establish and maintain a 
 registry of values for the "format" parameter as described in this 
 memo. The first registration would be the "default" value, using 
 the description provided in Appendix A. Subsequent values for the 
 "format" parameter MUST be accompanied by some form of 
 recognizable, complete, and legitimate specification, such as an 
 IESG-approved specification. or some kind of authoritative vendor 
 documentation. 
 
 7. Normative References 
 
 [RFC2046] Freed, N., Borenstein, N., "Multipurpose 
 Internet Mail Extensions (MIME) Part Two: 
 Media Types", RFC 2046, November 1996. 
 
 [RFC2048] Freed, N., Klensin, J., Postel, J., 
 "Multipurpose Internet Mail Extensions (MIME) 
 Part Four: Registration Procedures", BCP 13, 
 RFC 2048, November 1996. 
 
 
 Hall I-D Expires: August 2005 [page 6] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 [RFC2119] Bradner, S., "Key words for use in RFCs to 
 Indicate Requirement Levels", BCP 14, RFC 
 2119, March 1997. 
 
 [RFC2822] Resnick, P., "Internet Message Format", RFC 
 2822, April 2001. 
 
 
 Appendix A. The "default" mbox Database Format 
 
 In order to improve interoperability among messaging systems, this 
 memo defines a "default" mbox database format, which MUST be 
 supported by all implementations claiming to be compliant with 
 this specification. 
 
 The "default" mbox database format uses a linear sequence of 
 Internet messages, with each message being immediately prefaced by 
 a separator line, and being terminated by an empty line. More 
 specifically: 
 
 o Each message within the database MUST follow the syntax and 
 formatting rules defined in RFC 2822 [RFC2822] and its 
 related specifications, with the exception that the canonical 
 mbox database MUST use a single Line-Feed character (0x0A) as 
 the end-of-line sequence, and MUST NOT use a Carriage-
 Return/Line-Feed pair (NB: this requirement only applies to 
 the canonical mbox database as transferred, and does not 
 override any other specifications). This usage represents the 
 most common historical representation of the mbox database 
 format, and allows for the least amount of conversion. 
 
 o Messages within the default mbox database MUST consist of 
 seven-bit characters within an eight-bit stream. Eight-bit 
 data within the stream MUST be converted to a seven-bit form 
 (using an appropriate, standardized encoding) and 
 appropriately tagged (with the correct header fields) before 
 the database is transferred. 
 
 o Message headers and data in the default mbox database MUST be 
 fully-qualified, as per the relevant specification[s]. For 
 example, email addresses in the various header fields MUST 
 have legitimate domain names (as per RFC 2822), while 
 extended characters and encodings MUST be specified in the 
 appropriate location (as per the appropriate MIME 
 specifications), and so forth. 
 
 
 Hall I-D Expires: August 2005 [page 7] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 o Each message in the mbox database MUST be immediately 
 preceded by a single separator line, which MUST conform to 
 the following syntax: 
 
 The exact character sequence of "From"; 
 
 a single Space character (0x20); 
 
 the email address of the message sender (as obtained from 
 the message envelope or other authoritative source), 
 conformant with the "addr-spec" syntax from RFC 2822; 
 
 a single Space character; 
 
 a timestamp indicating the UTC date and time when the 
 message was originally received, conformant with the 
 syntax of the traditional UNIX 'ctime' output sans 
 timezone (note that the use of UTC precludes the need for 
 a timezone indicator); 
 
 an end-of-line marker. 
 
 o Each message in the database MUST be terminated by an empty 
 line, containing a single end-of-line marker. 
 
 Note that the first message in an mbox database will only be 
 prefaced by a separator line, while every other message will begin 
 with two end-of-line sequences (one at the end of the message 
 itself, and another to mark the end of the message within the mbox 
 database file stream) and a separator line (marking the new 
 message). The end of the database is implicitly reached when no 
 more message data or separator lines are found. 
 
 Also note that this specification does not prescribe any escape 
 syntax for message body lines that begin with the character 
 sequence of "From ". Recipient systems are expected to parse full 
 separator lines as they are documented above. 
 
 
 Acknowledgments 
 
 Funding for the RFC editor function is currently provided by the 
 Internet Society. 
 
 
 
 Hall I-D Expires: December 2004 [page 8] 
 Internet Draft draft-hall-mime-app-mbox-04.txt February 2005 
 
 
 Authors' Addresses 
 
 Eric A. Hall 
 ehall@ntrg.com 
 
 
 Full Copyright Statement 
 
 Copyright (C) The Internet Society 2004. This document is subject 
 to the rights, licenses and restrictions contained in BCP 78, and 
 except as set forth therein, the authors retain all their rights. 
 
 This document and the information contained herein are provided on 
 an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 
 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND 
 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, 
 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT 
 THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR 
 ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A 
 PARTICULAR PURPOSE. 
 
 
 Hall I-D Expires: December 2004 [page 9]