More Instant Messaging Interoperability (MIMI) Outer Layer

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. The reader may wish to note that one of the two RFC references in the preceding paragraph was normative; the other was informative. These will be correctly identified as such in the References section below.

Introduction Compatibility with existing email systems is a nice property to have. The email infrastructure is distributed, tries to deal with the spam problem using its own means and most email clients are robust and feature-rich, built with long-term archival in mind. MIME is still the standard format in email exchange. It definitely shows its age (it's rather complex to implement, text-only, self-contained, etc) but otherwise stood the test of time so could very well form the basis of a next generation messaging format. The JMAP Email object (§.4) is one such attempt -- it simplifies MIME processing by shedding obsolete features like support for non-unicode character encodings but keeps defining features like being text-only and recursive structure. The JMAP protocol also adds blob support which adds an alternate transport for binary data, which not only dramatically lowers the impact of using a text-only format, but also makes it possible to bundle arbitrary size or amount of attachments together. However, email lacks structure, except in very niche applications like meeting requests, which renders it non-suitable for most of instant messaging applications. The history of instant messaging so far makes it obvious that it's not possible to foresee all actions a client may implement. For example, at the height of its popularity, the MSN client famously let its users shake the windows of their peers. WhatsApp is very good at sending plain-text messages, but Snapchat came up with stickers and expiring messages, which other clients eventually had to implement. Any system that seeks to unify message exchange must be flexible enough to capture and encode any current and future needs of messaging applications.

MIMI-INK format We propose the MIMI-INK format, message/mimi-ink, to be renamed to message/mimi if it gets standardized, which is made of the following primitives:

A dict of headers. It "MUST" contain the defining entry named "Root-Content-Id", among other ones.
An optional message body in any number of formats. It's supposed to describe the purpose of the message for clients that don't support the attached structure, though of course it can be anything.
At least one blob that contains the main data structure with Content-Id: [Root-Content-Id-Value]defined in the message headers. This is called the "root content".
Zero or more additional blobs that may be referenced from inside the main structure for any reason.

In MIME terms;

headers with at least Root-Content-Id: XYZ and Content-Type: message/mimi-ink
multipart/mixed
- multipart/alternative
  - text/html
  - text/plain
  - etc.
- multipart/mixed
  - one of application/{xml,json,msgpack,etc} with content-id="XYZ"
  - one or more binary objects

The root content must at least denote a namespace, name and actual content. An optional errorcode could also be included, if the content designates an error message. The client "MUST" validate the content according to the information given in namespace/name values and refuse to process the message by resorting to interpret it a regular email message with attachments. Some examples:

https://github.com/plq/mimi/blob/main/reaction.eml
https://github.com/plq/mimi/blob/main/vibrate.eml

We omitted non-essential JMAP properties for sake of simplicity. The mimi-ink repository contains software that converts the MIME structure to the suggested jmap structure. It is assumed that there is a 1-to-1 releation between the MIME representation and the JMAP representation of a message, even though that's not correct -- whatever gets lost in translations is not of interest. The following key differences exist with the JMAP Email object:

Uses msgpack for the outermost layer instead of JSON.
The "content" property was added to represent inline data where appropriate.
The root content needs to represent an abstract structure, serialized as any popular format (json, xml, msgpack, etc.).
Add an XML-like namespacing structure so that both standards-compliant and proprietary objects can coexist. Or force the inner layer to be XML?
Not technically a difference, but: Using a message body with a well-defined structure makes the recursivity of the outer layer (JMAP/MIME) redundant. This kind of structure can be realized inside the payload.

Rationale

Msgpack msgpack is;

Binary
Simple to implement: Here's an implementation in ~400 lines of python: https://github.com/plq/msgpack-python-pure/blob/master/msgpack_pure/_core.py
Supports enough primitives to be useful: null, true, false, int64, uint64, float32, float64, string, bytearray, list, map
Doesn't overstep its boundaries by defining complex types like Date

However, there is stuff that needs to be further/better specified:

msgpack doesn't have a standard way of defining a schema. We could just imitate the relevant bits of XSD.
As said above, there is no standard way of serializing complex objects like dates.
It's very easy to prepend headers to MIME, which eg. makes it very easy to trace its origins via Receivedheaders. "Patching" msgpack like this doesn't seem practical. However, it's quite easy to tell concatenated msgpack objects apart. So it may be desirable to specify MIMI as a bunch of concatenated msgpack objects instead of just one object containing everything.

If there is a simpler binary format that provides equivalent functionality, it could be adopted instead. msgpack is not a hard requirement here but does have interesting properties that make it a strong contender.

Doing away with recursivity

Recursive formats like MIME/JMAP add a great deal of flexibility to the wrong layer.
MIMI outer shell needs to be as simple as possible. If a complex message bundle is needed, it can be easily expressed by the inner structure(s).
MIMI objects can be nested as attachments and the clients could choose to interpret it further down anyway.

Content

Object definitions When specified in onion-like layers like this, the object definitions become the subject of another standard. It's off-topic here.

Validation To standardize how clients could validate incoming content, we need to specify or choose a schema language first. XML is the clear winner as the inner format here as it already has everything, including widespread software support.

IANA Considerations The inner format may require IANA to maintain a supported MIMI object types registry.