IMAP UIDBATCHES Extension

The UIDBATCHES extension An IMAP server advertises support for the UIDBATCHES extension by including the UIDBATCHES capability in the CAPABILITY response / response code.

UIDBATCHES Command

Arguments:: Message count per batch.
Optional batch range.
Responses:: UIDBATCHES response
Result:: OK
BAD command unknown or arguments invalid

When the client sends a UIDBATCHES command to the server, the server will return the UID ranges that partition the messages in the currently selected mailbox into equally sized batches. Batches are arranged by descending UID order, with the first batch containing the highest UIDs. For a mailbox with <M> messages, requesting batches of size <N> returns UID ranges corresponding to the sequence numbers :

: ... ]]>

Usage and Example When the client selects a mailbox, it can use the UIDBATCHES command to find the UIDs that split the mailbox’s messages into batches. For example: The server’s response provides four UID ranges:

215295:99695
99695:20351
20350:7830
7829:1

Each range contains up to 2,000 messages, except the last range, which contains the remaining 823 messages. As new messages cannot appear within these UID ranges, the number of messages in each range will not increase. It may decrease, though, as messages are deleted. The client MUST NOT re-run UIDBATCHES unless at least one of the following conditions is met:

A different mailbox has been selected
More than N/2 messages have been expunged from the mailbox
More than N/2 new messages have been received into the mailbox

These limits are in place to protect servers, since it may be expensive for the server to run this command. The client MUST NOT excessively re-run UIDBATCHES while the mailbox remains selected. The client can keep track of the number of EXPUNGE or VANISH messages and re-run UIDBATCHES if many messages are deleted. As new messages arrive into the mailbox, the client should add these to a new message batch (starting at UID 215296 in the above example). Once N/2 or more new messages have been added to the mailbox, the client can ask for updated batches by re-running the UIDBATCHES command. The server MAY reject UIDBATCHES commands with a BAD response if the client exceeds these limits.

Response Format The server MUST reply with a UIDBATCHES response, even if no ranges are returned (see ). The UIDBATCHES response MUST include the tag of the command it relates to (similar to an ESEARCH response). The UID ranges in the response MUST be ordered in descending sequence, from the highest to the lowest UIDs.

Batch Sizes This extension enforces a hard limit on the minimum batch size that a client can request, and it gives the server some flexibility in the actual size being returned. This ensures the server has implementation flexibility, making the operation less resource-intensive. And to prevent clients from misusing this extension to infer message sequence numbers. The intent of this extension is to work well in combination with UIDONLY mode without creating a de-facto loophole that re-introduces sequence numbers. also outlines some reasoning for these limitation. The server MUST support batch sizes of 500 messages or larger. The server MUST respond with BAD and a response code TOOSMALL if the client uses a batch size that is smaller than the minimum allowed by the server, e.g. The server MUST NOT return ranges that contain more than the number of messages per batch requested by the client (2,000 in the above example). But the server MAY return fewer messages per range, notably if that makes the implementation simpler and/or more efficient. Servers SHOULD NOT return batches that are substantially smaller, and SHOULD aim to be within 90% of the requested size. The client is likely to pick a batch count based on what it wants to display to the user. A client may e.g. request 2 batches of size 1,000 if it wants to be able to display the last 2,000 messages to the user. A server MAY return batches that are substantially smaller if there are changes in mailbox state during the execution of the UIDBATCHES command, namely messages being expunged, such that the overall size of the mailbox changes. The client would be able to infer this from the `EXPUNGE`, `VANISHED`, or `EXISTS` responses it receives. If the total number of messages is not evenly divisible by the requested batch size, the last batch will contain the remainder. Thus, the last batch in the mailbox (i.e. the batch with the lowest UIDs) will usually have fewer messages than the requested number of messages. If the requested batch size is equal to or larger than the number of messages in the mailbox, the server MUST return a response with a single UID range that spans all messages.

UIDs The server MAY return UID ranges with UIDs that do not exist on the server. The client as a result MUST NOT make assumptions about the existence of messages. If the server returns the response there may not be any messages on the server with the UIDs such as 163886, 99703, 99696, etc. The range 163886:99703 will span approximately the requested number of messages (may be less, see ), but its start and end UIDs may not correspond to messages on the server. This gives the server implementation some flexibility as to which UID ranges to return. They might, e.g., return 163886:99697 and 99696:20358 instead of 163886:99703 and 99696:20358 -- assuming that there are no messages in the range 99704:99697. If there are fewer message in the mailbox than the requested batch size, the server would return a single batch that contains all messages in the mailbox. Servers SHOULD end the last UID batch in the mailbox with UID 1 even if this UID does not exist on the server. This makes it unambiguous to the client that this range is in fact the last range.

Batch Ranges A client can optionally provide a batch range. The server limits its response to UID ranges corresponding to the specified batch indices. For example, if the client sends for a mailbox with more than 40,000 messages, the server would return the 10th to 20th batches, corresponding to the 20,000th and 40,000th message respectively. Note that batches start at the highest UIDs: batch 1 is the batch with the highest UIDs. The UID ranges that the server returns would still split the mailbox’s messages into batches of the requested size (2,000 in the example). If the client requests more batches than exist on the server, the server would return those that do exist. For example if the client sends and the selected mailbox has 7,000 messages, the server would then return a UIDBATCHES response with only 4 UID ranges. Batch ranges such as 1:4 in the above example MUST be ordered lowest to highest, i.e. be sent as 1:4 and not as 4:1. Servers SHOULD reject batch ranges that are in the wrong order. If the client requests a range of batches that do not exist on the server, the server MUST still return an empty response. See section . Note that the number of messages per batch returned by the server may be approximate as detailed in . As a result, if the client needs to request consecutive batch ranges such as 1:100, 101:200, 201:300, and so on, the client may want to make these batch ranges overlap by e.g. requesting 1:100, 100:200, and 200:300. The client would then be able to check if the resulting UIDs do in fact overlap. Clients MUST NOT request batch ranges that span more than 100,000 messages, i.e. the number of batches multiplied by the batch size MUST NOT be larger than 100,000. The server MAY reject UIDBATCHES commands with a BAD response with the LIMIT response code if the client exceeds this limit.

Empty Responses When the client issues any valid UIDBATCHES command and the mailbox is empty, the server MUST reply with a UIDBATCHES response, e.g. If the client requests a range of batches that do not exist, the server MUST reply with an empty UIDBATCHES response. If the mailbox has 7,000 messages, and the client sends the server would respond with

Large Mailboxes The server may not be able to return all UID ranges if the mailbox contains an extremely large number of messages. The server MUST at least support returning UID ranges spanning 100,000 messages. See for details on this limit. If the server can not return all of the requested UID ranges, it MUST respond with a BAD response with the LIMIT response code. Notably, when the client requests all UID ranges and the mailbox has more than 100,000 messages, the server MAY reply with a BAD response. For example: The client should know what the message count in the mailbox is, and if the message count exceeds 100,000 it may choose to always request batch ranges as discussed in instead of requesting all batches.

Similarity to UID SEARCH Command The UIDBATCHES is in effect nothing more than shorthand for a UID SEARCH command of the form ,,,,... ]]> where M is the number of messages in the mailbox and N is the requested batch count. The special purpose UIDBATCHES command, though, tries to address two problems:

for many servers, UID SEARCH commands specifying sequence numbers are costly, especially for mailboxes with many messages.
the UIDONLY extension disallows the use of sequence numbers and thus makes it difficult for the client to split its commands into batches of a size that works well for the client and server.

By providing a special purpose command, servers can implement a different, optimized code path for determining message batches. And servers using the UIDONLY extension can provide a facility to let the client determine message batches without using sequence numbers in a UID SEARCH command. describes some implementation restrictions to ensure this.

Similarity to PARTIAL Extension The PARTIAL extension provides a different way for the client to split its commands into batches by using paged SEARCH and FETCH. The intention of the UIDBATCHES command is to let the client pre-determine message batches of a desired size. This makes it easier for the client to share implementation between servers regardless of their support of PARTIAL. And additionally, because the client can issue a corresponding UID SEARCH command to servers that do not implement UIDBATCHES, the client can use similar batching implementations for servers that support UIDBATCHES and those that do not.

Interaction with MESSAGELIMIT Extension When the server supports both the MESSAGELIMIT and UIDBATCHES extension, the client SHOULD request batches no larger than the specified maximum number of messages that can be processed in a single command. The client MAY choose to use a smaller batch size. Additionally, since servers MAY limit the number of UIDs returned in response to UIDBATCHES, it is reasonable to assume that they would at most return N UIDs where N is the limit the server announced as its MESSAGELIMIT.

Interaction with UIDONLY Extension As noted above, the UIDBATCHES extension allows for clients to create UID ranges for message batches even when the connection operates in UIDONLY mode which otherwise doesn't allow for using message sequence numbers.

Interaction with SEARCHRES Extension UIDBATCHES is not a SEARCH nor UID SEARCH command. Servers that support SEARCHRES MUST NOT store the result of UIDBATCHES in the $ variable.