AVTCore R. van Brandenburg Internet Draft H. Stokking Intended status: Standards Track M. van Deventer Expires: May 3, 2012 TNO Netherlands F. Boronat M. Montagud Universidad Politecnica de Valencia Kevin Gross AVA Networks October 31, 2011 RTCP for inter-destination media synchronization draft-ietf-avtcore-idms-02.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 3, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Brandenburg Expires May 3, 2012 [Page 1] Internet-Draft RTCP for IDMS October 31, 2011 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Abstract This document gives information on an RTCP Packet Type and RTCP XR Block Type including associated SDP parameters for inter-destination media synchronization (IDMS). The RTCP XR Block Type, registered with IANA based on an ETSI specification, is used to collect media play- out information from participants in a group playing-out (watching, listening, etc.) a specific RTP media stream. The RTCP packet type specified by this document is used to distribute a summary of the collected information so that the participants can synchronize play- out. Typical applications for IDMS are social TV, shared service control (i.e. applications where two or more geographically separated users are watching a media stream together), distance learning, networked video walls, networked speakers, etc. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Inter-destination Media Synchronization . . . . . . . . . . 3 1.2. Applicability of RTCP to IDMS . . . . . . . . . . . . . . . 3 1.3. Applicability of SDP to IDMS . . . . . . . . . . . . . . . 4 1.4. This document and ETSI TISPAN . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Overview of IDMS operation . . . . . . . . . . . . . . . . . . 4 4. Inter-destination media synchronization use cases . . . . . . . 6 5. Architecture for inter-destination media synchronization . . . 7 5.1. Media Synchronization Application Server (MSAS) . . . . . . 7 5.2. Synchronization Client (SC) . . . . . . . . . . . . . . . . 8 5.3. Communication between MSAS and SCs . . . . . . . . . . . . 8 6. RTCP XR Block Type for IDMS . . . . . . . . . . . . . . . . . . 9 7. RTCP Packet Type for IDMS (IDMS report) . . . . . . . . . . . . 11 8. Timing and NTP Considerations . . . . . . . . . . . . . . . . . 13 8.1. Leap Seconds . . . . . . . . . . . . . . . . . . . . . . . 14 9. SDP Parameter for RTCP XR IDMS Block Type . . . . . . . . . . . 15 10. SDP Parameter for RTCP IDMS Packet Type . . . . . . . . . . . 15 11. SDP parameter for clock source . . . . . . . . . . . . . . . . 16 12. Compatibility with ETSI TISPAN . . . . . . . . . . . . . . . . 18 13. On the use of presentation timestamps . . . . . . . . . . . . 19 14. Security Considerations . . . . . . . . . . . . . . . . . . . 19 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 Brandenburg Expires May 3, 2012 [Page 2] Internet-Draft RTCP for IDMS October 31, 2011 16. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 21 17. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 21 18. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22 18.1. Normative References . . . . . . . . . . . . . . . . . . . 22 18.2. Informative References . . . . . . . . . . . . . . . . . . 22 1. Introduction 1.1. Inter-destination Media Synchronization Inter-destination media synchronization (IDMS) refers to the play-out of media streams at two or more geographically distributed locations in a temporally synchronized manner. It can be applied to both unicast and multicast media streams and can be applied to any type and/or combination of streaming media, such as audio, video and text (subtitles). [Ishibashi2006] and [Boronat2009] provide an overview of technologies and algorithms for IDMS. IDMS requires the exchange of information on media receipt and playout times. It may also require signaling for the initiation and maintenance of IDMS sessions and groups. The presented RTCP specification for IDMS is independent of the used synchronization algorithm, which is out-of-scope of this document. 1.2. Applicability of RTCP to IDMS Currently, most multimedia applications make use of RTP and RTCP [RFC3550]. RTP (Real-time Transport Protocol) provides end-to-end network transport functions suitable for applications requiring real- time data transport, such as audio, video or data, over multicast or unicast network services. The timestamps and sequence number mechanisms provided by RTP are very useful to reconstruct the original media timing, reorder and detect some packet loss at the receiver side. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner that is scalable to large multicast networks, and to provide minimal control and identification functionality. RTP receivers and senders provide reception quality feedback by sending out RTCP Receiver Report (RR) and Sender Report (SR) packets [RFC3550] respectively, which may be augmented by eXtended Reports (XR) [RFC3611]. Thus, the feedback reporting features provided by RTCP make QoS monitoring possible and can be used for troubleshooting Brandenburg Expires May 3, 2012 [Page 3] Internet-Draft RTCP for IDMS October 31, 2011 and fault tolerance management in multicast distribution services such as IPTV. These protocols are intended to be tailored through modification and/or additions in order to include profile-specific information required by particular applications, and the guidelines on doing so are specified in [RFC5968]. IDMS involves the collection, summarizing and distribution of RTP packet arrival and play-out times. As information on RTP packet arrival times and play-out times can be considered reception quality feedback information, RTCP becomes a promising candidate for carrying out IDMS, which may facilitate implementation in typical multimedia applications. 1.3. Applicability of SDP to IDMS RTCP XR [RFC3611] defines the Extended Report (XR) packet type for the RTP Control Protocol (RTCP), and defines how the use of XR packets can be signaled by an application using the Session Description Protocol (SDP) [RFC4566]. SDP signaling is used to set up and maintain a synchronization group between Synchronization Clients (SCs). This document describes two SDP parameters for doing this, one for the RTCP XR block type and one for the new RTCP packet type. This document also allows for a receiver to indicate a used clock source for synchronizing the receiver clock used in the IDMS session. This is also done using an SDP parameter, which is described in this document. 1.4. This document and ETSI TISPAN ETSI TISPAN [TS 183 063] has specified architecture and protocol for IDMS using RTCP XR exchange and SDP signaling. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] and indicate requirement levels for compliant implementations. 3. Overview of IDMS operation This section provides a brief example of how the IDMS RTCP functionality is used. The section is tutorial in nature and does not Brandenburg Expires May 3, 2012 [Page 4] Internet-Draft RTCP for IDMS October 31, 2011 contain any normative statements. Alice's . . . . . . .tv:abc.com . . . . . . . . . Bob's TV (Sync Client) (Sync Server) Laptop (SC) | | | | Media Session | | |<=====================>| | | Invite(URL,Sync-group ID) | |------------------------------------------------->| | | Media Session Set-up | | |<========================>| | | | | Call set-up | |<================================================>| | | | | RTP Packet | RTP Packet | |<----------------------|------------------------->| | RR + IDMS XR | | |---------------------->| RR + IDMS XR | | |<-------------------------| | RTCP IDMS packet | RTCP IDMS packet | |<----------------------|------------------------->| | | | Alice is watching TV in her living room. At some point she sees that a football game of Bob's favorite team is on. She sends him an invite to watch the program together. Embedded in the invitation is the link to the media server and a unique sync-group identifier. Bob, who is also at home, receives the invite on his laptop. He accepts Alice's invitation and the RTP client on his laptop sets up a session to the media server. A VoIP connection to Alice's TV is also set up, so that Alice and Bob can talk while watching the program. As is common with RTP, both the RTP client in Alice's TV as well as the one in Bob's laptop send periodic RTCP Receiver Reports (RR) to the media server. However, in order to make sure Alice and Bob see the events in the football game at the same time, their clients also periodically send an IDMS XR block to the sync server function of the media server. Included in the XR blocks are timestamps on when both Alice and Bob have received (or played out) a particular RTP packet. The sync server function in the media server calculates a reference client from the received IDMS XR blocks (e.g. by selecting whichever client received the packet the latest as the reference client). It then sends an RTCP IDMS packet containing the play-out information of Brandenburg Expires May 3, 2012 [Page 5] Internet-Draft RTCP for IDMS October 31, 2011 this reference client to both Alice and Bob. In this case Bob has the slowest connection and the reference client therefore includes a delay similar to the one experienced by Bob. Upon reception of this information, Alice's RTP client can choose what to do with this information. In this case it decreases its play- out rate temporarily until it matches with the reference client play- out (and thus matches Bob's play-out). Another option for Alice's TV would be to simply pause playback until it catches up. The exact implementation of the synchronization algorithm is up to the client. Upon reception of the reference client RTCP IDMS packet, Bob's client does not have to do anything since it is already synchronized to the reference client (since it is based on Bob's delay). Note that other synchronization algorithms may introduce even more delay than the one experienced by the most delayed client, e.g. to account for delay variations, for new clients joining an existing synchronization group, etc. 4. Inter-destination media synchronization use cases There are a large number of use cases imaginable in which IDMS might be useful. This section will highlight some of them. It should be noted that this section is in no way meant to be exhaustive A first usage scenario for IDMS is Social TV. Social TV is the combination of media content consumption by two or more users at different devices and locations and real-time communication between those users. An example of Social TV, is when two or more users are watching the same television broadcast at different devices and locations, while communicating with each other using text, audio and/or video. A skew in the media play-out of the two or more users can have adverse effects on their experience. A well-known use case here is one friend experiencing a goal in a football match well before or after other friend(s). Thus IDMS is required to provide play-out synchronization. Another potential use case for IDMS is the video wall. A video wall consists of multiple computer monitors, video projectors, or television sets tiled together contiguously or overlapped in order to form one large screen. Each of the screens reproduces a portion of the larger picture. In some implementations, each screen may be individually connected to the network and receive its portion of the overall image from a network-connected video server or video scaler. Screens are refreshed at 60 hertz (every 16-2/3 milliseconds) or potentially faster. If the refresh is not synchronized, the effect of multiple screens acting as one is broken. Brandenburg Expires May 3, 2012 [Page 6] Internet-Draft RTCP for IDMS October 31, 2011 A third usage scenario is that of the networked loudspeakers, in which two or more speakers are connected to the network individually. Such situations can for example be found in large conference rooms, legislative chambers, classrooms (especially those supporting distance learning) and other large-scale environments such as stadiums. Since humans are more susceptible to differences in audio delay, this use case needs even more accuracy than the video wall use case. Depending on the exact application, the need for accuracy can then be in the range of microseconds. 5. Architecture for inter-destination media synchronization The architecture for IDMS, which is based on a sync-maestro architecture [Boronat2009], is sketched below. The Synchronization Client (SC) and Media Synchronization Application Server (MSAS) entities are shown as additional functionality for the RTP receiver and sender respectively. It should be noted that a master/slave type of architecture is also supported by having one of the SC devices also act as an MSAS. In this case the MSAS functionality is thus embedded in an RTP receiver instead of an RTP sender. +-----------------------+ +-----------------------+ | | SR + | | | RTP Receiver | RTCP | RTP Sender | | | IDMS | | | +-----------------+ | <----- | +-----------------+ | | | | | | | | | | | Synchronization | | | | Media | | | | Client | | | | Synchronization | | | | (SC) | | | | Application | | | | | | | | Server | | | | | | RR+XR | | (MSAS) | | | | | | -----> | | | | | +-----------------+ | | +-----------------+ | | | | | +-----------------------+ +-----------------------+ 5.1. Media Synchronization Application Server (MSAS) An MSAS collects RTP packet arrival times and play-out times from one or more SC(s) in a synchronization group. The MSAS summarizes and distributes this information to the SCs in the synchronization group as synchronization settings, e.g. by determining the SC with the most lagged play-out and using its reported RTP packet arrival time and play-out time as a summary. Brandenburg Expires May 3, 2012 [Page 7] Internet-Draft RTCP for IDMS October 31, 2011 5.2. Synchronization Client (SC) An SC reports RTP packet arrival times and play-out times of a media stream. It can receive summaries of such information, and use that to adjust its play-out buffer. 5.3. Communication between MSAS and SCs Two different message types are used for the communication between MSAS and SCs. For the SC->MSAS message containing the play-out information of a particular client, an RTCP XR Block Type is used (see Section 6). For the MSAS->SC message containing the synchronization settings instructions, a new RTCP Packet Type is defined in Section 7. Brandenburg Expires May 3, 2012 [Page 8] Internet-Draft RTCP for IDMS October 31, 2011 6. RTCP XR Block Type for IDMS This section describes the RTCP XR Block Type for reporting IDMS information on an RTP media stream. Its definition is based on [RFC3611]. The RTCP XR is used to provide feedback information on receipt times and presentation times of RTP packets to e.g. a Sender [RFC3611], a Feedback Target [RFC5576] or a Third Party Monitor [RFC3611]. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P| Resrv | PT=XR=207 | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | BT=12 | SPST |Resrv|P| block length=7 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PT | Resrv | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Media Stream Correlation Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, most significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, least significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received RTP timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Presented NTP timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The first 64 bits form the header of the RTCP XR, as defined in [RFC3611]. The SSRC of packet sender identifies the sender of the specific RTCP packet. The IDMS report block consists of 7 32-bit words, with the following fields: Block Type (BT): 8 bits. It identifies the block format. Its value SHALL be set to 12. Synchronization Packet Sender Type (SPST): 4 bits. This field identifies the role of the packet sender for this specific eXtended Report. It can have the following values: Brandenburg Expires May 3, 2012 [Page 9] Internet-Draft RTCP for IDMS October 31, 2011 SPST=0 Reserved For future use. SPST=1 The packet sender is an SC. It uses this XR to report synchronization status information. Timestamps relate to the SC input. SPST=2 This setting is reserved in order to preserve compatibility with ETSI TISPAN [TS 183 063]. See section 12. for more information. SPST=3-15 Reserved For future use. Reserved bits (Resrv): 3 bits. These bits are reserved for future definition. In the absence of such a definition, the bits in this field MUST be set to zero and MUST be ignored by the receiver. Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the Packet Presented NTP timestamp field contains a value, 0 if it is empty. If this flag is set to zero, then the Packet Presented NTP timestamp shall not be inspected. Block Length: 16 bits. This field indicates the length of the block in 32 bit words and shall be set to 7, as this RTCP Block Type has a fixed length. Payload Type (PT): 7 bits. This field identifies the format of the media payload, according to [RFC3551]. The media payload is associated with an RTP timestamp clock rate. This clock rate provides the time base for the RTP timestamp counter. Reserved bits (Resrv): 25 bits. These bits are reserved for future use and shall be set to 0. Media Stream Correlation Identifier: 32 bits. This identifier is used to correlate synchronized media streams. The value 0 (all bits are set "0") indicates that this field is empty. The value 2^32-1 (all bits are set "1") is reserved for future use. If the RTCP Packet Sender is an SC (SPST=1), then the Media Stream Correlation Identifier maps on the SyncGroupId to which the SC belongs. SSRC: 32 bits. The SSRC of the media source shall be set to the value of the SSRC identifier carried in the RTP header [RFC3550] of the RTP packet to which the XR relates. Packet Received NTP timestamp: 64 bits. This timestamp reflects the wall clock time at the moment of arrival of the first octet of the RTP packet to which the XR relates. It is formatted based on the NTP timestamp format as specified in [RFC5905]. See section 8 for more information on how this field is set. Brandenburg Expires May 3, 2012 [Page 10] Internet-Draft RTCP for IDMS October 31, 2011 Packet Received RTP timestamp: 32 bits. This timestamp has the value of the RTP time stamp carried in the RTP header [RFC3550] of the RTP packet to which the XR relates. Several consecutive RTP packets will have equal timestamps if they are (logically) generated at once, e.g., belong to the same video frame. It may well be the case that one receiver reports on the first RTP packet having a certain RTP timestamp and a second receiver reports on the last RTP packet having that same RTP timestamp. This would lead to an error in the synchronization algorithm due to the faulty interpretation of considering both reports to be on the same RTP packet. To solve this, an SC SHOULD report on RTP packets in which a certain RTP timestamp shows up for the first time. Packet Presented NTP timestamp: 32 bits. This timestamp reflects the wall clock time at the moment the data contained in the first octet of the associated RTP packet is presented to the user. It is based on the time format used by NTP and consists of the least significant 16 bits of the NTP seconds part and the most significant 16 bits of the NTP fractional second part. If this field is empty, then it SHALL be set to 0 and the Packet Presented NTP timestamp flag (P) SHALL be set to 0. Presented here means the moment the data is presented to the user of the system, i.e. sound played out through speakers, video images being displayed on some display, etc. The accuracy resulting from the synchronization algorithm will only be as good as the accuracy with which the receivers can determine the delay between receiving packets and presenting them to the end-user. 7. RTCP Packet Type for IDMS (IDMS report) This section specifies the RTCP Packet Type for indicating synchronization settings instructions to a receiver of the RTP media stream. Its definition is based on [RFC3550]. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P| Resrv | PT=TBD | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Media Stream Correlation Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, most significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Received NTP timestamp, least significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Brandenburg Expires May 3, 2012 [Page 11] Internet-Draft RTCP for IDMS October 31, 2011 | Packet Received RTP timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Presented NTP timestamp, most significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Packet Presented NTP timestamp, least significant word | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The first 64 bits form the header of the RTCP Packet Type, as defined in [RFC3550]. The SSRC of packet sender identifies the sender of the specific RTCP packet. The RTCP IDMS packet consists of 7 32-bit words, with the following fields: SSRC: 32 bits. The SSRC of the media source shall be set to the value of the SSRC identifier carried in the RTP header [RFC3550] of the RTP packet to which the RTCP IDMS packet relates. Media Stream Correlation Identifier: 32 bits. This identifier is used to correlate synchronized media streams. The value 0 (all bits are set "0") indicates that this field is empty. The value 2^32-1 (all bits are set "1") is reserved for future use. The Media Stream Correlation Identifier maps on the SyncGroupId of the group to which this packet is sent. Packet Received NTP timestamp: 64 bits. This timestamp reflects the wall clock time at the reference client at the moment it received the first octet of the RTP packet to which this packet relates. It can be used by the synchronization algorithm on the receiving SC to set the required playout delay. The timestamp is formatted based on the NTP timestamp format as specified in [RFC5905]. See section 8 for more information on how this field is set. Packet Received RTP timestamp: 32 bits. This timestamp has the value of the RTP time stamp carried in the RTP header [RFC3550] of the RTP packet to which the XR relates. This SHOULD relate to the first RTP packet containing this particular RTP timestamp, in case multiple RTP packets contain the same RTP timestamp. Packet Presented NTP timestamp: 64 bits. This timestamp reflects the wall clock time at the reference client at the moment it presented the data contained in the first octet of the associated RTP packet to the user. The timestamp is formatted based on the NTP timestamp format as specified in [RFC5905]. If this field is empty, then it SHALL be set to 0. This field MAY be left empty if none or only one of the receivers reported on presentation timestamps. Presented here means the moment the data is presented to the user of the system. Brandenburg Expires May 3, 2012 [Page 12] Internet-Draft RTCP for IDMS October 31, 2011 In some use cases (e.g. phased array transducers), the level of control a MSAS might need to have over the exact moment of playout is so precise that a 32bit Presentation Timestamp might not suffice. For this reason, this RTCP Packet Type for IDMS includes a 64bit Presentation Timestamp field. Since an MSAS will in practice always add some extra delay to the delay reported by the most lagged receiver (to account for packet jitter), it suffices for the IDMS XR Block Type with which the SCs report on their playout to have a 32bit Presentation Timestamp field. 8. Timing and NTP Considerations To achieve IDMS, the different receivers involved need synchronized clocks as a common timeline for synchronization. Depending on the synchronization accuracy required, different clock synchronization methods can be used. For social TV, synchronization accuracy should be achieved in order of hundreds of milliseconds. In that case, correct use of NTP on receivers will in most situations achieve the required accuracy. As a guideline, to deal with clock drift of receivers, receivers should synchronize their clocks at the beginning of a synchronized session. In case of high required accuracy, the synchronized clocks of different receivers should not drift beyond the accuracy required for the synchronization mechanism. In practice this can mean that receivers need to synchronize their clocks repeatedly during a synchronization session. Because of the stringent synchronization requirements for achieving good audio, a high accuracy will be needed. In this case, NTP usage may not be sufficient. Either a local NTP server could be setup, or some other more accurate clock synchronization mechanism could be used, such as using GPS time or the Precision Time Protocol [IEEE- 1588]. In this document, a new SDP parameter is introduced to signal the clock synchronization source or sources used or able to be used (see section 10). An SC can indicate which synchronization source is being used at the moment and the last time the SC synchronized with this source. An SC can also indicate any other synchronization sources available to it. This allows multiple SCs in an IDMS session to use the same or a similar clock synchronization source for their session. Applications performing IDMS may or may not be able to choose a synchronization method for the system clock. How applications deal with this is up to the implementation. The application might control the system clock, or it might use a separate application clock or even a separate IDMS session clock. It might also report on the system clock and the synchronization method used, without being able to change it. Brandenburg Expires May 3, 2012 [Page 13] Internet-Draft RTCP for IDMS October 31, 2011 8.1. Leap Seconds IDMS implementation is simplified by using a clock reference with a timescale which does not include leap seconds. IEEE 1588, GPS and other TAI (Inernational Atomic Time) references do not include leap seconds. NTP time, operating system clocks and other UTC (Coordinated Universal Time) references include leap seconds (though the ITU is studying a proposal which could eventually eliminate leap seconds from UTC). Leap seconds are potentially scheduled at the end of the last day of December and June each year. NTP inserts a leap second at the beginning of the last second of the day. This results in the clock freezing for one second immediately prior to the last second of the affected day. Most system clocks insert the leap second at the end of the last second. This results in repetition of the last second of the day. Generating or using timestamps during the entire last second of a day on which a leap second has been scheduled should therefore be avoided. Note that the period to be avoided has a real-time duration of two seconds. It is also important that all participants correctly implement leap seconds and have a working communications channel to receive notification of leap second scheduling. Without prior knowledge of leap second schedule, NTP servers and clients may be offset by exactly one second with respect to their UTC reference. This potential discrepancy begins when a leap second occurs and ends when all participants receive a time update from a server or peer (which, depending on the operating system and/or implementation, could be anywhere from a few minutes to a week). Such a long-lived discrepancy can be particularly disruptive to RTP and IDMS operation. Apart from the long-lived discrepancy due to dependence on both timing (e.g. NTP) updates and leap seconds scheduling updates, there is also the potential for a short-lived timing discontinuity having an effect on RTP and IDMS playout (even though leap seconds are quite rare). If a timescale with leap seconds is used for IDMS: - SCs must take care not to generate any IDMS XR reports in the immediate vicinity of the leap second. An MSAS must ignore any such reports that may be inadvertently generated. - RTP Senders using a leap-second-bearing reference must not generate sender reports (SR) containing an originating NTP timestamp in the vicinity of a leap second. Receivers should ignore timestamps in any such reports inadvertently generated. Brandenburg Expires May 3, 2012 [Page 14] Internet-Draft RTCP for IDMS October 31, 2011 - Receivers working to a leap-second-bearing reference must be careful to take leap seconds into account if a leap second occurs between the time a RTP packet is originated and when it is to be presented. 9. SDP Parameter for RTCP XR IDMS Block Type The SDP parameter sync-group is used to signal the use of the RTCP XR block for inter-destination media synchronization. It is also used to carry an identifier for the synchronization group to which clients belong or will belong. This SDP parameter extends rtcp-xr-attrib as follows, using Augmented Backus-Naur Form [RFC5234]. rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF ; Original definition from [RFC3611], section 5.1 xr-format =/ grp-sync ; Extending xr-format for inter-destination media synchronization grp-sync = "grp-sync" [",sync-group=" SyncGroupId] SyncGroupId = 1*DIGIT ; Numerical value from 0 till 4294967295 DIGIT = %x30-39 SyncGroupId is a 32-bit unsigned integer in network byte order and represented in decimal. SyncGroupId identifies a group of SCs for IDMS. It maps on the Media Stream Correlation Identifier as described in sections 6 and 7. The value SyncGroupId=0 represents an empty SyncGroupId. The value 4294967295 (2^32-1) is reserved for future use. The following is an example of the SDP attribute for IDMS a=rtcp-xr:grp-sync,sync-group=42 10. SDP Parameter for RTCP IDMS Packet Type The SDP parameter rtcp-idms is used to signal the use of the RTCP IDMS Packet Type for IDMS. It is also used to carry an identifier for the synchronization group to which clients belong or will belong. The SDP parameter is used as a media-level attribute during session setup. This SDP parameter is defined as follows, using Augmented Backus-Naur Form [RFC5234]. rtcp-idms = "a=" "rtcp-idms" ":" [sync-grp] CRLF sync-grp = "sync-group=" SyncGroupId Brandenburg Expires May 3, 2012 [Page 15] Internet-Draft RTCP for IDMS October 31, 2011 SyncGroupId = 1*DIGIT ; Numerical value from 0 till 4294967295 DIGIT = %x30-39 SyncGroupId is a 32-bit unsigned integer in network byte order and represented in decimal. SyncGroupId identifies a group of SCs for IDMS. The value SyncGroupId=0 represents an empty SyncGroupId. The value 4294967295 (2^32-1) is reserved for future use. The following is an example of the SDP attribute for IDMS. a=rtcp-idms:sync-group=42 11. SDP parameter for clock source The SDP parameter clocksource is used to signal the source for clock synchronization. This SDP parameter is specified as follows, using Augmented Backus-Naur Form [RFC5234]. clocksource = "a=" "clocksource" ":" source SP [last-synced] CRLF source = local / ntp / gps / gal / ptp local = "local" ntp = "ntp" ["=" ntp-server] ntp-server = host [ ":" port ] host = hostname / IPv4address / IPv6reference hostname = *( domainlabel "." ) toplabel [ "." ] domainlabel = alphanum / alphanum *( alphanum / "-" ) alphanum toplabel = ALPHA / ALPHA *( alphanum / "-" ) alphanum IPv4address = 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT IPv6reference = "[" IPv6address "]" IPv6address = hexpart [ ":" IPv4address ] hexpart = hexseq / hexseq "::" [ hexseq ] / "::" [ hexseq ] hexseq = hex4 *( ":" hex4) Brandenburg Expires May 3, 2012 [Page 16] Internet-Draft RTCP for IDMS October 31, 2011 hex4 = 1*4HEXDIG port = 1*DIGIT gps = "gps" gal = "gal" ptp = "ptp" SP ptp-version [ ":" ptp-id] ptp-version = "IEEE 1588-2002" / "IEEE 1588-2008" / "IEEE 802.1AS- 2011" ptp-id = 1*alphanum last-synced = date SP time UTCoffset date = 2DIGIT "-" 2DIGIT "-" 4DIGIT ; day month year (e.g., 02-06-1982) time = 2DIGIT ":" 2DIGIT ":" 2DIGIT "." 3DIGIT ; 00:00:00.000 - 23:59:59.999 UTCoffset = plusoffset / minusoffset plusoffset = + 2DIGIT ":" 2DIGIT ; +01:00 (+HH:MM) minusoffset = - 2DIGIT ":" 2DIGIT ; -00:00 (-HH:MM) alphanum = ALPHA / DIGIT EXAMPLE a=clocksource:ntp=139.63.192.5:123 19-02-2011 21:03:20.345+01:00 A client MAY include this attribute multiple times. If multiple time synchronization sources were used in the past, the client MUST only report the 'last synced' parameter on the latest synchronization performed. If a client supports a specific synchronization method, but does not know any sources to use for synchronization, it SHOULD indicate the method without specifying the source. A client MAY indicate itself as source if it is a clock synchronization source, Brandenburg Expires May 3, 2012 [Page 17] Internet-Draft RTCP for IDMS October 31, 2011 but it SHOULD do so using a publicly reachable address. The parameter can be used as both a session or media level attribute. It will normally be a session level parameter, since it is not directly media-related. In case of IDMS however, it can be used in conjunction with the rtcp-idms SDP parameter, and then it SHOULD be used as a media-level parameter as well. The meaning of 'local' is that no clock synchronization is performed. Allthough not re-used, the meaning of 'gps' (Global Positioning System), 'gal' (Galileo Positioning System) and ptp (Precision Time Protocol) follows the use of reference identifiers in NTP [RFC5905]. When using ptp, the ptp-id MUST contain the proper identifier from the used IEEE specification. The 'last synced' parameter is used as an indication for the receiver of the parameter on the accuracy of the clock. If the indicated last synchronization time is very recent, this is an indication that the clock can be trusted to be accurate, given the method of clock synchronization used. If the indicated last synchronization time is longer ago or in the future, either the clock synchronization has been performed long ago, or the clock is synchronized to an incorrect synchronization source. Either way, this shows that the clock used can not be trusted to be accurate. 12. Compatibility with ETSI TISPAN As described in section 1.4, ETSI TISPAN has also described a mechanism for IDMS in [TS 183 063]. One of the main differences between the TISPAN document and this document is the fact that the TISPAN solution uses an RTPC XR block for both the SC->MSAS message as well as for the MSAS->SC message (by selecting different SPST- types), while this document specifies a new RTCP Packet Type for the MSAS->SC message. The message from MSAS to SC is not in any way a report on how a receiver sees a session, and therefore a separate RTCP packet type is more appropriate then the XR block solution chosen in ETSI TISPAN. In order to maintain backward-compatibility, the RTCP XR block used for SC->MSAS signaling specified in this document is fully compatible with the TISPAN defined XR block. For the MSAS->SC signaling, it is recommended to use the RTCP IDMS Packet Type defined in this document. The TISPAN XR block with SPST=2 MAY be used for purposes of compatibility with the TISPAN solution, but MUST NOT be used if all nodes involved support the new RTCP IDMS Packet Type. Brandenburg Expires May 3, 2012 [Page 18] Internet-Draft RTCP for IDMS October 31, 2011 The above means that the IANA registry contains two SDP parameters for the MSAS->SC signaling; one for the ETSI TISPAN solution and one for the IETF solution. This also means that if all elements in the SDP negotiation support the IETF solution they SHOULD use the new RTCP IDMS Packet Type. 13. On the use of presentation timestamps A receiver can report on different timing events, i.e. on packet arrival times and on playout times. A receiver SHALL report on arrival times and a receiver MAY report on playout times. RTP packet arrival times are relatively easy to report on. Normally, the processing and play-out of the same media stream by different receivers will take roughly the same amount of time. By synchronizing on packet arrival times, you may loose some accuracy, but it will be adequate for many applications, such as social TV. Also, if the receivers are in some way controlled, e.g. having the same buffer settings and decoding times, high accuracy can be achieved. However, if all receivers in a synchronization session have the ability to report on, and thus synchronize on, actual playout times, or packet presentation times, this may be more accurate. It is up to applications and implementations of this RTCP extension whether to implement and use this. 14. Security Considerations The specified RTCP XR Block Type in this document is used to collect, summarize and distribute information on packet reception- and playout -times of streaming media. The information may be used to orchestrate the media play-out at multiple devices. Errors in the information, either accidental or malicious, may lead to undesired behavior. For example, if one device erroneously reports a two-hour delayed play-out, then another device in the same synchronization group could decide to delay its play-out by two hours as well, in order to keep its play-out synchronized. A user would likely interpret this two hour delay as a malfunctioning service. Therefore, the application logic of both Synchronization Clients and Media Synchronization Application Servers should check for inconsistent information. Differences in play-out time exceeding configured limits (e.g. more than ten seconds) could be an indication of such inconsistent information. No new mechanisms are introduced in this document to ensure confidentiality. Encryption procedures, such as those being suggested for a Secure RTP (SRTP) at the time that this document was written, can be used when confidentiality is a concern to end hosts. Brandenburg Expires May 3, 2012 [Page 19] Internet-Draft RTCP for IDMS October 31, 2011 15. IANA Considerations New RTCP Packet Types and RTCP XR Block Types are subject to IANA registration. For general guidelines on IANA considerations for RTCP XR, refer to [RFC3611]. [TS 183 063] assigns the block type value 12 in the RTCP XR Block Type Registry to "Inter-destination Media Synchronization Block". [TS 183 063] also registers the SDP [RFC4566] parameter "grp-sync" for the "rtcp-xr" attribute in the RTCP XR SDP Parameters Registry. Further, this document defines a new RTCP packet type called IDMS report. This new packet type is registered with the IANA registry of RTP parameters, based on the specification in section 7. Further, this document defines a new SDP parameter "rtcp-idms" within the existing IANA registry of SDP Parameters. The SDP attribute "rtcp-idms" defined by this document is registered with the IANA registry of SDP Parameters as follows: SDP Attribute ("att-field"): Attribute name: rtcp-idms Long form: RTCP report block for IDMS Type of name: att-field Type of attribute: media level Subject to charset: no Purpose: see sections 7 and 10 of this document Reference: this document Values: see this document Further, this document defines a new SDP attribute, "clocksource", within the existing IANA registry of SDP Parameters. The SDP attribute "clocksource" defined by this document is registered with the IANA registry of SDP Parameters as follows: SDP Attribute ("att-field"): Attribute name: clocksource Brandenburg Expires May 3, 2012 [Page 20] Internet-Draft RTCP for IDMS October 31, 2011 Long form: clock synchronization source Type of name: att-field Type of attribute: session level Subject to charset: no Purpose: see sections 8 and 11 of this document Reference: this document Values: see this document and registrations below The attribute has an extensible parameter field and therefore a registry for these parameters is required. This document creates an IANA registry called the Clocksource Source Parameters Registry. It contains the five parameters defined in Section 11: "local", "ntp", "gps", "gal" and "ptp". 16. Contributors The following people have participated as co-authors or provided substantial contributions to this document: Omar Niamut, Fabian Walraven, Ishan Vaishnavi, Rufael Mekuria 17. Conclusions This document describes the RTCP XR block type for IDMS, the RTCP IDMS report and the associated SDP parameters for inter-destination media synchronization. It also describes an SDP parameter for indicating which source is used for synchronizing a (systems) (wall) clock. Brandenburg Expires May 3, 2012 [Page 21] Internet-Draft RTCP for IDMS October 31, 2011 18. References 18.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC5234] Crocker, D., Ed., and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video Conferences with Minimal Control", STD 65, RFC 3551, July 2003. [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., "RTP Control Protocol Extended Reports (RTCP XR)", RFC 3611, November 2003. [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific Media Attributes in the Session Description Protocol (SDP)", RFC 5576, June 2009. [RFC5760] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control Protocol (RTCP) Extensions for Single-Source Multicast Sessions with Unicast Feedback", RFC 5760, February 2010. [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, June 2010. [TS 183 063] ETSI TISPAN, "IMS-based IPTV stage 3 specification", TS 183 063 v3.4.1, June 2010. 18.2. Informative References [RFC5968] Ott, J. and C. Perkins, "Guidelines for Extending the RTP Control Protocol (RTCP)", RFC 5968, September 2010. Brandenburg Expires May 3, 2012 [Page 22] Internet-Draft RTCP for IDMS October 31, 2011 [Boronat2009] Boronat, F., et al, "Multimedia group and inter-stream synchronization techniques: A comparative study", Elsevier Information Systems 34 (2009), pp. 108-131 [Ishibashi2006] Ishibashi Y. et al, "Subjective Assesment of Fairness among users in multipoint communications". Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, 2006. [IEEE-1588] IEEE Standards Association, "1588-2008 - IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems", 2008 Brandenburg Expires May 3, 2012 [Page 23] Internet-Draft RTCP for IDMS October 31, 2011 Authors' Addresses Ray van Brandenburg TNO Brassersplein 2, Delft, the Netherlands Phone: +31 88 86 63609 Email: ray.vanbrandenburg@tno.nl Hans M. Stokking TNO Brassersplein 2, Delft, the Netherlands Phone: +31 88 86 67278 Email: hans.stokking@tno.nl M. Oskar van Deventer TNO Brassersplein 2, Delft, the Netherlands Phone: +31 88 86 67078 Email: oskar.vandeventer@tno.nl Fernando Boronat IGIC Institute, Universidad Politecnica de Valencia-Campus de Gandia C/ Paraninfo, 1, Grao de Gandia, 46730, Valencia, Spain Phone: +34 962 849 341 Email: fboronat@dcom.upv.es Mario Montagud IGIC Institute, Universidad Politecnica de Valencia-Campus de Gandia C/ Paraninfo, 1, Grao de Gandia, 46730, Valencia, Spain Phone: +34 962 849 341 Email: mamontor@posgrado.upv.es Kevin Gross AVA Networks Brandenburg Expires May 3, 2012 [Page 24]