Internet Engineering Task Force J. Border INTERNET-DRAFT Hughes Network Systems M. Kojo University of Helsinki Jim Griner NASA Glenn Research Center G. Montenegro Sun Microsystems, Inc. December 3, 1999 Performance Enhancing Proxies draft-ietf-pilc-pep-01.txt Status of This Memo The document is an Internet-Draft and is in full conformance with all of the provisions of Section 10 of RFC 2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Intenet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Distribution of this draft is unlimited. Comments on this draft should be sent to the authors or to the PILC mailing list at pilc@grc.nasa.gov. This draft expires on June 3, 2000. Abstract This document provides a high level overview of Performance Enhancing Proxies. Motivations for their development and use are described as well as some of the consequences of using them. Expires June 3, 2000 [Page 1] INTERNET DRAFT Performance Enhancing Proxies December 1999 Table of Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Types of Performance Enhancing Proxies . . . . . . . . . . . . . . 4 2.1 Layering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1.1 Transport Layer PEPs . . . . . . . . . . . . . . . . . . . . . 4 2.1.2 Application Layer PEPs . . . . . . . . . . . . . . . . . . . . 5 2.2 Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Asymmetric vs. Symmetric PEP . . . . . . . . . . . . . . . . . . 6 2.4 Split Connections . . . . . . . . . . . . . . . . . . . . . . . 6 2.5 Transparency . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. PEP Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1 TCP ACK Handling . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1.1 TCP ACK Spacing . . . . . . . . . . . . . . . . . . . . . . . 8 3.1.2 Local TCP Acknowledgements . . . . . . . . . . . . . . . . . . 8 3.1.3 Local TCP Retransmissions . . . . . . . . . . . . . . . . . . 9 3.2 Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.3 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.4 Handling Periods of Link Disconnection with TCP . . . . . . . . 11 3.5 Priority-based Multiplexing . . . . . . . . . . . . . . . . . . 11 3.6 Other Link Specific Enhancements . . . . . . . . . . . . . . . . 12 3.6.1 Protocol Booster Mechanisms . . . . . . . . . . . . . . . . . 12 3.6.2 TCP ACK Filtering and Reconstruction . . . . . . . . . . . . . 12 3.6.3 Other Possible Mechanisms . . . . . . . . . . . . . . . . . . 12 4 Implications of Using PEP . . . . . . . . . . . . . . . . . . . . 13 4.1 The End-to-end Argument . . . . . . . . . . . . . . . . . . . . 13 4.1.1 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.1.2 Fate Sharing . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.1.3 End-to-end Reliability . . . . . . . . . . . . . . . . . . . . 15 4.1.4 End-to-end Failure Diagnostics . . . . . . . . . . . . . . . . 16 4.2 Asymmetric Routing . . . . . . . . . . . . . . . . . . . . . . . 16 4.3 Mobile Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.4 Other Implications . . . . . . . . . . . . . . . . . . . . . . . 17 4.4.1 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.4.2 Multi-Homing Environments . . . . . . . . . . . . . . . . . . 17 4.4.3 QoS Transparency . . . . . . . . . . . . . . . . . . . . . . . 17 4.4.4 Others . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5 PEP Environment Examples . . . . . . . . . . . . . . . . . . . . . 18 5.1 VSAT Environments . . . . . . . . . . . . . . . . . . . . . . . 18 5.1.1 VSAT Network Characteristics . . . . . . . . . . . . . . . . . 18 5.1.2 VSAT Network PEP Implementations . . . . . . . . . . . . . . . 19 5.1.3 VSAT Network PEP Motivation . . . . . . . . . . . . . . . . . 20 5.2 W-WAN Environments . . . . . . . . . . . . . . . . . . . . . . . 21 5.2.1 W-WAN Network Characteristics . . . . . . . . . . . . . . . . 21 5.2.2 W-WAN PEP Implementations . . . . . . . . . . . . . . . . . . 21 5.3 W-LAN Environments . . . . . . . . . . . . . . . . . . . . . . . 22 5.3.1 W-LAN Network Characteristics . . . . . . . . . . . . . . . . 22 5.3.2 W-LAN PEP Implementations: Snoop . . . . . . . . . . . . . . . 23 6 Security Considerations . . . . . . . . . . . . . . . . . . . . . 25 7 Appendix - PEP Terminology Summary . . . . . . . . . . . . . . . . 25 7.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 26 8 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . 28 9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31 11 Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 32 Expires June 3, 2000 [Page 2] INTERNET DRAFT Performance Enhancing Proxies December 1999 1 Introduction The Transmission Control Protocol [RFC0793] (TCP) is used as the transport layer protocol by many Internet and intranet applications. However, in certain environments, TCP and other higher layer protocol performance is limited by the link characteristics of the environment. [Karn99] discusses various link layer design considerations that should be taken into account when designing a link layer service that is intended to support the Internet protocols. Such design choices may have a significant influence on the performance and efficiency of the Internet. However, not all link characteristics, for example, high latency, can be compensated for by such choices in the link layer design. And, the cost of compensating for some link characteristics may be prohibitive for some technologies. A Performance Enhancing Proxy (PEP) is used to improve the performance of the Internet protocols on network paths where native performance suffers due to characteristic of a link or subnetwork on the path. This document does not intend to advocate use of PEPs in general. On the contrary, we believe that the end-to-end principle in designing Internet protocols should be retained as the prevailing approach and PEPs should be used only in specific environments and circumstances where end-to-end mechanisms providing similar performance enhancements are not available. In any environment where one might consider employing PEP for improved performance, an end user should be aware of the PEP and the choice of employing PEP functionality should be under the control of the end user, especially if employing the PEP would interfere with end-to-end usage of IP layer security mechanisms or otherwise have undesirable implications in some circumstances. This would allow the user to choose end-to-end IP at all times but, of course, without performance enhancements that employing the PEP may yield. The remainder of this document is organized as follows. Section 2 provides an overview of different kinds of PEP implementations. Section 3 discusses some of the mechanisms which PEPs may employ in order to improve performance. Section 4 discusses some of the implications with respect to using PEPs, especially in the context of the global Internet. Finally, Section 5 discusses some example environments where PEPs are used: satellite very small aperture terminal (VSAT) environments, mobile wireless WAN (W-WAN) environments and wireless LAN (W-LAN) environments. A summary of PEP terminology is included in an appendix (Section 7). NOTE: This is a working draft and it may fail to cover many important aspects related to PEPs. In particular, this version does not Expires June 3, 2000 [Page 3] INTERNET DRAFT Performance Enhancing Proxies December 1999 necessarily list all the possible implications of using PEPs nor does the included text on each of the implications cover all the aspects related to the particular implication. Any suggestions to improve the text are solicited. 2 Types of Performance Enhancing Proxies There are many types of Performance Enhancing Proxies. Different types of PEPs are used in different environments to overcome different link characteristics which affect protocol performance. Note that enhancing performance is not necessarily limited in scope to throughput. Other performance related aspects, like usability of a link, may also be addressed. For example, [M-TCP] addresses the issue of keeping TCP connections alive during periods of disconnection in wireless networks. The following sections describe some of the key characteristics which differentiate different types of PEPs. 2.1 Layering A PEP implementation may function at the transport layer or at the application layer. In principle, a PEP implementation may function at other layers (e.g., at the network layer) as well. Such PEPs, however, are out of scope of this (version of the) document. 2.1.1 Transport Layer PEPs Transport layer PEPs operate at the transport level. They may be aware of the type of application being carried by the transport layer but, at most, only use this information to influence their behaviour with respect to the transport protocol; they do not modify the application protocol in any way, but let the application protocol operate end-to-end. Most transport layer PEP implementations interact with TCP. Such an implementation is called a TCP Performance Enhancing Proxy (TCP PEP). For example, in an environment where ACKs may bunch together, a TCP proxy may be used to simply modify the ACK spacing in order to improve performance. On the other hand, in an environment with a large bandwidth*delay product, a TCP proxy may be used to alter the behaviour of the TCP connection by generating local acknowledgements to TCP data segments in order to improve the connection's throughput. (The term TCP spoofing is sometimes used synonymously for TCP PEP functionality. However, the term TCP spoofing more accurately Expires June 3, 2000 [Page 4] INTERNET DRAFT Performance Enhancing Proxies December 1999 applies to only a subset of TCP PEP implementations.) 2.1.2 Application Layer PEPs Application layer PEPs operate above the transport layer. Today, different kinds of application layer proxies are widely used in the Internet. Such proxies include Web caches and relay Mail Transfer Agents (MTA) and they typically try to improve performance or service availability and reliability in general and in a way which is applicable in any environment but they do not necessarily include any optimizations that are specific to certain link characteristics. Application layer PEPs, on the other hand, can be implemented to improve application protocol as well as transport layer performance with respect to a particular application being used with a particular type of link. An application layer PEP may have the same functionality as the corresponding regular proxy for the same application (e.g., relay MTA or Web caching proxy) but extended with link-specific optimizations of the application protocol operation. Some application protocols employ extraneous round trips, overly verbose headers and/or inefficient header encoding which may have a significant impact on performance, in particular, with long delay and slow links. This unnecessary overhead can be reduced, in general or for a particular type of link, by using an application layer PEP in an intermediate node. Some examples of application layer PEPs which have been shown to improve performance on slow wireless WAN links are described in [LHKR96] and [CTC+97]. 2.2 Distribution A PEP implementation may be integrated, i.e., it comprises a single PEP component implemented within a single node, or distributed, i.e., it comprises two or more PEP components, typically implemented in multiple nodes. An integrated PEP implementation represents a single point at which performance enhancement is applied. For example, a single PEP component might be implemented to provide impedance matching at the point where wired and wireless links meet. A distributed PEP implementation is generally used to surround a particular link for which performance enhancement is desired. For example, a PEP implementation for a satellite connection may be distributed between two PEPs located at each end of the satellite link. Expires June 3, 2000 [Page 5] INTERNET DRAFT Performance Enhancing Proxies December 1999 2.3 Asymmetric vs. Symmetric PEP A PEP implementation may be symmetric or asymmetric. Symmetric PEPs use identical behaviour in both directions. Asymmetric PEPs operate differently in each direction. The direction can be defined in terms of link (e.g., uplink or downlink) or in terms of protocol traffic (e.g., the direction of TCP data flow, often called TCP data channel, or the direction of TCP ACK flow, often called ACK channel). (A distributed PEP may have yet another type of asymmetry. A symmetric implementation uses two or more PEPs with essentially identical behaviour and implemantation. An asymmetric implementation uses two or more PEPs which behave differently and have different implementation. Such an asymmetric distributed PEP implementation generally used with an asymmetric link and uses two PEPs with is different implementation at each side of the link.) 2.4 Split Connections A split connection TCP implementation terminates the TCP connection received from an end system and establishes a corresponding TCP connection to the other end system. In a distributed PEP implementation, this is typically done to allow the use of a third connection between two PEPs optimized for the link. This might be a TCP connection optimized for the link or it might be another protocol, for example, a proprietary protocol running on top of UDP. Also, the distributed implementation might use a separate connection between the proxies for each TCP connection or it might multiplex the data from multiple TCP connections across a single connection between the PEPs. In an integrated PEP split connection TCP implementation, PEP again terminates the connection from one end system and originates a separate connection to the other end system. [I-TCP] documents an example of a single PEP split connection implementation. Many integrated PEPs use a split connection implementation in order to address a mismatch in TCP capabilities between two end systems. For example, the TCP window scaling option [RFC1323] can be used to extend the maximum amount of TCP data which can be "in flight" (i.e., sent and awaiting acknowledgement). This is useful for filling a link which has a high bandwidth*delay product. If one end system is capable of using scaled TCP windows but the other is not, the end system which is not capable can set up its connection with PEP on its side of the high bandwidth*delay link. Split connection PEP then sets up a TCP connection with window scaling over the link to the other end system. Expires June 3, 2000 [Page 6] INTERNET DRAFT Performance Enhancing Proxies December 1999 Split connection TCP implementations can effectively leverage TCP performance enhancements optimal for a particular link but which cannot necessarily be employed safely over the global Internet. Note that using split connection PEPs does not necessarily exclude simultaneous use of IP for end-to-end connectivity. If a split connection is managed per application or per connection and is under the control of the end user, the user can decide whether a particular TCP connection or application makes use of split connection PEP or whether it operates end-to-end. When PEP is employed on a last hop link, the end user control is relatively easy to implement. In effect, application layer proxies for TCP-based applications are split connection TCP implementations with end systems using PEPs as a service related to a particular application. Therefore, all transport (TCP) layer enhancements that are available with split connection TCP implementations can also be employed with application layer PEPs in conjunction with application layer enhancements. 2.5 Transparency Another key characteristic of a PEP is its degree of transparency. PEPs may operate totally transparently to the end systems, transport endpoints, and/or applications involved (in a connection), requiring no modifications to the end systems, transport endpoints, or applications. On the other hand, a PEP implementation may require modifications to both ends in order to be used. In between, a PEP implementation may require modifications to only one of the ends involved. Either of this kind of PEP implementations is non-transparent, at least to the layer requiring modification. It is sometimes useful to think of the degree of transparency of a PEP implementation at four levels, transparency with respect to the end systems (network-layer transparent PEP), transparency with respect to the transport endpoints (transport-layer transparent PEP), transparency with respect to the applications (application-layer transparent PEP) and transparency with respect to the users. For example, a user who subscribes to a satellite Internet access service may be aware that the satellite terminal is providing a performance enhancing service even though the TCP/IP stack and the applications in the user's PC are not aware of PEP which implements it. Note that the issue of transparency is not the same as the issue of maintaining the end-to-end semantics. For example, a PEP implementation which simply uses a TCP ACK spacing mechanism Expires June 3, 2000 [Page 7] INTERNET DRAFT Performance Enhancing Proxies December 1999 maintains the end-to-end semantics of the TCP connection while a split connection PEP implementation may not. Yet, both can be implemented transparently to the transport endpoints at both ends. The implications of not maintaining the end-to-end semantics, in particular the end-to-end semantics of TCP connections, are discussed in Section 4. 3. PEP Mechanisms An obvious key characteristic of a PEP implementation is the mechanism(s) it uses to improve performance. Some examples of PEP mechanisms are described in the following subsections. A PEP implementation might implement more than one of these mechanisms. 3.1 TCP ACK Handling Many TCP PEP implementations are based on TCP ACK manipulation. The handling of TCP acknowledgements can differ significantly between different TCP PEP implementations. The following subsections describe various TCP ACK handling mechanisms. Many implementations combine some of these mechanisms and possibly employ some additional mechanisms as well. 3.1.1 TCP ACK Spacing Some TCP PEP implementations are concerned only with manipulating TCP acknowledgements. ACK spacing is used to smooth out the flow of TCP acknowledgements traversing a link in order to improve performance by eliminating bursts of TCP data segments [Part98]. 3.1.2 Local TCP Acknowledgements In some PEP implementations, TCP data segments received by the PEP are locally acknowledged by the PEP. This is very useful over network paths with a large bandwidth*delay product as it speeds up TCP slow start and allows the sending TCP to quickly open up its congestion window. Local acknowledgements are automatically employed with split connection TCP implementations. When local acknowledgements are used, the burden falls upon the TCP PEP to recover any data which is dropped after the PEP acknowledges it. Expires June 3, 2000 [Page 8] INTERNET DRAFT Performance Enhancing Proxies December 1999 3.1.3 Local TCP Retransmissions A TCP PEP may locally retransmit data segments lost on the path between the TCP PEP and the receiving end system, thus aiming at faster recovery from lost data. In order to achieve this the TCP PEP may use acknowledgements arriving from the end system that receives the TCP data segments, along with appropriate timeouts, to determine when to locally retransmit lost data. TCP PEPs sending local acknowledgements to the sending end system, are required to employ local retransmissions towards the receiving end system. Some PEP implementations perform local retransmissions even though they do not use local acknowledgements to alter TCP connection performance. Basic Snoop [SNOOP] is a well know example of such a PEP implementation. Snoop caches TCP data segments it receives and forwards and then monitors the acknowledgements coming from the receiving TCP end system for duplicate acknowledgements (DUPACKs). When DUPACKs are received, Snoop locally retransmits the lost TCP data segments from its cache, suppressing the DUPACKs flowing to the sending TCP end system until acknowledgements for new data are received (See Section 5.2 for details.) 3.2 Tunneling A Performance Enhancing Proxy may encapsulate messages to carry the messages across a particular link. PEP at the other end of the encapsulation tunnel removes the tunnel wrappers before final delivery to the receiving end system. A tunnel might be used by a distributed split connection TCP implementation as the means for connecting split connection PEPs. A tunnel might also be used to support forcing TCP connections which use asymmetric routing to go through the end points of a distributed PEP implementation. 3.3 Compression Many PEP implementations include support for one or more forms of compression. In some PEP implementations, compression may even be the only mechanism used for performance improvement. Compression reduces the number of bytes which need to be sent across a link. This is useful in general and can be very important for bandwidth limited links. Some of the benefits of using compression include: - Improved link efficiency; Expires June 3, 2000 [Page 9] INTERNET DRAFT Performance Enhancing Proxies December 1999 - Higher effective link utilization; - Reduced latency; - Improved interactive response time; - Decreased overhead; - Reduced packet loss rate over lossy links. Where appropriate, link layer compression is used. TCP and IP header compression are also frequently used with PEP implementations. [RFC1144] describes a widely deployed method for compressing TCP headers. Other header compression algorithms are described in [RFC2507], [RFC2508] and [RFC2509]. Payload compression is also desirable and is increasing in importance with today's increased emphasis on Internet security. Network (IP) layer (and above) security mechanisms convert IP payloads into random bit streams which defeat applicable link layer compression mechanisms by removing or hiding redundant "information." Therefore, compression of the payload needs to be applied before security mechanisms are applied. [RFC2393] defines a framework where common compression algorithms can be applied to arbitrary IP segment payloads. However, [RFC2393] compression is not always applicable. Many types of IP payloads (e.g. images, audio, video and "zipped" files being transferred) are already compressed. And, when security mechanisms such as TLS [RFC2246] are applied above the network (IP) layer, the data is already compressed; resulting additional transport or network layer compression will compact only those headers, which are small, and possibly already covered by separate compression algorithms of their own. With application layer PEPs one can employ application-specific compression. In particular, with slow links any compression that effectively reduces transfer volume is tremendously useful. Typically an application-specific (or content-specific) compression mechanism is much more efficient than any generic compression mechanism. For example, a distributed Web PEP implementation may implement more efficient binary encoding of HTTP headers, or PEP can employ lossy compression that reduces the image quality of inline-images on Web pages according to end user instructions, thus reducing the number of bytes transferred over the slow link and consequently the response time perceived by the user [LHKR96]. Expires June 3, 2000 [Page 10] INTERNET DRAFT Performance Enhancing Proxies December 1999 3.4 Handling Periods of Link Disconnection with TCP Periods of link disconnection or link outage are very common with some wireless links. During these periods, a TCP sender does not receive the expected acknowledgements. Upon expiration of the retransmit timer, this causes TCP to close its congestion window with all of the related drawbacks. A TCP PEP may monitor the traffic coming from the TCP sender towards the TCP receiver behind the disconnected link. The TCP PEP retains the last ACK, so that it can shut down the TCP sender's window by sending the last ACK with a window set to zero. Thus, the TCP sender will go into persist mode. To make this work in both directions with integrated TCP PEP implementation, the TCP receiver behind the disconnected link must be aware of the current state of the connection and, in the event of a disconnection, it must be capable of freezing all timers. [M-TCP] implements such operation. Another possibility is that the disconnected link is surrounded by a distributed PEP pair. In split connection TCP implementations, a period of link disconnection can easily be hidden from the end host on the other side of PEP thus precluding the TCP connection from breaking even if the period of link disconnection lasts a very long time. Consequently, the proxy and its counterpart behind the disconnected link can employ a modified TCP version which retains the state and all unacknowledged data segments across the period of disconnection and then performs local recovery as the link is reconnected. The period of link disconnection may or may not be hidden from the application and user, depending upon what application the user is using the TCP connection for. 3.5 Priority-based Multiplexing Implementing priority-based multiplexing of data with a slow (expensive) link may improve the usability of the link and performance for selected applications or connectios. A user behind a slow link would experience the link more feasible to use in case of simultaneous data transfers, if urgent data transfers (e.g., interactive connections) could have shorter response time (better performance) than less urgent transfers. This kind of operation can be controlled by assigning different priorities for different connections (or applications). In flight TCP segments of an end-to-end TCP connection (with low priority) can not be delayed for a long time. Otherwise, the TCP timer at the sending end would expire, resulting in suboptimal Expires June 3, 2000 [Page 11] INTERNET DRAFT Performance Enhancing Proxies December 1999 performance. A split connection PEP implementation allows a PEP in an intermediate node to reschedule freely the order in which it forwards data of different connections to the destination host behind the slow link. This can further be assisted, if the protocol stacks on both sides of the slow link implement priority based scheduling of connections. With such a PEP implementation together with user-controlled priorities the user can assign higher priority for some interactive connection(s) and in this way have much shorter response time for selected connections, even if there are simultaneous low priority bulk data transfers (which would in regular end-to-end operation eat almost all available bandwidth of the slow link). These low priority bulk data transfers would then proceed nicely during the idle periods of interactive connections, allowing the user to keep the slow and expensive link (e.g., wireless WAN) fully utilized. 3.6 Other Link Specific Enhancements < Editor's comment: the following subsections provide placeholders for describing other link specific enhancements. Any help is appreciated and contributions on these subjects are solicited. > 3.6.1 Protocol Booster Mechanisms A number of possible protocol booster mechanisms are described in [FMSBMR98]. 3.6.2 TCP ACK Filtering and Reconstruction < Editor's note: the upcoming text for this subsection is to be moved under the section 3.1. > On paths with highly asymmetric bandwidth the TCP ACKs flowing on the low-speed direction may get congested if the asymmetry ratio is high enough. This issue is discussed in [AGG+99] and in a companion PILC document on Implications of Network Asymmetry [BaPa99]. 3.6.3 Other Possible Mechanisms < Editor's note: contributions describing other mechanisms are solicited. > Expires June 3, 2000 [Page 12] INTERNET DRAFT Performance Enhancing Proxies December 1999 4 Implications of Using PEP The following sections describe some of the implications of using PEP. 4.1 The End-to-end Argument As indicated in [RFC1958], the end-to-end argument [SRC84] is one of the architectural principles of the Internet. The basic argument is that, as a first principle, certain required end-to-end functions can only be correctly performed by the end systems themselves. Most of the negative implications associated with using PEPs are related to the possibility of breaking the end-to-end semantics of connections. This is one of the main reasons why PEPs are not recommended for general use. As indicated in Section 2.5, not all PEP implementations break the end-to-end semantics of connections. Correctly designed PEPs do not attempt to replace any application level end-to-end function, but only attempt to add performance optimizations to a subpath of the end-to-end path between the application endpoints. Doing this can be consistent with the end-to-end argument. 4.1.1 Security The most detrimental negative implication of breaking the end-to-end semantics of a connection is that it disables end-to-end use of network (IP) layer security (IPsec). If, on the other hand, IPsec is employed end-to-end, it precludes PEPs from working because they need to examine transport or application headers but encryption of IP packets via IPsec's ESP header (in either transport or tunnel mode) renders the TCP header and payload unintelligible to intermediate PEPs. However, if an end user can select end-to-end IP for the IPsec traffic and use a PEP for other traffic, the problem is considerably alleviated although the encrypted traffic is not subject to possible performance enhancements while the other traffic is. If a PEP implementation is non-transparent to the users and the users trust the PEP in the middle, IPsec can be used separately between each end system and PEP. However, in most cases this is an undesirable or unacceptable alternative as the end systems can not trust PEPs in general. In addition, this is not as secure as end-to-end security. And, it can lead to potentially misleading security level assumptions by the end systems. If the two end systems negotiate different levels of security with the PEP, the end system which negotiated the stronger level of security may not be aware that Expires June 3, 2000 [Page 13] INTERNET DRAFT Performance Enhancing Proxies December 1999 a lower level of security is being provided for part of the connection. The PEP could be implemented to prevent this from happening by being smart enough to force the same level of security to each end system. With a transparent PEP implementation, it is difficult for the end systems to trust the PEP because they may not be aware of its existence. However, IPsec can be implemented between the two PEPs of a distributed PEP implementation. And, if the PEP implementation is non-transparent to the users, the users could configure their end systems to use PEPs as the end points of an IPsec tunnel. In any case, the authors are currently not aware of any PEP implementations, transparent or non-transparent, which provide support for IPsec. Note that even when a PEP implementation does not break the end-to-end semantics of a connection, the PEP implementation may not be able to function in the presence of IPsec. For example, it is difficult to do ACK spacing if the proxy cannot reliably determine which IP packets contain ACKs of interest. In most cases, security applied above the transport layer can be used with PEPs, especially transport layer PEPs. 4.1.2 Fate Sharing Another important aspect of the end-to-end argument is fate sharing. If a failure occurs in the network, the ability of the connection to survive the failure depends upon how much state is being maintained on behalf of the connection in the network and whether the state is self-healing. If no connection specific state resides in the network or such state is self-healing as in case of regular end-to-end operation, then a failure in the network will break the connection only if there is no alternate path through the network between the end systems. And, if there is no path, both end systems can detect this. However, if the connection depends upon some state being stored in the network (e.g. in a PEP), then a failure in the network (e.g. the node containing a PEP crashes) causes this state to be lost, forcing the connection to terminate even if an alternate path through the network exists. The importance of this aspect of the end-to-end argument with respect to PEPs is very implementation dependent. Sometimes coincidentally but more often by design, PEPs are used in environments where there is no alternate path between the end systems and, therefore, a Expires June 3, 2000 [Page 14] INTERNET DRAFT Performance Enhancing Proxies December 1999 failure of the intermediate node containing a PEP would result in the termination of the connection in any case. And, even when this is not the case, the risk of losing the connection in the case of regular end-to-end operation may exist as the connection could break for some other reason, for example, a long enough link outage of a last-hop wireless link to the end host. Therefore, the users may choose to accept the risk of a PEP crashing in order to take advantage of the performance gains offered by the PEP implementation. Note that accepting the risk must be under the control of the user and the user must always have the option to choose end-to-end operation. 4.1.3 End-to-end Reliability Another aspect of the end-to-end argument is that of acknowledging the receipt of data end-to-end in order to achieve reliable end-to-end delivery of data. An application aiming at reliable end-to-end delivery must implement an end-to-end check and recovery at the application level. According to the end-to-end argument, this is the only possibility to correctly implement reliable end-to-end operation. Otherwise the application violates the end-to-end argument. This also means that a correctly designed application can never fully rely on the transport layer (e.g., TCP) or any other communication subsystem to provide reliable end-to-end delivery. First, a TCP connection may break down for some reason and result in lost data that must be recovered at the application level. Second, the checksum provided by TCP may be considered inadequate, resulting in undetected data corruption [Pax99] and requiring application level check for data corruption. Third, a TCP acknowledgement only indicates that data was delivered to the TCP implementation on the other end system. It does not guarantee that the data was delivered to the application layer on the other end system. Therefore, a well designed application must use an application layer acknowledgement to ensure end-to-end delivery of application layer data. Note that this does not diminish the value of a reliable transport protocol (i.e., TCP) as such a protocol allows efficient implementation of several essential functions (e.g., congestion control) for an application. If a PEP implementation acknowledges application data prematurely (before the PEP receives application ACK from the other endpoint), end-to-end reliability cannot be guaranteed. Typically, application layer PEPs do not acknowledge data prematurely. Some Internet applications do not necessarily operate end-to-end in their regular operation, thus abandoning any end-to-end reliability guarantee. For example, Internet email delivery often operates via relay MTAs (relay SMTP servers): an originating MTA (SMTP server) Expires June 3, 2000 [Page 15] INTERNET DRAFT Performance Enhancing Proxies December 1999 sends the mail message to a relay MTA that receives the mail message, stores it in non-volatile storage (e.g., on disk) and then sends an application level acknowledgement. The relay MTA then takes "full responsibility" for delivering the mail message to the destination SMTP server (maybe via another relay MTA); it tries to forward the message for a relatively long time (typically around 5 days). This scheme does not give a 100% guarantee of email delivery, but reliability is considered "good enough". An application layer PEP for this kind of an application may acknowledge application data (mail message) without essentially decreasing reliability, as long as PEP operates according to the same procedure as a relay MTA. Transport layer PEP implementations, including TCP PEPs, generally do not interfere with end-to-end application layer acknowledgements as they let applications to operate end-to-end. 4.1.4 End-to-end Failure Diagnostics - Implications due to PEPs breaking the end-to-end failure diagnostics. < Editor's note: contributions providing text are solicited > 4.2 Asymmetric Routing Deploying a PEP implementation requires that traffic to and from the end hosts be routed through the intermediate node(s) where PEPs reside. With some networks, this cannot be accomplished, or it might require that the intermediate node is located several hops away from the target link edge which in turn is unpractical in many cases and may result in non-optimal routing. 4.3 Mobile Hosts In mobile host environments where a PEP implementation is used to serve mobile hosts, additional problems are encountered as the PEP related state information should be transferred to the new PEP node during a handoff. When a mobile host moves, it is subject to handovers by the serving base station. If the base station acts as the intermediate node and home for the serving PEP, any state information that the PEP maintains and is required for continuous operation must be transferred to the new intermediate node to ensure continued operation of the connection. This requires extra work and causes overhead. If the mobile host moves to another IP network, routing Expires June 3, 2000 [Page 16] INTERNET DRAFT Performance Enhancing Proxies December 1999 to and from the mobile host may need to be changed to traverse the new PEP node. In most W-WAN wireless networks today, unlike W-LANs, the W-WAN base station does not provide the mobile host with the connection point to the wireline Internet (such base stations may not even have an IP stack). Instead, the W-WAN network takes care of the mobility and retains the connection point to the wireline Internet unchanged while the mobile host moves. Thus, PEP state handover is not required in most W-WANs when the host moves. 4.4 Other Implications The following subsections describe other implications of using PEPs. < Editor's note: text for the subsections to be added in later versions. > 4.4.1 Scalability - PEPs require more work and therefore will always be (at least) one step behind routers. The higher the link bandwidth and the number of connections (packets) traversing through PEP is, more likely it is that performance becomes an issue. 4.4.2 Multi-Homing Environments - the effect of multi-homing environments < Editor's note: contributions providing text are solicited > 4.4.3 QoS Transparency - QoS transparency implications < Editor's note: contributions providing text are solicited > 4.4.4 Others - other possible implications < Editor's note: contributions addressing other implications and providing text are solicited > Expires June 3, 2000 [Page 17] INTERNET DRAFT Performance Enhancing Proxies December 1999 5 PEP Environment Examples The following sections describe examples of environments where PEP is currently used to improve performance. 5.1 VSAT Environments Today, VSAT networks are implemented with geosynchronous satellites. VSAT data networks are typically implemented using a star topology. A large hub earth station is located at the center of the star with VSATs used at the remote sites of the network. Data is sent from the hub to the remote sites via an outroute. Data is sent from the remote sites to the hub via one or more inroutes. VSATs represent an environment with highly asymmetric links, with an outroute typically much larger than an inroute. Multiple inroutes can be used with each outroute but any particular VSAT only has access to a single inroute at a time. VSAT networks are generally used to implement private networks (i.e. intranets) for enterprises (e.g. corporations) with geographically dispersed sites. VSAT networks are rarely, if ever, used to implement Internet connectivity except at the edge of the Internet (i.e. as the last hop). Connection to the Internet for the VSAT network is usually implemented at the VSAT network hub site using appropriate firewall and (when necessary) NAT [RFC2663] devices. 5.1.1 VSAT Network Characteristics With respect to TCP performance, VSAT networks exhibit the following subset of the satellite characteristics documented in [RFC2488]: Long feedback loops Propagation delay from a sender to a receiver in a geosynchronous satellite network can range from 240 to 280 milliseconds, depending on where the sending and receiving sites are in the satellite footprint. This makes the round trip time just due to propagation delay at least 480 milliseconds. Queueing delay and delay due to shared channel access methods can sometimes increase the total delay up to the order of a few seconds. Large bandwidth*delay products VSAT networks can support capacity ranging from a few kilobits per second up to multiple megabits per second. When combined Expires June 3, 2000 [Page 18] INTERNET DRAFT Performance Enhancing Proxies December 1999 with the relatively long round trip time, TCP needs to keep a large number of packets "in flight" in order to fully utilize the satellite link. Asymmetric capacity As indicated above, the outroute of a VSAT network is usually significantly larger than an inroute. Even though multiple inroutes can be used within a network, a given VSAT can only access one inroute at a time. Therefore, the incoming (outroute) and outgoing (inroute) capacity for a VSAT is often very asymmetric. As outroute capacity has increased in recent years, ratios of 400 to 1 or greater are becoming more and more common. With a TCP maximum segment size of 1460 bytes and delayed acknowledgements [RFC1122] in use, the ratio of IP packet bytes for data to IP packet bytes for ACKs is only (3000 to 40) 75 to 1. Thus, inroute capacity for carrying ACKs can have a significant impact on TCP performance. With respect to the other satellite characteristics listed in [RFC2488], VSAT networks typically do not suffer from intermittent connectivity or variable round trip times. Also, VSAT networks generally include a significant amount of error correction coding. This makes the bit error rate very low during clear sky conditions, approaching the bit error rate of a typical terrestrial network. In severe weather, the bit error rate may increase significantly but such conditions are rare (when looked at from a network availability point of view) and VSAT networks are generally engineered to work during these conditions but not to optimize performance during these conditions. 5.1.2 VSAT Network PEP Implementations Performance Enhancing Proxies implemented for VSAT networks generally focus on improving throughput (for applications such as FTP and HTTP web page retrievals). To a lesser degree, PEP implementations also work to improve interactive response time for small transactions. There is not a dominant PEP implementation used with VSAT networks. Each VSAT network vendor tends to implement their own version of PEP functionality, integrated with the other features of their VSAT product. [HNS] and [SPACENET] describe VSAT products with integrated PEP capabilities. There are also third party PEP implementations designed to be used with VSAT networks. These products run on nodes external to the VSAT network at the hub and remote sites. SatBooster [FLASH] and Venturi [FOURELLE] are examples of such products. Expires June 3, 2000 [Page 19] INTERNET DRAFT Performance Enhancing Proxies December 1999 VSAT network PEP implementations generally share the following characteristics: - They focus on improving TCP performance; - They use an asymmetric distributed implementation; - They use a split connection approach with local acknowledgements and local retransmissions; - They support some form of compression to reduce the amount of bandwidth required (with emphasis on saving inroute bandwidth). The key differentiators between VSAT network PEP implementations are: - The maximum throughput they attempt to support (mainly a function of the amount of buffer space they use); - The protocol used over the satellite link. Some implementations use a modified version of TCP while others use a proprietary protocol running on top of UDP; - The type of compression used. Third party VSAT network PEP implementations generally focus on application (e.g. HTTP) specific compression algorithms while PEP implementations integrated into the VSAT network generally focus on link specific compression. PEP implementations integrated into a VSAT product are generally transparent to the end systems. Third party PEP implementations used with VSAT networks are usually translucent, requiring a configuration change in the remote site end system to route TCP packets to the remote site proxy. In some cases, the PEP implementation is actually integrated transparently into the end system node itself, using a "bump in the stack" approach. In all cases, the use of a PEP is non-transparent to the user, i.e. the user is aware when a PEP implementation is being used to boost performance. 5.1.3 VSAT Network PEP Motivation TBD (in later versions): - Intranet versus Internet - Highly asymmetric links Expires June 3, 2000 [Page 20] INTERNET DRAFT Performance Enhancing Proxies December 1999 - Support for "non-modern" TCP stacks 5.2 W-WAN Environments < Editor's note: text covering W-WAN example(s) is intended to include at least the following issues: > 5.2.1 W-WAN Network Characteristics W-WAN links typically exhibit some combination of the following link characteristics: - low bandwidth - high latency - high BER, or long variable delays due to local link-layer error recovery - some W-WAN links have a lot internal buffers which tend to accumulate data, thus resulting in increased round-trip delay due to long variable queuing delays - unexpected link disconnections may occur frequently (or intermittent link outages) - (re)setting link-connection up takes long time - typically last-hop link to the end user - W-WAN network typically takes care of terminal mobility: the connection point to the Internet is retained while the user moves with the mobile host 5.2.2 W-WAN PEP Implementations - Mowgli approach [MOWGLI]: Split TCP together with application layer proxies and W-WAN link specific protocol [KRLKA97] or with corresponding protocol modifications, compression in various forms, reduction of round trips, priority-based multiplexing of data over the W-WAN link, link-level flow control to prevent data from accumulating into the W-WAN link internal buffers, ... Expires June 3, 2000 [Page 21] INTERNET DRAFT Performance Enhancing Proxies December 1999 - transfer volume must be reduced to make Internet access usable, (long) link disconnections must not abort active (bulk data) transfers, slow W-WAN link should be efficiently shielded from excess traffic and the global (wired) Internet congestion, (all) applications can not be made mobility/W-WAN aware in short time frame or maybe ever, interactive traffic must be transmitted in timely manner even if there are other simultaneous bandwidth intensive (background) transfers... 5.3 W-LAN Environments Wireless LANs (W-LAN) are typically organized in a cellular topology where a base station with a W-LAN transceiver controls a single cell. A cell is defined in terms of the coverage area of the base station. The base stations are directly connected to the wired network. The base station in each of the cells is responsible for forwarding packets to and from the hosts located in the cell. Often the hosts with W-LAN tranceivers are mobile. When such a mobile host moves from one cell to another cell, the responsibility for forwarding packets between the wired network and the mobile host must be transferred to the base station of the new cell. This is known as a handoff. Many W-LAN systems also support an operation mode enabling ad-hoc networking. In this mode base stations are not necessarily needed, but hosts with W-LAN tranceiver can communicate directly with the other hosts within the tranceiver's transmission range. 5.3.1 W-LAN Network Characteristics Current wireless LANs typically provide link bandwidth from 1 Mbps to 10 Mbps, most typically bandwidth being 1 or 2 Mbps. In the future, wide deployment of higher bandwidths up to 20 Mbps or even higher can be expected. The round-trip delay with wireless LANs is on the order of a few milliseconds or tens of milliseconds. Examples of W-LANs include ... <[TBD>. Wireless LANs are error-prone due to wireless link corruption. TCP performance over W-LANs or a network path involving a W-LAN link suffers as packet losses due to wireless bit errors tend to occur in bursts. In addition, consecutive packet losses may occur also during handoffs. As TCP wrongly interprets these packet losses to be network congestion, the TCP sender reduces its congestion window and is often forced to timeout in order to recover from the consecutive Expires June 3, 2000 [Page 22] INTERNET DRAFT Performance Enhancing Proxies December 1999 losses. The result is often unacceptably poor end-to-end performance. 5.3.2 W-LAN PEP Implementations: Snoop Berkeley's Snoop protocol [SNOOP] is a TCP-specific approach in which a TCP-aware module, a Snoop agent, is deployed at the W-LAN base station that acts as the last-hop router to the mobile host. Snoop aims at retaining the TCP end-to-end semantics. The Snoop agent monitors every packet that passes through the base station in either direction and maintains soft state for each TCP connection. The Snoop agent is an asymmetric PEP implementation as it operates differently on TCP data and ACK channels as well as on the uplink (from the mobile host) and downlink (to the mobile host) TCP segments. For a data transfer to a mobile host, the Snoop agent caches unacknowledged TCP data segments which it forwards to the TCP receiver and monitors the corresponding ACKs. It does two things: 1. Retransmits any lost data segments locally by using local timers and TCP duplicate ACKs to identify packet loss, instead of waiting for the TCP sender to do so end-to-end. 2. Suppresses the duplicate ACKs on their way from the mobile host back to the sender, thus avoiding fast retransmit and congestion avoidance at the latter. Suppressing the duplicate ACKs is required to avoid unnecessary fast retransmits by the TCP sender as the Snoop agent retransmits a packet locally. Consider a system that employs the Snoop agent and a TCP sender S that sends packets to receiver R via a base station BS. Assume that S sends packets A, B, C, D, E (in that order) which are forwarded by BS to the wireless receiver R. Assume the first transmission of packet B is lost due to errors on the wireless link. In this case, R receives packets A, C, D, E and B (in that order). Receipt of packets C, D and E trigger duplicate ACKs. When the S receives three duplicate ACKs, it triggers fast retransmit (which results in a retransmission, as well as reduction of the congestion window). The Snoop agent also retransmits B locally, when it receives three duplicate ACKs. The fast retransmit at S occurs despite the local retransmit on the wireless link, degrading throughput. Snoop deals with this problem by dropping TCP duplicate ACKs appropriately at BS. For a data transfer from a mobile host, the Snoop agent detects the packet losses on the wireless link by monitoring the data segments it forwards. It then employs either Negative Acknowledgements (NAK) Expires June 3, 2000 [Page 23] INTERNET DRAFT Performance Enhancing Proxies December 1999 locally or Explicit Loss Notifications (ELN) to inform the mobile sender that the packet loss was not related to congestion, thus allowing the sender to retransmit without triggering normal congestion control procedures. To implement this, changes at the mobile host are required. When a Snoop agent uses NAKs to inform the TCP sender of the packet losses on the wireless link, one possibility to implement them is using the Selective Acknowledgment (SACK) option of TCP [RFC2018]. This requires enabling SACK processing at the mobile host. The Snoop agent sends a TCP SACK, when it detects a hole in the transmission sequence from the mobile host or when it has not received any new packets from the mobile host for a certain time period. This approach relies on the advisory nature of the SACKs: the mobile sender is advised to retransmit the missing segments indicated by SACK, but it must not assume successful end-to-end delivery of the segments acknowledged with SACK as these segments might get lost in the later path to the receiver. Instead, the sender must wait for a cumulative ACK to arrive. When the ELN mechanism is used to inform the mobile sender of the packet losses, Snoop uses one of the 'unreserved' bits in the TCP header for ELN [SNOOPELN]. The Snoop agent keeps track of the holes that correspond to segments lost over the wireless link. When a (duplicate) ACK corresponding to a hole in the sequence space arrives from the TCP receiver, the Snoop agent sets the ELN bit on the ACK to indicate that the loss is unrelated to congestion and then forwards the ACK to the TCP sender. When the sender receives a certain number of (duplicate) ACKs with ELN (a configurable variable at the mobile host, e.g., two), it retransmit the missing segment without performing any congestion control measures. The ELN mechanism using one of the six bits reserved for future use in the TCP header is dangerous as it exercises checks that might not be correctly implemented in TCP stacks, and may expose bugs. A scheme such as Snoop is needed only if the possibility of a fast retransmit due to wireless errors is non-negligible. In particular, if the wireless link uses link-layer recovery for lost data, then this scheme is not beneficial. Also, if the TCP window tends to stay smaller than four segments, for example, due to congestion related losses on the wired network, the probability that the Snoop agent will have an opportunity to locally retransmit a lost packet is small. This is because at least three duplicate ACKs are needed to trigger the local retransmission, but due to small window the Snoop agent may not be able to forward three new packets after the lost packet and thus induce the required three duplicate ACKs. Conversely, when the TCP window is large enough, Snoop can provide significant Expires June 3, 2000 [Page 24] INTERNET DRAFT Performance Enhancing Proxies December 1999 performance improvement (compared with standard TCP). < TBD: some text how Snoop tries to alleviate the problem with small windows > Snoop requires the intermediate node (base station) to examine and operate on the traffic between the mobile host and the other end host on the wired Internet. Hence, Snoop does not work if the IP traffic is encrypted. Possible solutions involve: - making the Snoop agent a party to the security association between the client and the server; - IPsec tunneling mode, terminated at the Snooping base station. However, these techniques require that users trust base stations. Snoop also requires that both the data and the corresponding ACKs traverse the same base station. Furthermore, the Snoop agent may duplicate efforts by the link layer as it retransmits the TCP data segments "at the transport layer" across the wireless link. (Snop has been described by its designers as a TCP-aware link layer. This is the right approach: the link and network layers can be much more aware of each other than strict layering suggests.) - to alleviate local link pkt drops due to high-BER (wireless) link 6 Security Considerations The security implications of using PEP are discussed in Section 4.1.1. 7 Appendix - PEP Terminology Summary This appendix provides a summary of terminology frequently used during discussion of Performance Enhancing Proxies. (In some cases, these terms have different meanings from their non-PEP related usage.) Expires June 3, 2000 [Page 25] INTERNET DRAFT Performance Enhancing Proxies December 1999 7.1 Definitions ACK spacing Delayed forwarding of acknowledgements in order to space them appropriately. application layer PEP Performance enhancement operating above the transport layer. May be aimed at improving application or transport protocol performance (or both). asymmetric link A link which has different rates for the forward channel (used for data segments) and the back (or return) channel (used for ACKs). available bandwidth The total capacity of a link available to carry information at any given time. May be lower than the raw bandwidth due to competing traffic. bandwidth utilization The actual amount of information delivered over a link in a given period, expressed as a percent of the raw bandwidth of the link. gateway Has several meanings depending on context: - An access point to a particular link; - A device capable of initiating and terminating connections on behalf of a user or end system (e.g. a firewall or proxy). Not necessarily, but could be, a router. in flight (data) Data sent but not yet acknowledged. More precisely, data sent for which the sender has not yet received the acknowledgement. local acknowledgement The generation of acknowledgements by an entity in the path Expires June 3, 2000 [Page 26] INTERNET DRAFT Performance Enhancing Proxies December 1999 between two end systems in order to allow the sending system to transmit more data without waiting for end-to-end acknowledgements. performance enhancing proxy An entity in the network acting on behalf of an end system or user (with or without the knowledge of the end system or user) in order to enhance protocol performance. raw bandwidth The total capacity of an unloaded link available to carry information. Snoop A TCP-aware link layer developed for wireless packet radio and cellular networks. It works by caching segments at a wireless base station. If the base station sees duplicate acknowledgements for a segment that it has cached, it retransmits the missing segment while suppressing the duplicate acknowledgement stream being forwarded back to the sender until the wireless receiver starts to acknowledge new data. Described in detail in [SNOOP]. split connection A connection that has been terminated before reaching the intended destination end system in order to initiate another connection towards the end system. TCP PEP Performance enhancement operating at the transport layer with TCP. Aimed at improving TCP performance. TCP splitting Using one or more split connections to improve TCP performance. TCP spoofing ( Sometimes used as a synonym for TCP PEP but more accurately refers to using transparent mechanisms to improve TCP performance. ) Expires June 3, 2000 [Page 27] INTERNET DRAFT Performance Enhancing Proxies December 1999 transparent ( Requires no changes to be made to either end system involved in a connection.) tunneling The process of wrapping a packet for transmission over a particular link. 8 Acknowledgements This document grew out of the Internet-Drafts "TCP Performance Enhancing Proxy Terminology" and "Long Thin Networks" and the work done in the IETF TCPSAT working group. 9 References [AGG+99] M. Allman, D. Glover, J. Griner, T. Henderson, J. Heidemann, H. Kruse, S. Ostermann, K. Scott, J. Semke, J. Touch, D. Tran, "Ongoing TCP Research Related to Satellites," Internet Draft (draft-ietf-tcpsat-res-issues-12.txt), Work in progress, October 1999. [BaPa99] H. Balakrishnan, V.N. Padmanabhan, "TCP Performance Implications of Network Asymmetry," Internet Draft (draft-ietf-pilc-asym-00.txt), Work in progress, September 1999. [CTC+97] H. Chang, C. Tait, N. Cohen, M. Shapiro, S. Mastrianni, R. Floyd, B. Housel, D. Lindquist, "Web Browsing in a Wireless Environment: Disconnected and Asynchronous Operation in ARTour Web Express," in proceedings of MobiCom'97, Budapest, Hungary, September 1997. [FMSBMR98] D.C. Feldmeier, A.J. McAuley, J.M. Smith, D.S. Bakin, W.S. Marcus, T.M. Raleigh, "Protocol Boosters," in IEEE Journal on Selected Areas of Communication, volume 16, number 3, April 1998. [FLASH] Flash Networks Ltd., performance boosting products technology vendor based in Kerselia, Israel. Website at Expires June 3, 2000 [Page 28] INTERNET DRAFT Performance Enhancing Proxies December 1999 http://www.flash-networks.com/ [FOURELLE] Fourelle Systems, performance boosting products technology vendor based in Santa Clara, California. Website at http://www.fourelle.com/ [HNS] Hughes Network Systems, Inc., VSAT technology vendor based in Germantown, Maryland. Website at http://www.hns.com/ [I-TCP] A. Bakre, B.R. Badrinath, "I-TCP: Indirect TCP for Mobile Hosts," in proceedings of the 15th International Conference on Distributed Computing Systems (ICDCS), May 1995. [Karn99] P. Karn, A. Falk, J. Touch, M-J. Montpetit, J. Mahdavi, G., Montenegro, "Advice for Internet Subnetwork Designers," Internet Draft (draft-ietf-pilc-link-design-01.txt), Work in progress, October 1999. [KRA94] M. Kojo, K. Raatikainen, T. Alanko, "Connecting Mobile Workstations to the Internet over a Digital Cellular Telephone Network," in Proc. Workshop on Mobile and Wireless Information Systems (MOBIDATA), Rutgers University, NJ, November 1994. Revised version published in Mobile Computing, pp. 253-270, Kluwer, 1996. [KRLKA97] M. Kojo, K. Raatikainen, M. Liljeberg, J. Kiiskinen, T. Alanko, "An Efficient Transport Service for Slow Wireless Telephone Links," in IEEE Journal on Selected Areas of Communication, volume 15, number 7, September 1997. [LHKR96] M. Liljeberg, H. Helin, M. Kojo, K. Raatikainen, "Mowgli WWW Software: Improved Usability of WWW in Mobile WAN Environments," in proceedings of IEEE Global Internet 1996 Conference, London, UK, November 1996. [M-TCP] K. Brown, S. Singh, "M-TCP: TCP for Mobile Cellular Networks," ACM Computer Communications Review Volume 27(5), 1997. Available at ftp://ftp.ece.orst.edu/pub/singh/papers/mtcp.ps.gz. [Part98] C. Partridge, "ACK Spacing for High Delay-Bandwidth Paths with Insufficient Buffering," September 1998. Internet-Draft draft-rfced-info-partridge-01.txt (work in progress). [Pax99] V. Paxson, "End-to-End Internet Packet Dynamics," IEEE/ACM Transactions on Networking, Vol 7, Number 3, 1999, pp 277-292. [RFC0793] J. Postel, "Transmission Control Protocol," STD 7, Expires June 3, 2000 [Page 29] INTERNET DRAFT Performance Enhancing Proxies December 1999 RFC 793, September 1981. [RFC1122] R. Braden, "Requirements for Internet Hosts -- Communications Layers," STD 3, RFC 1122, October 1989. [RFC1144] V. Jacobson, "Compressiing TCP/IP Headers for Low-Speed Serial Links," RFC 1144, February 1990. [RFC1323] V. Jacobson, R. Braden, D. Borman, "TCP Extensions for High Performance," RFC 1323, May 1992. [RFC1958] B. Carpenter, "Architectural Principles of the Internet," RFC 1958, June 1996. [RFC2018] M. Mathis, J. Mahdavi, S. Floyd, and A. Romanow, "TCP Selective Acknowledgment Options," RFC 2018, October, 1996. [RFC2246] T. Dierk, E. Allen, "TLS Protocol Version 1", RFC 2246, January 1999. [RFC2393] A. Shacham, R. Monsour, R. Pereira, M. Thomas, "IP Payload Compression Protocol (IPcomp)," RFC 2393, December 1998. [RFC2488] M. Allman, D. Glover, L. Sanchez, "Enhancing TCP Over Satellite Channels using Standard Mechanisms," BCP 28, RFC 2488, January 1999. [RFC2507] M. Degermark, B. Nordgren, S. Pink, "IP Header Compression," RFC 2507, February 1999. [RFC2508] S. Casner, V. Jacobson, "Compressing IP/UDP/RTP Headers for Low-Speed Serial Links," RFC 2508, February 1999. [RFC2509] M. Engan, S. Casner, C. Bormann, "IP Header Compression over PPP," RFC 2509, February 1999. [RFC2663] P. Srisuresh, M. Holdrege, "IP Network Address Translator (NAT) Terminology and Considerations," RFC 2663, August 1999. [SNOOP] H. Balakrishnan, S. Seshan, E. Amir, R. Katz, "Improving TCP/IP Performance over Wireless Networks," in proceedings of the 1st ACM Conference on Mobile Communications and Networking (Mobicom), Berkeley, CA, November 1995. [SNOOPELN] H. Balakrishnan, R. Katz, "Explicit Loss Notification and Wireless Web Performance," In Proc. IEEE Globecom 1998, Internet Mini-Conference, Sydney, Australia, November 1998. Expires June 3, 2000 [Page 30] INTERNET DRAFT Performance Enhancing Proxies December 1999 [SPACENET] Spacenet, VSAT technology vendor based in Mclean, Virginia. Website at http://www.spacenet.com/ [SRC84] J.H. Saltzer, D.P. Reed, D.D. Clark, "End-To-End Arguments in System Design," ACM TOCS, Vol 2, Number 4, November 1984, pp 277-288. [Zhang99] Y. Zhang, "Multi-Layer Protection Scheme for IPSEC," Internet Draft (draft-zhang-ipsec-mlipsec-00.txt), Work in progress, October 1999. 10 Authors' Addresses Questions about this document may be directed to: Expires June 3, 2000 [Page 31] INTERNET DRAFT Performance Enhancing Proxies December 1999 John Border Hughes Network Systems 11717 Exploration Lane Germantown, Maryland 20876 Voice: +1-301-601-4099 Fax: +1-301-601-4275 E-Mail: border@hns.com Markku Kojo University of Helsinki/Department of Computer Science P.O. Box 26 (Teollisuuskatu 23) FIN-00014 HELSINKI Finland Voice: +358-9-7084-4179 Fax: +358-9-7084-4441 E-Mail: kojo@cs.helsinki.fi Jim Griner NASA Glenn Research Center MS: 54-2 21000 Brookpark Orad Cleveland, Ohio 44135-3191 Voice: +1-216-433-5787 Fax: +1-216-433-8705 E-Mail: jgriner@grc.nasa.gov Gabriel E. Montenegro Sun Labs Networking and Security Group Sun Microsystems, Inc. 901 San Antonio Road Mailstop UMPK 15-214 Mountain View, California 94303 Voice: +1-650-786-6288 Fax: +1-650-786-6445 E-Mail: gab@sun.com 11 Full Copyright Statement Copyright (C) The Internet Society (1999). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it Expires June 3, 2000 [Page 32] INTERNET DRAFT Performance Enhancing Proxies December 1999 or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Expires June 3, 2000 [Page 33]