Internet Engineering Task Force Mark Allman, Editor INTERNET DRAFT Spencer Dawkins File: draft-ietf-tcpsat-res-issues-05.txt Dan Glover Jim Griner Tom Henderson John Heidemann Hans Kruse Shawn Ostermann Keith Scott Jeffrey Semke Joe Touch Diepchi Tran November, 1998 Expires: May, 1999 Ongoing TCP Research Related to Satellites Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet- Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). NOTE This document is not to be taken as a finished product. Some of the sections are rough and are included in order to obtain comments from the community that will benefit future iterations of this document. This is simply a step in the ongoing conversation about this document. Finally, all the authors of this draft do not necessarily agree with and/or advocate all the mechanisms outlined in this document. Abstract This document outlines TCP mechanisms that may help better utilize the available bandwidth in TCP transfers over long-delay satellite channels. The work outlined in this document is preliminary and has not yet been judged to be safe for use in the shared Internet. In addition, some of the work outlined in this document has been shown Expires: May, 1999 [Page 1] draft-ietf-tcpsat-res-issues-05.txt November 1998 to be unsafe for shared networks, but may be acceptable for use in private networks. Table of Contents 1 Introduction. . . . . . . . . . . . . . . . . 4 2 Satellite Architectures . . . . . . . . . . . 4 2.1 Asymmetric Satellite Networks . . . . . . . . 4 2.2 Satellite Link as Last Hop. . . . . . . . . . 4 2.3 Hybrid Satellite Networks . . . . . . . . 5 2.4 Point-to-Point Satellite Networks . . . . . . 5 2.5 Point-to-Multipoint Satellite Networks . . . 5 2.6 Multiple Satellite Hops . . . . . . . . . . . 5 3 Mitigations . . . . . . . . . . . . . . . . . 5 3.1 TCP For Transactions. . . . . . . . . . . . . 6 3.1.1 Mitigation Description. . . . . . . . . . . . 6 3.1.2 Research. . . . . . . . . . . . . . . . . . . 6 3.1.3 Implementation Issues . . . . . . . . . . . . 6 3.1.4 Topology Considerations . . . . . . . . . . . 6 3.2 Slow Start. . . . . . . . . . . . . . . . . . 6 3.2.1 Larger Initial Window . . . . . . . . . . . . 7 3.2.1.1 Mitigation Description. . . . . . . . . . . . 7 3.2.1.2 Research. . . . . . . . . . . . . . . . . . . 7 3.2.1.3 Implementation Issues . . . . . . . . . . . . 7 3.2.1.4 Topology Considerations . . . . . . . . . . . 8 3.2.2 Byte Counting . . . . . . . . . . . . . . . . 8 3.2.2.1 Mitigation Description. . . . . . . . . . . . 8 3.2.2.2 Research. . . . . . . . . . . . . . . . . . . 8 3.2.2.3 Implementation Issues . . . . . . . . . . . . 8 3.2.2.4 Topology Considerations . . . . . . . . . . . 8 3.2.3 Delayed ACKs After Slow Start . . . . . . . . 9 3.2.3.1 Mitigation Description. . . . . . . . . . . . 9 3.2.3.2 Research. . . . . . . . . . . . . . . . . . . 9 3.2.3.3 Implementation Issues . . . . . . . . . . . . 9 3.2.3.4 Topology Considerations . . . . . . . . . . . 9 3.2.4 Terminating Slow Start. . . . . . . . . . . . 9 3.2.4.1 Mitigation Description. . . . . . . . . . . . 9 3.2.4.2 Research. . . . . . . . . . . . . . . . . . . 10 3.2.4.3 Implementation Issues . . . . . . . . . . . . 10 3.2.4.4 Topology Considerations . . . . . . . . . . . 10 3.3 Loss Recovery . . . . . . . . . . . . . . . . 10 3.3.1 Non-SACK Based Mechanisms . . . . . . . . . . 10 3.3.1.1 Mitigation Description . . . . . . . . . . 10 3.3.1.2 Research. . . . . . . . . . . . . . . . . . . 11 3.3.1.3 Implementation Issues . . . . . . . . . . . . 11 3.3.1.4 Topology Considerations . . . . . . . . . . . 11 3.3.2 SACK Based Mechanisms . . . . . . . . . . . . 11 3.3.2.1 SACK "pipe" Algorithm . . . . . . . . . . . . 11 3.3.2.1.1 Mitigation Description. . . . . . . . . . . . 11 3.3.2.1.2 Research. . . . . . . . . . . . . . . . . . . 12 3.3.2.1.3 Implementation Issues . . . . . . . . . . . . 12 3.3.2.1.4 Topology Considerations . . . . . . . . . . . 12 3.3.2.2 Forward Acknowledgments . . . . . . . . . . . 12 3.3.2.2.1 Mitigation Description. . . . . . . . . . . . 12 Expires: May, 1999 [Page 2] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.3.2.2.2 Research. . . . . . . . . . . . . . . . . . . 13 3.3.2.2.3 Implementation Issues . . . . . . . . . . . . 13 3.3.2.2.4 Topology Considerations . . . . . . . . . . . 13 3.3.3 Explicit Congestion Notification. . . . . . . 13 3.3.3.1 Mitigation Description. . . . . . . . . . . . 13 3.3.3.2 Research. . . . . . . . . . . . . . . . . . . 14 3.3.3.3 Implementation Issues . . . . . . . . . . . . 14 3.3.3.4 Topology Considerations . . . . . . . . . . . 14 3.3.4 Detecting Corruption Loss . . . . . . . . . . 15 3.3.4.1 Mitigation Description. . . . . . . . . . . . 15 3.3.4.2 Research. . . . . . . . . . . . . . . . . . . 16 3.3.4.3 Implementation Issues . . . . . . . . . . . . 16 3.3.4.4 Topology Considerations . . . . . . . . . . . 17 3.4 Congestion Avoidance. . . . . . . . . . . . . 17 3.4.1 Mitigation Description. . . . . . . . . . . . 17 3.4.2 Research. . . . . . . . . . . . . . . . . . . 17 3.4.3 Implementation Issues . . . . . . . . . . . . 17 3.4.4 Topology Considerations . . . . . . . . . . . 18 3.5 Spoofing. . . . . . . . . . . . . . . . . . . 18 3.5.1 Mitigation Description. . . . . . . . . . . . 18 3.5.2 Research. . . . . . . . . . . . . . . . . . . 18 3.5.3 Implementation Issues . . . . . . . . . . . . 18 3.5.4 Topology Considerations . . . . . . . . . . . 18 3.6 snoop . . . . . . . . . . . . . . . . . . . . 18 3.7 Multiple Data Connections . . . . . . . . . . 19 3.7.1 Mitigation Description. . . . . . . . . . . . 19 3.7.2 Research. . . . . . . . . . . . . . . . . . . 20 3.7.3 Implementation Issues . . . . . . . . . . . . 20 3.7.4 Topological Considerations. . . . . . . . . . 20 3.8 Pacing TCP Segments . . . . . . . . . . . . . 20 3.8.1 ACK Spacing . . . . . . . . . . . . . . . . . 20 3.8.1.1 Mitigation Description. . . . . . . . . . . . 20 3.8.1.2 Research. . . . . . . . . . . . . . . . . . . 21 3.8.1.3 Implementation Issues . . . . . . . . . . . . 21 3.8.1.4 Topology Considerations . . . . . . . . . . . 21 3.8.2 Rate-Based Pacing . . . . . . . . . . . . . . 21 3.8.2.1 Mitigation Description. . . . . . . . . . . . 21 3.8.2.2 Research. . . . . . . . . . . . . . . . . . . 21 3.8.2.3 Implementation Issues . . . . . . . . . . . . 22 3.8.2.4 Topology Considerations . . . . . . . . . . . 22 3.9 TCP Header Compression. . . . . . . . . . . . 22 3.9.1 Mitigation Description. . . . . . . . . . . . 22 3.9.2 Research. . . . . . . . . . . . . . . . . . . 24 3.9.3 Implementation Issues . . . . . . . . . . . . 25 3.9.4 Topology Considerations . . . . . . . . . . . 25 3.10 Sharing TCP State Among Similar Connections . 25 3.10.1 Mitigation Description. . . . . . . . . . . . 25 3.10.2 Research. . . . . . . . . . . . . . . . . . . 26 3.10.3 Implementation Issues . . . . . . . . . . . . 26 3.10.4 Topology Considerations . . . . . . . . . . . 27 3.11 ACK Congestion Control. . . . . . . . . . . . 27 3.11.1 Mitigation Description. . . . . . . . . . . . 28 3.11.2 Research. . . . . . . . . . . . . . . . . . . 28 3.11.3 Implementation Issues . . . . . . . . . . . . 29 Expires: May, 1999 [Page 3] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.11.4 Topology Considerations . . . . . . . . . . . 29 3.12 ACK Filtering . . . . . . . . . . . . . . . . 29 3.12.1 Mitigation Description. . . . . . . . . . . . 29 3.12.2 Research. . . . . . . . . . . . . . . . . . . 30 3.12.3 Implementation. . . . . . . . . . . . . . . . 30 3.12.4 Topology Considerations . . . . . . . . . . . 30 4 Mitigation Interactions . . . . . . . . . . . 31 5 Conclusions . . . . . . . . . . . . . . . . . 31 6 References. . . . . . . . . . . . . . . . . . 31 7 Author's Addresses: . . . . . . . . . . . . . 35 1 Introduction This document outlines mechanisms that may help the Transmission Control Protocol (TCP) [Pos81] better utilize the bandwidth provided by long-delay satellite environments. These mechanisms may also help in other environments. The proposals outlined in this document are currently being studied throughout the research community. Therefore, these mechanisms SHOULD NOT be used in the shared Internet. If, at some point, the mechanisms discussed in this memo prove safe and appropriate for general use, the appropriate IETF documents will be written. Until that time, these mechanisms should be used for research and in private networks only. It should be noted that non-TCP mechanisms that help performance over satellite channels do exist (e.g., application-level changes, queueing disciplines, etc.). However, outlining these non-TCP mitigations is left as future work. 2 Satellite Architectures Satellite characteristics are discussed in [AG98]. This section discusses several ways that satellites might be used in the Internet. 2.1 Asymmetric Satellite Networks Some satellite networks exhibit a bandwidth asymmetry, with a larger data rate in one direction than the other, because of limits on the transmission power and the antenna size at one end of the link. Meanwhile, other satellite systems are one way only and use a non-satellite return path (such as a dialup modem link). The nature of most TCP traffic is asymmetric with data flowing in one direction and acknowledgments in opposite direction. However, the term asymmetric in this document refers to different physical capacities in the forward and return channels. 2.2 Satellite Link as Last Hop Satellite links that provide service directly to end users may allow for specialized design of protocols used over the last hop. Some satellite providers use the satellite channel as a shared high speed downlink to users with a lower speed, non-shared terrestrial channel that is used as a return channel for requests and acknowledgments. Expires: May, 1999 [Page 4] draft-ietf-tcpsat-res-issues-05.txt November 1998 Many times this creates an asymmetric network, as discussed in section 2.1. 2.3 Hybrid Satellite Networks In the more general case, satellites may be located at any point in the network topology. In this case, the satellite link acts as just another channel between two gateways. In this environment, a given connection may be sent over terrestrial channels (including wireless), as well as satellite channels. On the other hand, a connection could also travel over only the terrestrial network or only over the satellite portion of the network. 2.4 Point-to-Point Satellite Networks In point-to-point satellite networks, the only hop in the network is over the satellite channel. There is no terrestrial traffic to contend with in this environment. This pure satellite environment exhibits only the problems associated with the satellite channels, as outlined in [AG98]. Since this is a private network, some mitigations to TCP's inefficiencies can be used that are not suitable for shared networks, such as the Internet. 2.5 Point-to-Multipoint Satellite Networks Satellites have a natural advantage in point-to-multipoint communication. Although satellite communications began as a trunking method for telephony, the broadcast advantages of satellites were quickly recognized and utilized for television program distribution. One signal can be transmitted up to a satellite and then relayed back down to a large geographic area. Any ground station in that area can pick up the signal if tuned to the right channel. In the same way, data can be transmitted to small ground stations located over large geographic distances without loading terrestrial networks. Satellites have found use in corporate intranets and VSAT (very small aperture terminal) networks especially for database applications, but advantages for WWW caching, distributing network news, and multicasting applications are obvious and could help to reduce network congestion. While this is a valuable use of satellite systems, it is considered out of scope in this document, as TCP is a unicast-only protocol. 2.6 Multiple Satellite Hops In some cases, service may be provided over multiple satellite hops. This aggravates the satellite characteristics described in [AG98]. 3 Mitigations The following sections will discuss various techniques for mitigating the problems TCP faces in the satellite environment. Each of the following sections will be organized as follows: First, each mitigation will be briefly outlined. Next, research work involving the mechanism in question will be briefly discussed. The Expires: May, 1999 [Page 5] draft-ietf-tcpsat-res-issues-05.txt November 1998 implementation issues of the mechanism will be discussed next. Finally, the mechanism's benefits in each of the environments above will be outlined. 3.1 TCP For Transactions 3.1.1 Mitigation Description TCP uses a three-way handshake to setup a connection between two hosts [Pos81]. This connection setup requires 1-1.5 RTTs, depending upon whether the data sender started the connection actively or passively. This startup time can be eliminated by using TCP extensions for transactions (T/TCP) [Bra94]. In many situations, T/TCP is able to bypass the three-way handshake. This allows the data sender to begin transmitting data in the first segment sent (along with the SYN). This is especially helpful for short request/response traffic, as it saves a potentially long setup phase when no useful data is being transmitted. 3.1.2 Research T/TCP is outlined and analyzed in [Bra92] and [Bra94]. 3.1.3 Implementation Issues T/TCP requires changes in the TCP stacks of both the data sender and the data receiver. There are some security implications of sending data in the first data segment. These will be briefly presented and/or pointed at in a future iteration of this document. In addition, some researchers feel that the costs associated with implementing T/TCP outweigh the potential benefits. 3.1.4 Topology Considerations It is expected that T/TCP will be equally beneficial in all environments outlined in section 2. 3.2 Slow Start The slow start algorithm is used to gradually increase the size of TCP's sliding window [Jac88] [Ste97]. The algorithm is an important safe-guard against transmitting an inappropriate amount of data into the network when the connection starts up. However, slow start can also waste available capacity [All97a] [Hay97]. Slow start is particularly inefficient for transfers that are short compared to the delay*bandwidth product of the network (e.g., WWW transfers). Delayed ACKs are another source of wasted capacity during the slow start phase. RFC 1122 [Bra89] allows data receivers to refrain from ACKing every incoming data segment. However, every second full-sized segment must be ACKed. If a second full-sized segment does not arrive within a given timeout, an ACK must be generated (this timeout cannot exceed 500 ms). Since the data sender increases the size of cwnd based on the number of arriving ACKs, Expires: May, 1999 [Page 6] draft-ietf-tcpsat-res-issues-05.txt November 1998 reducing the number of ACKs slows the cwnd growth rate. In addition, when TCP starts sending, it sends 1 segment. When using delayed ACKs a second segment must arrive before an ACK is sent. Therefore, the receiver is always forced to wait for the delayed ACK timer to expire before ACKing the first segment, which also increases the transfer time. Several proposals have suggested ways to make slow start less time consuming. These proposals are briefly outlined below and references to the research work given. 3.2.1 Larger Initial Window 3.2.1.1 Mitigation Description One method that will reduce the amount of time required by slow start (and therefore, the amount of wasted capacity) is to make the initial value of cwnd be more than a single segment, as required by [Bra89] and [Ste97]. An experimental TCP extension outlined in [AFP98] allows the initial size of cwnd to be increased, according to equation 1. min (4*MSS, max (2*MSS, 4380 bytes)) (1) By increasing the initial value of cwnd, more packets are sent during the first RTT of data transmission, which will trigger more ACKs, allowing the congestion window to open more rapidly. In addition, by sending at least 2 segments initially, the first segment does not need to wait for the delayed ACK timer to expire as is the case when the initial size of cwnd is 1 segment (as discussed above). Therefore, the value of cwnd given in equation 1 saves up to 3 RTTs and a delayed ACK timeout when compared to an initial cwnd of 1 segment. 3.2.1.2 Research Several researchers have studied the use of a larger initial window in various environments. [Nic97] and [KAGT98] show a reduction in WWW page transfer time over hybrid fiber coax (HFC) and satellite channels respectively. Furthermore, it has been shown that using an initial cwnd of 4 packets does not negatively impact overall performance over dialup modem channels with a small number of buffers [SP97]. [AHO98] shows an improvement in transfer time for 16 KB files across the Internet and dialup modem channels when using a larger initial value for cwnd. However, a slight increase in retransmitted segments was also shown. Finally, [PN98] shows improved transfer time for WWW traffic in simulations with competing traffic, in addition to a small increase in the drop rate. 3.2.1.3 Implementation Issues The use of a larger initial cwnd value requires changes to the sender's TCP stack. Expires: May, 1999 [Page 7] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.2.1.4 Topology Considerations It is expected that the use of a large initial window would be equally beneficial to all network architectures outlined in section 2. 3.2.2 Byte Counting 3.2.2.1 Mitigation Description As discussed above, the wide-spread use of delayed ACKs increases the time needed by a TCP sender to increase the size of the congestion window during slow start. One mechanism that can mitigate this problems caused by delayed ACKs is the use of ``byte counting'' [All97a] [All98]. Using this mechanism, the cwnd increase is based on the number of previously unacknowledged bytes ACKed, rather than on the number of ACKs received. This makes the increase relative to the amount of data transmitted, rather than being dependent on the ACK interval used by the receiver. Byte counting leads to slightly larger line-rate bursts of segments. This increase in burstiness may increase the loss rate on some networks. The size of the line-rate burst increases if the receiver generates ``stretch ACKs'' [Pax97] (either by design [Joh95] or due to implementation bugs [All97b] [PADHV97]), since a stretch ACK covers more previously unacknowledged bytes than a normal delayed ACK. Therefore, the increase in cwnd when using byte counting may cause an inappropriate line-rate burst. One way to prevent these line-rate bursts is to use a ``limited byte counting'' mechanism, as outlined in [All98]. In this form of byte counting, cwnd is increased by the number of previously unacknowledged bytes ACKed by each incoming ACK, but the increase is limited to 2 segments. [All98] shows that this approach prevents large line-rate bursts that hurt performance. 3.2.2.2 Research Using byte counting, as opposed to standard ACK counting, has been shown to reduce the amount of time needed to increase the value of cwnd to an appropriate size in satellite networks [All97a]. In addition, [All98] presents a comparison of byte counting and the standard cwnd increase algorithm in uncongested networks and networks with competing traffic. This study found that the limited form of byte counting outlined above can improve performance, while also increasing the drop rate slightly. 3.2.2.3 Implementation Issues Changing from ACK counting to byte counting requires changes to the data sender's TCP stack. 3.2.2.4 Topology Considerations It has been suggested by some (and roundly criticized by others) Expires: May, 1999 [Page 8] draft-ietf-tcpsat-res-issues-05.txt November 1998 that byte counting will allow TCP to provide uniform cwnd increase, regardless of the ACKing behavior of the receiver. In addition, byte counting mitigates the retarded window growth provided by receivers that generate stretch ACKs because of the capacity of the return channel, as discussed in [BPK97]. 3.2.3 Delayed ACKs After Slow Start 3.2.3.1 Mitigation Description As discussed above, TCP senders use the number of incoming ACKs to increase the congestion window during slow start. And, since delayed ACKs reduce the number of ACKs returned by the receiver by roughly half, the rate of growth of the congestion window is reduced. One proposed solution to this problem is to use delayed ACKs only after the slow start phase. This provides more ACKs while TCP is agressivly increasing the congestion window and less ACKs while TCP is in steady state, which conserves network resources. 3.2.3.2 Research [All98] shows that in simulation, using delayed ACKs after slow start (DAASS) improves transfer time when compared to a receiver that always generates delayed ACKs. However, DAASS also slightly increases the loss rate due to the increased rate of cwnd growth. 3.2.3.3 Implementation Issues The major problem with DAASS is in the implementation. The receiver has to somehow detect when the sender is using the slow start algorithm. The receiver could implement a heuristic that attempts to watch the change in the amount of data being received and change the ACKing behavior accordingly. Or, the sender could send a message (a flipped bit, possibly) indicating that it was using slow start. The implementation of DAASS is, therefore, an open issue. 3.2.3.4 Topology Considerations DAASS should work equally well in all scenarios presented in section 2. However, in asymmetric networks it may aggrivate ACK congestion in the return channel, due to the increased number of ACKs (see sections 3.11 and 3.12 for a more detailed discussion of ACK congestion). 3.2.4 Terminating Slow Start 3.2.4.1 Mitigation Description The initial slow start phase is used by TCP to determine an appropriate congestion window size for the given network conditions [Jac88]. Slow start is terminated when TCP detects congestion, or when the size of cwnd reaches the size of the receiver's advertised window. Slow start is also terminated if cwnd grows beyond a Expires: May, 1999 [Page 9] draft-ietf-tcpsat-res-issues-05.txt November 1998 certain size. The threshold at which TCP ends slow start and begins using the congestion avoidance [Jac88] algorithm is called "ssthresh". The initial value for ssthresh is the receiver's advertised window. During slow start, TCP roughly doubles the size of cwnd every RTT and therefore can overwhelm the network with at most twice as many segments as the network can handle. By setting ssthresh to a value less than the receiver's advertised window initially, the sender may avoid overwhelming the network with twice the appropriate number of segments. Hoe [Hoe96] proposes using the packet-pair algorithm [Kes91] to determine a more appropriate value for ssthresh. The algorithm observes the spacing between the first few returning ACKs to determine the bandwidth of the bottleneck link. Together with the measured RTT, the delay*bandwidth product is determined and ssthresh is set to this value. When TCP's cwnd reaches this reduced ssthresh, slow start is terminated and transmission continues with congestion avoidance, which is a more conservative algorithm for increasing the size of the congestion window. 3.2.4.2 Research It has been shown that estimating ssthresh can improve performance and decrease packet loss in simulations [Hoe96]. However, obtaining an accurate estimate of the available bandwidth in a dynamic network remains an open research area. Therefore, before this mechanism is widely deployed, it must be studied in a more dynamic network environment. 3.2.4.3 Implementation Issues Estimating ssthresh requires changes to the data sender's TCP stack. 3.2.4.4 Topology Considerations It is expected that this mechanism will work well in all symmetric topologies outlined in section 2. However, asymmetric channels pose a special problem, as the rate of the returning ACKs may not be the bottleneck bandwidth in the forward direction. This can lead to the sender setting ssthresh too low. Premature termination of slow start can hurt performance, as congestion avoidance opens cwnd more conservatively. 3.3 Loss Recovery 3.3.1 Non-SACK Based Mechanisms 3.3.1.1 Mitigation Description Several similar algorithms have been developed and studied that improve TCP's ability to recover from multiple lost segments in a window of data without relying on the (often long) retransmission timeout. In addition, these algorithms are sender-side only algorithms that do not depend on the availability of selective Expires: May, 1999 [Page 10] draft-ietf-tcpsat-res-issues-05.txt November 1998 acknowledgements (SACKs). These algorithms generally work by updating the fast recovery phase to use information provided by ``partial ACKs'' to trigger retransmissions. A partial ACK covers some new data, but does not cover all data outstanding when a particular loss event occurs. For instance, consider the case when segment N is retransmitted using the fast retransmit algorithm and segment M is the last segment sent when segment N is resent. If segment N is the only segment lost, the ACK elicited by the retransmission of segment N would be for segment M. If, however, segment N+1 was also lost, the ACK elicited by the retransmssion of segment N will be N+1. This can be taken as an indication that segment N+1 was lost and used to trigger a retransmission. Several methods for using partial ACKs to trigger retransmissions have been deleveloped. See [FH98] for a more detailed description of the various algorithms. 3.3.1.2 Research Hoe [Hoe95,Hoe96] introduced the idea of using partial ACKs to trigger retransmissions and showed that doing so could improve performance. [FF96] shows that in some cases using partial ACKs to trigger retransmissions reduces the time required to recover from multiple lost segments. However, [FF96] also shows that in some cases (many lost segments) relying on the RTO timer can improve performance over simply using partial ACKs to trigger all retransmissions. [HK99] shows that using partial ACKs to trigger retransmissions improves performance when compared to TCP using fast retransmit/fast recovery in a satellite environment. 3.3.1.3 Implementation Issues Implementing these fast recovery enhancements requires changes to the sender-side TCP stack. 3.3.1.4 Topology Considerations It is expected that these changes will work well in all environments outlined in section 2. 3.3.2 SACK Based Mechanisms 3.3.2.1 SACK "pipe" Algorithm 3.3.2.1.1 Mitigation Description The so-called SACK-based pipe algorithm was developed as a conservative extension of the standard fast retransmit/fast recovery algorithms. The algorithm replaces the fast recovery algorithm and uses incoming SACK information to clock retransmissions into the network. The algorithm starts after fast retransmit triggers the resending of a segment. As with fast retransmit, the algorithm Expires: May, 1999 [Page 11] draft-ietf-tcpsat-res-issues-05.txt November 1998 cuts cwnd in half. The algorithm keeps a variable called "pipe", which is an estimate of the number of outstanding segments in the network. The pipe variable is decremented by 1 segment when a duplicate ACK arrives with new SACK information. The pipe variable is incremented by 1 when a new or retransmitted segment is sent. A segment (retransmit or new, if there are no segments to retransmit) may be sent when the value of pipe is less than cwnd. This algorithm generally allows TCP to recover from multiple segment losses in a window of data within the first RTT of loss detection. Like the FACK algorithm described below, the SACK information allows the pipe algorithm to decouple the choice of when to send a segment from the choice of what segment to send. 3.3.2.1.2 Research [FF96] shows that the pipe algorithm performs better than several non-SACK based recovery algorithms when 1--4 segments are lost from a window of data. [AHKO97] shows that the pipe algorithm improves performance over satellite channels. 3.3.2.1.3 Implementation Issues The pipe algorithm is implemented in the sender's TCP stack. However, it relies on SACK information generated by the receiver. 3.3.2.1.4 Topology Considerations It is expected that the pipe algorithm will work equally well in all scenarios presented in section 2. 3.3.2.2 Forward Acknowledgments 3.3.2.2.1 Mitigation Description The Forward Acknowledgment (FACK) algorithm [MM96a] [MM96b] was developed to improve TCP congestion control during loss recovery. FACK uses TCP SACK options to glean additional information about the congestion state, adding more precise control to the injection of data into the network during recovery. FACK decouples the congestion control algorithms from the data recovery algorithms to provide a simple and direct way to use SACK to improve congestion control. Due to the separation of these two algorithms, new data may be sent during recovery to sustain TCP's self-clock when there is no further data to retransmit. The most recent version of FACK is Rate-Halving, in which one packet is sent for every two ACKs received during recovery. ACKing every-other packet has the result of reducing the congestion window in one round trip to half of the number of packets that were successfully handled by the network (so when cwnd is too large by more than a factor of two it still gets reduced to half of what the network can sustain). Another important aspect of FACK with Rate-Halving is that it sustains the ACK self-clock during recovery Expires: May, 1999 [Page 12] draft-ietf-tcpsat-res-issues-05.txt November 1998 because transmitting a packet for every-other ACK does not require half a cwnd of data to drain from the network before transmitting, as required by the fast recovery algorithm [Ste97]. In addition, the FACK with Rate-Halving implementation provides Thresholded Retransmission to each lost segment. Tcprexmtthresh is the number of duplicate ACKs required by Reno to enter recovery. FACK applies thresholded retransmission to all segments by waiting until tcprexmtthresh SACK blocks indicate that a given segment is missing before resending the segment. This allows reasonable behavior on links that reorder segments. As described above, FACK sends a segment for every second ACK received during recovery. New segments are transmitted except when tcprexmtthresh SACK blocks have been observed for a dropped segment, at which point the dropped segment is retransmitted. 3.3.2.2.2 Research The original FACK algorithm was presented at Sigcomm'96 [MM96a]. The algorithm was later enhanced to include Rate-Halving [MM96b]. The real-world performance of FACK with Rate-Halving was shown to be much closer to the theoretical maximum for TCP than either SACK or Reno [MSMO97]. 3.3.2.2.3 Implementation Issues In order to use FACK, the sender's TCP stack must be modified. In addition, the receiver must be able to generate SACK options to obtain the full benefit of using FACK. 3.3.2.2.4 Topology Considerations FACK is expected to improve performance in all environments outlined in section 2. Since it is better able to sustain its self-clock than Reno, it may be considerably more attractive over long delay paths. 3.3.3 Explicit Congestion Notification 3.3.3.1 Mitigation Description Explicit congestion notification (ECN) allows routers to inform TCP senders about congestion levels. Two major forms of ECN have been studied. If a router employs backward ECN (BECN), it transmits packets to the data originator informing them of congestion. IP routers can accomplish this with an ICMP Source Quench message. The arrival of a BECN signal may or may not mean that a TCP data segment has been dropped, but it is a clear indication that the TCP sender should reduce its cwnd value. The second major form of congestion notification is forward ECN (FECN). In this form of ECN, routers mark data segments when congestion is imminent, but forward the data segment. The data receiver echos the congestion information back to the sender in the ACK packet. A current IETF proposal specifies an implementation of FECN [RF98]. Expires: May, 1999 [Page 13] draft-ietf-tcpsat-res-issues-05.txt November 1998 As proposed in [RF98], TCP senders transmit segments with an ``ECN-capable'' bit set in the packet header. If a router employing an active queueing strategy, such as Random Early Detection (RED) [FJ93] [BCC+98], would otherwise drop this segment, an ``ECN experienced'' bit is set instead. The TCP receiver echos this information back to the sender in ACK segments. The TCP sender reacts just as it would if a segment was dropped, and reduces its congestion window appropriately. A side-effect of implementing ECN, as suggested in [RF98], is that the intervening routers will employ active queueing mechanisms. This allows the routers to signal congestion by sending TCP a small number of ``congestion signals'' (segment drops or ECN messages), rather than discarding a large number of segments, as can happen when TCP overwhelms a router queue. Since satellite networks generally have higher bit-error rates than terrestrial networks, determining whether a segment was lost due to congestion or corruption may allow TCP to achieve better performance in high BER environments than currently possible (due to TCP's assumption that all loss is due to congestion). While not a complete solution to this problem, adding an ECN mechanism to TCP may help achieve this goal. See section 3.3.4 for a more detailed discussion of differentiating between corruption and congestion based losses. 3.3.3.2 Research [Flo94] shows that ECN is effective in reducing the segment loss rate in short and interactive TCP connections. Furthermore, [Flo94] also shows that ECN avoids some unnecessary, and costly TCP retransmission timeouts. Finally, [Flo94] also considers some of the advantages and disadvantages of FECN and BECN. A proposal for implementing ECN in the Internet is currently being discussed within the IETF [RF98]. 3.3.3.3 Implementation Issues Deployment of ECN requires changes to the TCP implementation on both sender and receiver. Deployment of ECN requires deployment of some active queue management infrastructure in routers. RED is assumed in most ECN discussions, because RED is already identifying segments to drop, even before its buffer space is exhausted. ECN simply allows the delivery of a ``marked'' segments while still notifying the end nodes that congestion is occurring along the path. 3.3.3.4 Topology Considerations It is expected that none of the environments outlined in section 2 will present a bias towards ECN traffic. Expires: May, 1999 [Page 14] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.3.4 Detecting Corruption Loss Differentiating between congestion (loss of segments due to router buffer overflow or imminent buffer overflow) and corruption (loss of segments due to damaged bits) is a difficult problem for TCP. This differentiation is particularly important because the action that TCP should take in the two cases is entirely different. In the case of corruption, TCP should merely retransmit the damaged segment as soon as its loss is detected; there is no need for TCP to adjust its congestion window. On the other hand, as has been widely discussed above, when the TCP sender detects congestion, it should immediately reduce its congestion window to avoid making the congestion worse. TCP's defined behavior, as motivated by [Jac88] [Jac90] and defined in [Bra89] [Ste97], is to assume that all loss is due to congestion and to trigger the congestion control algorithms, as defined in [Ste97]. In the worst case, this loss is detected by the expiration of TCP's retransmission timer. Alternately, the loss might be inferred from duplicate ACKs (by the fast retransmit algorithm [Jac90] [Ste97]) or by a hole in a SACK block. TCP's assumption that loss is due to congestion rather than corruption is a conservative mechanism that prevents congestion collapse [Jac88] [FF98]. Over satellite networks, however, as in many wireless environments, loss due to corruption is more common than on terrestrial networks. One common partial solution to this problem is to add Forward Error Correction (FEC) to the data that's sent over the satellite/wireless link. A more complete discussion of the benefits of FEC can be found in [AG98]. However, given that FEC does not always work or cannot be universally applied, other mechanisms have been studied to attempt to make TCP able to differentiate between congestion-based and corruption-based loss. TCP segments that have been corrupted are most often dropped by intervening routers when link-level checksum mechanisms detect that an incoming frame has errors. Occasionally, a TCP segment containing an error may survive without detection until it arrives at the TCP receiving host, at which point it will almost always either fail the IP header checksum or the TCP checksum and be discarded as in the link-level error case. Unfortunately, in either of these cases, it's not generally safe for the node detecting the corruption to return information about the corrupt packet to the TCP sender because the sending address itself might have been corrupted. 3.3.4.1 Mitigation Description Because the probability of link errors on a satellite link is relatively greater than on a terrestrial link, it is particularly important that the TCP sender retransmit these lost segments without reducing its congestion window. Because corrupt segments do not indicate congestion, there is no need for the TCP sender to enter a congestion avoidance phase, which may waste available bandwidth. Simulations performed in [SF98] show a performance improvement when Expires: May, 1999 [Page 15] draft-ietf-tcpsat-res-issues-05.txt November 1998 TCP can properly differentiate between between corruption and congestion of wireless links. Perhaps the greatest research challenge in detecting corruption is getting TCP (a transport-layer protocol) to receive appropriate information from either the network layer (IP) or the link layer. Much of the work done to date has involved link-layer mechanisms that retransmit damaged segments. The challenge seems to be to get these mechanisms to make repairs in such a way that TCP understands what happened and can respond appropriately. 3.3.4.2 Research Research into corruption detection to date has focused primarily on making the link level detect errors and then perform link-level retransmissions. This work is summarized in [BKVP97] [BPSK96]. One of the problems with this promising technique is that it causes an effective reordering of the segments from the TCP receiver's point of view. As a simple example, if segments A B C D are sent across a noisy channel and segment B is corrupted, segments C and D may have already crossed the channel before B can be retransmitted at the link level, causing them to arrive at the TCP receiver in the order A C D B. This segment reordering would cause the TCP receiver to generate duplicate ACKs upon the arrival of segments C and D, which may trigger the fast retransmit algorithm in TCP sender, in some cases. Research presented in [MV98] proposes the idea of suppressing or delaying the DUPACKs in the reverse direction to counteract this behavior. A more high-level approach, outlined in the SCPS-TP work [DMT96], uses a new "corruption experienced" ICMP error message generated by routers that detect corruption. These messages are sent in the forward direction, toward the packet's destination, rather than in the reverse direction as is done with ICMP Source Quench messages. Sending the error messages in the forward direction allows this feedback to work over asymmetric paths. As noted above, generating an error message in response to a damaged packet is problematic because the source and destination addresses may not be valid. SCPS gets around this problem by having the routers maintain a small cache of recent packet destinations; when the router experiences an error rate above some threshold, it sends an ICMP corruption-experienced message to all of the destinations in its cache. Each TCP receiver then must return this information to its respective TCP sender (through a TCP option). Upon receiving an ACK with this "corruption-experienced" option, the TCP sender assumes that packet loss is due to corruption rather than congestion for two round trip times or until it receives additional link state information (such as "link down", source quench, or additional "corruption experienced" messages). 3.3.4.3 Implementation Issues All of the techniques discussed above require changes to at least the TCP sending and receiving stacks, as well as intermediate Expires: May, 1999 [Page 16] draft-ietf-tcpsat-res-issues-05.txt November 1998 routers. 3.3.4.4 Topology Considerations It is expected that corruption detection, in general would be beneficial in all environments outlined in section 2. It would be particularly beneficial in the satellite/wireless environment over which these errors may be more prevalent. 3.4 Congestion Avoidance 3.4.1 Mitigation Description During congestion avoidance, in the absence of loss, the TCP sender adds approximately one segment to its congestion window during each RTT. Several researchers have observed that this policy leads to unfair sharing of bandwidth when multiple connections with different RTTs traverse the same bottleneck link, with the long RTT connections obtaining only a small fraction of their fair share of the bandwidth. One effective solution to this problem is to deploy fair queueing and TCP-friendly buffer management in network routers [Sut98]. However, in the absence of help from the network, other researchers have investigated changes to the congestion avoidance policy at the TCP sender, as described in [Flo91] and [Hen98]. 3.4.2 Research The ``Constant-Rate'' increase policy has been studied in [Flo91] and [Hen98]. It attempts to equalize the rate at which TCP senders increase their sending rate during congestion avoidance. Both [Flo91] and [Hen98] illustrate cases in which the ``Constant-Rate'' policy largely corrects the bias against long RTT connections, although [Hen98] presents some evidence that such a policy may be difficult to incrementally deploy in an operational network. The proper selection of a constant (for the constant rate of increase) is an open issue. The ``Increase-by-K'' policy can be selectively used by long RTT connections in a heterogeneous environment. This policy simply changes the slope of the linear increase, with connections over a given RTT threshold adding ``K'' segments to the congestion window every RTT, instead of one. [Hen98] presents evidence that this policy, when used with small values of ``K'', may be successful in reducing the unfairness while keeping the link utilization high, when a small number of connections share a bottleneck link. The selection of the constant ``K,'' the RTT threshold to invoke this policy, and performance under a large number of flows are all open issues. 3.4.3 Implementation Issues Implementation of either the ``Constant-Rate'' or ``Increase-by-K'' policies requires a change to the congestion avoidance mechanism at the TCP sender. In the case of ``Constant-Rate,'' such a change must be implemented globally. Additionally, the TCP sender must have a reasonably accurate estimate of the RTT of the connection. Expires: May, 1999 [Page 17] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.4.4 Topology Considerations These solutions are applicable to all satellite networks that are integrated with a terrestrial network, in which satellite connections may be competing with terrestrial connections for the same bottleneck link. 3.5 Spoofing [Editor's Note: This section may be removed in the future, depending upon the evolution of the "tcppep" initiative.] 3.5.1 Mitigation Description TCP spoofing is a technique used to split a TCP connection between a client (such as a mobile host or a hybrid terminal) and a server (such as fixed terminal or Internet server) into two parts: one between the client and its gateway router over a satellite/wireless link and the other between the gateway router and the server over the Internet/wired link. The gateway effectively breaks incoming TCP connections in two by acting on the client's behalf in interactions with the server. This allows the server to complete the transfer without incurring delays introduced by the satellite. Furthermore, spoofing allows the gateway to use a more appropriate transport protocol (or version of TCP) over the satellite hop. This mechanism is criticized by some as breaking the end-to-end semantics associated with the TCP protocol. 3.5.2 Research The TCP spoofing technique has been used to improve the overall throughput for asymmetric Internet access over satellite-terrestrial network [ASBD96] and for transferring data to mobile clients over wireless-wired network [BPSK97] [BB95]. In addition, [ASBD96] with spoofing and an increased ACK interval (i.e., decreased frequency of ACKs), it has been found that the throughput increased up to 400Kbps compare to 120Kbps of the system without these techniques. By using spoofing and the SMART retransmission technique [KM97], [BPSK97] shows that the TCP throughput improved from 0.7 Mbps to 1.3 Mbps in LAN environments and from 0.3 Mbps to 1.1 Mbps in WAN environments. 3.5.3 Implementation Issues The use of TCP spoofing requires modification to the gateway routers to enable them to act on the behalf of the end hosts. 3.5.4 Topology Considerations TCP spoofing should help performance over all topologies outlined above. However, TCP spoofing is an especially useful technique in asymmetric networks. 3.6 snoop Expires: May, 1999 [Page 18] draft-ietf-tcpsat-res-issues-05.txt November 1998 [Editor's Note: This section may be removed in the future, depending upon the evolution of the "tcppep" initiative.] 3.7 Multiple Data Connections 3.7.1 Mitigation Description One method that has been used to overcome TCP's inefficiencies in the satellite environment is to use multiple TCP flows to transfer a given file. The use of N TCP connections makes the sender N times more aggressive and therefore can benefit throughput in some situations. Using N multiple TCP connections can impact the transfer and the network in a number of ways, which are listed below. 1. The transfer is able to start transmission using an effective congestion window of N segments, rather than a single segment as one TCP flow uses. This allows the transfer to more quickly increase the effective cwnd size to an appropriate size for the given network. However, in some circumstances an initial window of N segments is inappropriate for the network conditions. In this case, a transfer utilizing more than one connection may aggravate congestion. 2. During the congestion avoidance phase, the transfer increases the effective cwnd by N segments per RTT, rather than one segment per RTT that a single TCP connection would. Again, this can aid the transfer by more rapidly increasing the effective cwnd to an appropriate point. However, this rate of increase can also be too aggressive for the network conditions. In this case, the use of multiple data connections can aggravate congestion in the network. 3. Using multiple connections can provide a very large overall cwnd size. This can be an advantage for TCP implementations that do not support the TCP window scaling extension [JBB92]. However, the aggregate cwnd size across all N connections is equivalent to using a TCP implementation that supports large windows. 4. The overall cwnd decrease in the face of dropped segments is reduced when using N parallel connections. A single TCP connection reduces the effective size of cwnd to half when segment loss is detected. Therefore, when utilizing N connections each using a window of W bytes, a single drop reduces the window to: (N * W) - (W / 2) Clearly this is a less dramatic reduction in the effective cwnd size than when using a single TCP connection. The use of multiple data connections can increase the ability of non-SACK TCP implementations to quickly recover from multiple dropped segments, assuming the dropped segments cross Expires: May, 1999 [Page 19] draft-ietf-tcpsat-res-issues-05.txt November 1998 connections. The use of multiple parallel connections makes TCP overly aggressive for many environments and can contribute to congestive collapse in shared networks [FF98]. The advantages provided by using multiple TCP connections are now largely provided by TCP extensions (larger windows, SACKs, etc.). Therefore, the use of a single TCP connection is more ``network friendly'' than using multiple parallel connections. However, using multiple parallel TCP connections may provide performance improvement in private networks. 3.7.2 Research Research on the use of multiple parallel TCP connections shows improved performance [IL92] [Hah94] [AOK95] [AKO96]. In addition, research has shown that multiple TCP connections can outperform a single modern TCP connection (with large windows and SACK) [AHKO97]. However, these studies did not consider the impact of using multiple TCP connections on competing traffic. [FF98] argues that using multiple simultaneous connections to transfer a given file may lead to congestive collapse in shared networks. 3.7.3 Implementation Issues To utilize multiple parallel TCP connections a client application and the corresponding server must be customized. 3.7.4 Topological Considerations As stated above, [FF98] outlines that the use of multiple parallel connections in a shared network, such as the Internet, may lead to congestive collapse. However, the use of multiple connections may be safe and beneficial in private networks. The specific topology being used will dictate the number of parallel connections required. Some work has been done to determine the appropriate number of connections on the fly [AKO96], but such a mechanism is far from complete. 3.8 Pacing TCP Segments 3.8.1 ACK Spacing 3.8.1.1 Mitigation Description Routes with high bandwidth*delay products (such as those found in geostationary satellite links) are capable of utilizing large TCP congestion windows. However, it can take a long time before TCP can fully utilize this large window. One possible cause of this delay are small router buffers, since in an idealized situation the router buffer should be one half the bandwidth*delay product in order to avoid losing segments [Par97]. This arises during slow start, because it is possible for the sender to burst data at twice the rate of the bottleneck router. Expires: May, 1999 [Page 20] draft-ietf-tcpsat-res-issues-05.txt November 1998 Using ACK spacing, the bursts can be spread over time by using a separation of at least two segments between ACKs [Par97]. Since the ACK rate is used to determine the rate packets are sent, ACK spacing would allow the sender to transmit at the correct rate. 3.8.1.2 Research Currently an implementation of ACK spacing does not exist, beyond a mere thought exercise. An algorithm has not been developed to determine the proper ACK spacing, which may be different depending on whether TCP is in slow start or congestion avoidance. 3.8.1.3 Implementation Issues ACK spacing can be implemented in the router, which eliminates the need to change either the sender or receiver's TCP stack. 3.8.1.4 Topology Considerations It may not be necessary to use ACK spacing in an asymmetrical routes, because of their inherent nature. 3.8.2 Rate-Based Pacing 3.8.2.1 Mitigation Description Slow-start takes several round trips to fully open the TCP congestion window over routes with high bandwidth-delay product. For short TCP connections (common in web traffic with HTTP/1.0), this slow-start overhead can preclude effective use of the high-bandwidth satellite channels. When senders implement slow-start restart after a TCP connection goes idle (suggested by Jacobson and Karels [JK92]), performance is reduced in long-lived (but bursty) connections [Hei97a]. Rate-based pacing (RBP) is a technique, used in the absence of incoming ACKs, where the data sender temporarily paces TCP segments at a given rate to restart the ACK clock. Upon receipt of the first ACK, pacing is discontinued and normal TCP ACK clocking resumes. The pacing rate may either be known from recent traffic estimates (when restarting an idle connection or from recent prior connections), or may be known through external means (perhaps in a point-to-point or point-to-multipoint satellite network where available bandwidth can be assumed to be large). In addition, pacing data during the first RTT of a transfer may allow TCP to make effective use of high bandwidth-delay links even for short transfers or intermittent senders. Pacing can also be used to reduce bursts in general (due to buggy TCPs or byte counting, see section 3.2.2 for a discussion on byte counting). 3.8.2.2 Research Simulation studies of rate-paced pacing for web-like traffic has Expires: May, 1999 [Page 21] draft-ietf-tcpsat-res-issues-05.txt November 1998 been shown to reduce router congestion and drop rates [VH97a]. In this environment, RBP substantially improves performance compared to slow-start-after-idle for intermittent senders, and it slightly improves performance over burst-full-cwnd-after-idle (because of drops) [VH98]. More recently, pacing has been suggested to eliminate burstiness in networks with ACK filtering [BPK97]. 3.8.2.3 Implementation Issues RBP requires only sender-side changes to TCP. Prototype implementations of RBP are available [VH97b]. RBP requires an additional sender timer for pacing. The overhead of timer-driven data transfer is often considered to high for practical use. Preliminary experiments suggest that in RBP this overhead is minimal because RBP only requires this timer for the first RTT of transmission [VH98]. 3.8.2.4 Topology Considerations RBP could be used to restart an idle TCP connection for all topologies in Section 2. Use at the beginning of new connections would be restricted to topologies where available bandwidth can be estimated out-of-band. 3.9 TCP Header Compression The TCP and IP header information needed to reliably deliver packets to a remote site across the Internet can add significant overhead, especially for interactive applications. Telnet packets, for example, typically carry only 1 byte of data per packet, and standard IPv4/TCP headers add at least 40 bytes to this; IPv6/TCP headers add at least 60 bytes. Much of this information remains relatively constant over the course of a session and so can be replaced by a short session identifier. 3.9.1 Mitigation Description Many fields in the TCP and IP headers either remain constant during the course of a session, change very infrequently, or can be inferred from other sources. For example, the source and destination addresses, as well as the IP version, protocol, and port fields generally do not change during a session. Packet length can be deduced from the length field of the underlying link layer protocol provided that the link layer packet is not padded. Packet sequence numbers in a forward data stream generally change with every packet, but increase in a predictable manner. The TCP/IP header compression methods described in [DNP97], [DENP97] and [Jac90] all reduce the overhead of TCP sessions by replacing the data in the TCP and IP headers that remains constant, changes slowly, or changes in a predictable manner with a short 'connection number'. Using these methods, the sender first sends a full TCP header, including in it a connection number that the sender will use Expires: May, 1999 [Page 22] draft-ietf-tcpsat-res-issues-05.txt November 1998 to reference the connection. The receiver stores the full header and uses it as a template, filling in some fields from the limited information contained in later, compressed headers. This compression can reduce the size of an IPv4/TCP header from 20 to as few as 3 or 4 bytes. Compression and decompression happen below the IP layer, and there is a separate compressor / decompressor pair for each serial link. Each compression pair maintains some state about some number of TCP connections which may use the link concurrently, and the decompresser passes complete, uncompressed packets to the IP layer. Thus header compression is transparent to routing, for example, since an incoming packet with compressed headers is expanded before being passed to the IP layer. A variety of methods can be used by the endpoints of a connection to negotiate the use of header compression. The PPP serial line protocol allows for an option exchange, during which time the endpoints can agree on whether or not to use header compression. For older SLIP implementations, [Jac90] describes a mechanism that uses the first bit in the IP packet as a flag. The reduction in overhead is especially useful when the link is bandwidth-limited such as terrestrial wireless and mobile satellite links, where the overhead associated with transmitting the header bits is nontrivial. Header compression has the added advantage that for the case of uniformly distributed bit errors, compressing TCP/IP headers can provide a better quality of service by decreasing the packet error probability. The shorter, compressed packets are less likely to be corrupted, and the reduction in errors increases the connection's throughput. Extra space is saved by encoding changes in fields that change relatively slowly by sending only their difference from their values in the previous packet instead of their absolute values. In order to decode headers compressed this way, the receiver keeps a copy of each full, reconstructed TCP header after it is decoded, and applies the delta values from the next decoded compressed header to the reconstructed full header template. A disadvantage to using this delta encoding scheme where values are encoded as deltas from their values in the previous packet is that if a single compressed packet it lost, subsequent packets with compressed headers can become garbled if they contain fields which depend on the lost packet. Consider a forward data stream of packets with compressed headers and increasing sequence numbers. If packet N is lost, the full header of packet N+1 will be reconstructed at the receiver using packet N-1's full header as a template. Thus the sequence number, which should have been calculated from packet N's header, will be wrong, the checksum will fail, and the packet will be discarded. When the sending TCP times out it retransmits a packet with a full header in order to re-synch the decompresser. Expires: May, 1999 [Page 23] draft-ietf-tcpsat-res-issues-05.txt November 1998 It is important to note that the compressor does not maintain any timers, nor does the decompresser know when an error occured (only the receiving TCP knows this, when the TCP checksum fails). A single bit error will cause the decompresser to lose synch, and subsequent packets with compressed headers will be dropped by the receiving TCP, since they will all fail the TCP checksum. When this happens, no duplicate acknowledgments will be generated, and the decompresser can only resynch when it receives a packet with an uncompressed header. This means that when header compression is being used, both fast retransmit and selective acknowledgments will not be able correct packets lost on a compressed link. The twice algorithm, described below, may be a partial solution to this. [DNP97] and [DENP97] describe TCP/IPv4 and TCP/IPv6 compression algorithms including compressing the various IPv6 extension headers as well as methods for compressing non-TCP streams. [DENP97] also augments TCP header compression by introducing the twice algorithm. If a particular packet fails to decompress properly, the twice algorithm modifies its assumptions about the inferred fields in the compressed header, assuming that a packet identical to the current one was dropped between the last correctly decoded packet and the current one. Twice then tries to decompress the received packet under the new assumptions and, if the checksum passes, the packet is passed to IP and the decompresser state has been re-synched. This procedure can be extended to three or more decoding attempts. Additional robustness can be achieved by caching full copies of packets which don't decompress properly in the hopes that later arrivals will fix the problem. Finally, the performance improvement if the decompresser can explicitly request a full header is discussed. Simulation results show that twice, in conjunction with the full header request mechanism, can improve throughput over uncompressed streams. 3.9.2 Research [Jac90] outlines a simple header compression scheme for TCP/IP. In [DENP97] the authors present the results of simulations showing that header compression is advantageous for both low and medium bandwidth links. Simulations show that the twice algorithm, combined with an explicit header request mechanism, improved throughput by 10-15% over uncompressed sessions across a wide range of bit error rates. Much of this improvement may have been due to the twice algorithm quickly re-synchronizing the decompresser when a packet is lost. This is because the twice algorithm, applied one or two times when the decompresser becomes unsynchronized, will re-synch the decompresser in between 83% and 99% of the cases. This is incredibly valuable, since packets received correctly after twice has resynched the decompresser will cause duplicate acknowledgments. This re-enables the use of both fast retransmit and SACK in conjunction with header compression. Expires: May, 1999 [Page 24] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.9.3 Implementation Issues Implementing TCP/IP header compression requires changes at both the sending (compressor) and receiving (decompresser) ends of each link that uses compression. The twice algorithm requires very little extra machinery over and above header compression, while the explicit header request mechanism of [DENP97] requires more extensive modifications to the sending and receiving ends of each link that employs header compression. 3.9.4 Topology Considerations TCP header compression is applicable to all of the environments discussed in section 2, but will provide relatively more improvement in situations where packet sizes are small (i.e., overhead is large) and there is medium to low bandwidth and/or higher BER. When TCP's congestion window size is large, implementing the explicit header request mechanism, the twice algorithm, and caching packets which fail to decompress properly become more critical. 3.10 Sharing TCP State Among Similar Connections 3.10.1 Mitigation Description Persistent TCP state information can be used to overcome limitations in the configuration of the initial state, and to automatically tune TCP to environments using satellite channels. TCP includes a variety of parameters, many of which are set to initial values which can severely affect the performance of satellite connections, even though most TCP parameters are adjusted later while the connection is established. These include initial size of cwnd and initial MSS size. Various suggestions have been made to change these initial conditions, to more effectively support satellite links. It is difficult to select any single set of parameters which is effective for all environments, however. Instead of attempting to select these parameters a-priori, TCB sharing keeps persistent state between incarnations of TCP connections, and considers this state when initializing a new connection. For example, if all connections to subnet 10 result in extended congestion windows of 1 megabyte, it is probably more efficient to start new connections with this value, than to rediscover it by requiring the cwnd to increase using slow start over a period of dozens of round-trip times. Sharing state among connections brings up a number of questions such as what to share, with whom to share, how to share it, and how to age shared information. First, what information is to be shared must be determined. Some information may be appropriate to share among TCP connections, while some information sharing may be inappropriate or not useful. Next, we need to determine with whom to share information. Sharing may be appropriate for TCP connections sharing a common path to a given host. Information may Expires: May, 1999 [Page 25] draft-ietf-tcpsat-res-issues-05.txt November 1998 be shared among connections within a host, or even among connections between different hosts, such as hosts on the same LAN. However, sharing information between connections not traversing the same network may not be appropriate. Given the state to share and the parties that share it, a mechanism for the sharing is required. Simple state, like MSS and RTT, is easy to share, but congestion window information can be shared a variety of ways. The sharing mechanism determines priorities among the sharing connections, and a variety of fairness criteria need to be considered. Also, the mechanisms by which information is aged require further study. Finally, the security concerns associated with sharing a piece of information need to be carefully considered before introducing such a mechanism. 3.10.2 Research The opportunity for such sharing, both among a sequence of connections, as well as among concurrent connections, is described in more detail in [Tou97]. The state management itself is largely an implementation issue; the point of TCB sharing is to raise this to a research issue, and to further specify the ways in which the information should be shared, regardless of the implementation. 3.10.3 Implementation Issues Much of TCB sharing is an implementation issue only. The TCP specifications do not preclude sharing information across connections, or using some information from previous connections to affect the state of new connections. The goal of TCB sharing is to decouple the effect of connection initialization from connection performance, to obviate the desire to have persistent connections solely to maintain efficiency. This allows separate connections to be more correctly used to indicate separate associations, distinct from the performance implications current implementations suffer. Each TCP connection maintains state, usually in a data structure called the TCP Control Block (TCB). The TCB contains information about the connection state, its associated local process, and feedback parameters about the connection's transmission. As originally specified, and usually implemented, the TCB is maintained on a per-connection basis. An alternate implementation can share some of this state across similar connection instances and among similar simultaneous connections. The resulting implementation can have better transient performance, especially where long-term TCB parameters differ widely from their typical initial values. These changes can be constrained to affect only the TCB initialization, and so have no effect on the long-term behavior of TCP after a connection has been established. They can also be more broadly applied to coordinate concurrent connections. We note that the notion of sharing TCB state was originally Expires: May, 1999 [Page 26] draft-ietf-tcpsat-res-issues-05.txt November 1998 documented in T/TCP [Bra92], and is used there to aggregate RTT values across connection instances, to provide meaningful average RTTs, even though most connections are expected to persist for only one RTT. T/TCP also shares a connection identifier, a sequence number separate from the window number and address/port pairs by which TCP connections are typically distinguished. As a result of this shared state, T/TCP allows a receiver to pass data in the SYN segment to the receiving application, prior to the completion of the three-way handshake, without compromising the integrity of the connection. In effect, this shared state caches a partial handshake from the previous connection, which is a variant of the more general issue of TCB sharing. Other implementation considerations are outlined in [Tou97] in detail. Many instances of the implementation are the subject of ongoing research. 3.10.4 Topology Considerations TCB sharing aggregates state information. The set over which this state is aggregated is critical to the performance of the sharing. Worst case, nothing is shared, which degenerates to the behavior of current implementations. Best case, information is shared among connections sharing a critical property. In earlier work [Tou97], the possibility of aggregating based on destination subnet, or even routing path is considered. For example, on a host connected to a satellite link, all connections out of the host share the critical property of large propagation latency, and are dominated by the bandwidth of the satellite link. In this case, all connections with the same source would share information. It is expected that sharing state across TCP connections may be useful in all network environments presented in section 2. 3.11 ACK Congestion Control In highly asymmetric networks, a low-speed return channel can restrict the performance of the data flow on a high-speed forward channel by limiting the flow of acknowledgements returned to the data sender. For example, if the data sender uses 1500 byte segments, and the receiver generates 40 byte acknowledgements (IPv4, TCP without options), the reverse channel will congest with ACKs for asymmetries of more than 75:1 if delayed acks are used, and 37:1 if every segment is acknowledged. For a 1.5 Mb/second data channel, ACK congestion will occur for reverse channel speeds below 20 kilobits/sec. These levels of asymmetry will readily occur if the reverse channel is shared among multiple satellite receivers, as is common in many VSAT satellite networks. If a terrestrial modem link is used as a reverse channel, ACK congestion is also likely, especially as the speed of the forward channel is increased. Current congestion control mechanisms are aimed at controlling the flow of data segments, but do not effect the flow of Expires: May, 1999 [Page 27] draft-ietf-tcpsat-res-issues-05.txt November 1998 acknowledgements. In [KVR98] the authors point out that the flow of acknowledgements can be restricted on the low-speed link not only by the bandwidth of the link, but also by the queue length of the router. The router may limit its queue length by counting packets, not bytes, and therefore begin discarding ACKs even if there is enough bandwidth to forward them. 3.11.1 Mitigation Description ACK Congestion Control extends the concepts of flow control for data segments to acknowledgement segments. In the method described in [BPK97], any intermediate gateway can mark an acknowledgement with an Explicit Congestion Notification (ECN) bit once the queue occupancy in the router exceeds a given threshold. The data sender (which receives the acknowledgement) must ``echo'' the ECN bit back to the data receiver (see section 3.3.3 for a more detailed discussion of ECN). The proposed algorithm for marking ACK segments with an ECN bit is Random Early Detection (RED) [FJ93]. In response to the receipt of ECN marked data segments, the receiver will dynamically reduce the rate of acknowledgements using a multiplicative backoff. Once segments without ECN are received, the data receiver speeds up acknowledgements using a linear increase, up to a rate of either 1 (no delayed acks) or 2 (delayed acks) data segments per ACK. The authors suggest that a ack must be generated at least once per window, and ideally a few times per window. As in the RED congestion control mechanism for data flow, the bottleneck gateway can randomly discard acknowledgements, rather than marking them with an ECN bit, once the queue fills beyond a given threshold. 3.11.2 Research [BPK97] analyze the effect of ACK Congestion Control (ACC) on the performance of an asymmetric network. They note that the use of ACC, and indeed the use of any scheme which reduces the frequency of acknowledgements, has potential unwanted side effects. Since each ACK will acknowledge more than the usual one or two data segments, the likelyhood of segment bursts from the data sender is increased. In addition, congestion window growth may be impeded if the receiver grows the window by counting received ACKs, as mandated by [APS98]. The authors therefore combine ACC with a series of modifications to the data sender, refered to as TCP Sender Adaptation (SA). SA combines a limit on the number of segments sent in a burst, regardless of window size. In addition, byte counting (as opposed to ACK counting) is employed for window growth. Note that byte counting has been studied elsewhere and can introduce side-effects, as well [All98]. The results presented in [BPK97] indicate that the use of ACC and SA will reduce the bursts produced by ACK losses in unmodified (Reno) TCP. In cases where these bursts would lead to data loss at Expires: May, 1999 [Page 28] draft-ietf-tcpsat-res-issues-05.txt November 1998 an intermediate router, the ACC and SA modification significantly improve the throughput for a single data transfer. The results further suggest that the use of ACC and SA significantly improve fairness between two simultaneous transfers. ACC is further reported to prevent the increase in round trip time (RTT) that occurs when an unmodified TCP fills the reverse router queue with acknowledgements. In networks where the forward direction is expected to suffer losses in one of the gateways, due to queue limitiations, the authors report at best a very slight improvement in performance for ACC and SA, compared to unmodified Reno TCP. 3.11.3 Implementation Issues Both ACC and SA require modification of the host as well as the bottleneck gateway. The current research suggests that implementing ACC without the SA modifications results in a data sender which generates potentially disruptive segment bursts. It should be noted that ACC does require host modifications if it is implemented in the way proposed in [BPK97]. The authors note that ACC can be implemented by discarding ACKs (which requires only a gateway modification, but no changes in the hosts), as opposed to marking them with ECN. Such an implementation may, however, produce bursty data senders if it is not combined with a burst mitigation technique. 3.11.4 Topology Considerations Neither ACC nor SA require the storage of state in the gateway. These schemes should therefore be applicable for all topologies, provided that the hosts using the satellite or hybrid network can be modified. The existing research does not directly address fairness between ACC/SA modified and unmodified hosts. 3.12 ACK Filtering ACK Filtering (AF) is designed to address the same ACK Compression effects described in 3.11. Contrary to ACC, however, AF is designed to operate without host modifications. 3.12.1 Mitigation Description AF takes advantage of the cumulative acknowldgement structure of TCP. The bottleneck router in the reverse direction (the low speed channel) must be modified to implement AF. Upon receipt of a segment which represents a TCP acknowledgement, the router scans the queue for redundant ACKs for the same connection, i.e. ACKs which acknowledge portions of the window which are included in the most recent ACK. All of these "earlier" ACKs are removed from the queue and discarded. The router does not store state information, but does need to implement the additional processing required to find and remove segments from the queue upon receipt of an ACK. Expires: May, 1999 [Page 29] draft-ietf-tcpsat-res-issues-05.txt November 1998 3.12.2 Research [BPK97] analyzes the effects of AF. As is the case in ACC, the use of ACK filtering alone would produce significant sender bursts, since the ACKs will be acknowledging more previously-unacknowledged data. The SA modifications described in 3.11.2 could be used to prevent those bursts, at the cost of requiring host modifications. To prevent the need for modifications in the TCP stack, AF is more likely to be paired with the ACK Reconstruction (AR) technique, which can be implemented at the router where segments exit the slow reverse link. AR inspects ACKs exiting the channel, and if it detects large ``gaps'' in the ACK sequence, it generates additional ACKs to reconstruct an acknowledgement flow which more closely resembles what the data sender would have seen had ACK Filtering not been introduced. AR requires two parameters; one parameter is the desired ACK frequency, while the second controls the spacing, in time, between the release of consecutive reconstructed ACKs. In [BPK97], the authors show the combination of AF and AR to increase throughput, in the networks studied, over both unmodified TCP and the ACC/SA modifications. Their results also strongly suggest that the use of AF alone, in networks where congestion losses are expected, decreases performance (even below the level of unmodified Reno) due to sender bursting. 3.12.3 Implementation Both ACK Filtering and ACK Reconstruction require only router modification. However, the implementation of AR requires some storage of state information in the exit router. While AF does not require storage of state information, its use without AR (or SA) would appear to produce undesired side effects. Further research appears to be required regarding appropriate ranges for the parameters needed in AR. 3.12.4 Topology Considerations AF and AR appear applicable to all topologies, assuming that the storage of state information in AR does not prove to be prohibitive for routers which handle large numbers of flows. The fact that host modifications are not required for AF/AR makes this approach attractive for hybrid networks and networks with diverse types of hosts. On the other hand, the implementation of AF/AR requires the routers to examine the TCP header, which prohibits their use in secure networks where IPSEC is deployed. In such networks, AF/AR can be effective only inside the security permimeter of a private, or virtual private network, or in private networks where the satellite link is protected only by link-layer encryption (as opposed to IPSEC). Expires: May, 1999 [Page 30] draft-ietf-tcpsat-res-issues-05.txt November 1998 4 Mitigation Interactions 5 Conclusions 6 References [AFP98] Mark Allman, Sally Floyd, Craig Partridge. Increasing TCP's Initial Window, September 1998. RFC 2414. [AHKO97] Mark Allman, Chris Hayes, Hans Kruse, Shawn Ostermann. TCP Performance Over Satellite Links. In Proceedings of the 5th International Conference on Telecommunication Systems, March 1997. [AHO98] Mark Allman, Chris Hayes, Shawn Ostermann. An Evaluation of TCP with Larger Initial Windows. Computer Communication Review, 28(3), July 1998. [AKO96] Mark Allman, Hans Kruse, Shawn Ostermann. An Application-Level Solution to TCP's Satellite Inefficiencies. In Proceedings of the First International Workshop on Satellite-based Information Services (WOSBIS), November 1996. [AG98] Mark Allman, Dan Glover. Enhancing TCP Over Satellite Channels using Standard Mechanisms, September 1998. Internet-Draft draft-ietf-tcpsat-stand-mech-06.txt (work in progress). [All97a] Mark Allman. Improving TCP Performance Over Satellite Channels. Master's thesis, Ohio University, June 1997. [All97b] Mark Allman. Fixing Two BSD TCP Bugs. Technical Report CR-204151, NASA Lewis Research Center, October 1997. [All98] Mark Allman. On the Generation and Use of TCP Acknowledgments. July, 1998. Submitted to ACM Computer Communication Review. [AOK95] Mark Allman, Shawn Ostermann, Hans Kruse. Data Transfer Efficiency Over Satellite Circuits Using a Multi-Socket Extension to the File Transfer Protocol (FTP). In Proceedings of the ACTS Results Conference, NASA Lewis Research Center, September 1995. [ASBD96] Vivek Arara, Narin Suphasindhu, John S. Baras, Douglas Dillon. Asymmetric Internet Access Over Satellite-Terrestrial Networks. Proceedings of the AIAA: 16th International Communications Satellite Systems Conference and Exhibit, Part1, pp. 476-482, Washington, D.C, February 25-29, 1996. [BB95] Ajay Bakre, B.R. Badrinath. I-TCP: Indirect TCP for Mobile Hosts. In Proceeding of the 15th International Conference on Distributed Computing Systems (ICDCS), May 1995. Expires: May, 1999 [Page 31] draft-ietf-tcpsat-res-issues-05.txt November 1998 [BCC+98] B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, D. Estrin, S. Floyd, V. Jacobson, G. Minshall, C. Partridge, L. Peterson, K. Ramakrishnan, S. Shenker, J. Wroclawski, L. Zhang, Recommendations on Queue Management and Congestion Avoidance in the Internet, April 1998. RFC 2309. [BKVP97] B. Bakshi and P. Krishna and N. Vaidya and D. Pradham, "Improving Performance of TCP over Wireless Networks", 17th International Conference on Distributed Computing Systems (ICDCS), May 1997 [BPK97] Hari Balakrishnan, Venkata N. Padmanabhan, and Randy H. Katz. The Effects of Asymmetry on TCP Performance. In Proceedings of the ACM/IEEE Mobicom, Budapest, Hungary, ACM. September, 1997. [BPSK96] H. Balakrishnan and V. Padmanabhan and S. Sechan and R. Katz, "A Comparison of Mechanisms for Improving TCP Performance over Wireless Links", ACM SIGCOMM, August 1996. [BPSK97] Hari Balakrishnan, Venkata N. Padmanabhan, Srinivasan Seshan, Randy H. Katz. A Comparison of Mechanism for Improving TCP Performance over Wireless Links. IEEE/ACM Transactions on Networking, December 1997. [Bra89] Robert Braden. Requirements for Internet Hosts -- Communication Layers, October 1989. RFC 1122. [Bra92] Robert Braden. Transaction TCP -- Concepts, September 1992. RFC 1379. [Bra94] Robert Braden. T/TCP -- TCP Extensions for Transactions: Functional Specification, July 1994. RFC 1644. [DENP97] Low-Loss TCP/IP Header Compression for Wirelesss Networks. Wireless Networks, vol.3, no.5, p. 375-87 [DMT96] R. C. Durst and G. J. Miller and E. J. Travis, "TCP Extensions for Space Communications", MOBICOMM 96, ACM, USA, 1996. [DNP97] Mikael Degermark, Bjorn Nordgren, and Stephen Pink. IP Header Compression, December 1997. Internet-Draft draft-degermark-ipv6-hc-05.txt (work in progress). [FF98] Sally Floyd, Kevin Fall. Promoting the Use of End-to-End Congestion Control in the Internet. Submitted to IEEE Transactions on Networking. [FH98] Sally Floyd and Tom Henderson. The NewReno Modification to TCP's Fast Recovery Algorithm, November 1998. Internet-Draft draft-ietf-tcpimpl-newreno-00.txt (work in progress). Expires: May, 1999 [Page 32] draft-ietf-tcpsat-res-issues-05.txt November 1998 [FJ93] Sally Floyd and Van Jacobson. Random Early Detection Gateways for Congestion Avoidance, IEEE/ACM Transactions on Networking, V. 1 N. 4, August 1993. [Flo91] Sally Floyd. Connections with Multiple Congested Gateways in Packet-Switched Networks, Part 1: One-way Traffic. ACM Computer Communications Review, V. 21, N. 5, October 1991. [Flo94] Sally Floyd. TCP and Explicit Congestion Notification, ACM Computer Communication Review, V. 24 N. 5, October 1994. [Hah94] Jonathan Hahn. MFTP: Recent Enhancements and Performance Measurements. Technical Report RND-94-006, NASA Ames Research Center, June 1994. [Hay97] Chris Hayes. Analyzing the Performance of New TCP Extensions Over Satellite Links. Master's Thesis, Ohio University, August 1997. [Hen98] Tom Henderson. On Improving the Fairness of TCP Congestion Avoidance. Proceedings of IEEE Globecom `98 Conference, 1998. [HK99] Tim Henderson, Randy Katz. Transport Protocols for Internet-Compatible Satellite Networks, IEEE Journal on Selected Areas of Communication, 1999. [Hoe95] J. Hoe, Startup Dynamics of TCP's Congestion Control and Avoidance Schemes. Master's Thesis, MIT, 1995. [Hoe96] Janey Hoe. Improving the Startup Behavior of a Congestion Control Scheme for TCP. In ACM SIGCOMM, August 1996. [IL92] David Iannucci and John Lakashman. MFTP: Virtual TCP Window Scaling Using Multiple Connections. Technical Report RND-92-002, NASA Ames Research Center, January 1992. [Jac88] Van Jacobson. Congestion Avoidance and Control. In Proceedings of the SIGCOMM '88, ACM. August, 1988. [Jac90] Van Jacobson. Compressing TCP/IP Headers, February 1990. RFC 1144. [JBB92] Van Jacobson, Robert Braden, and David Borman. TCP Extensions for High Performance, May 1992. RFC 1323. [JK92] Van Jacobson and Mike Karels. Congestion Avoidance and Control. Originally appearing in the proceedings of SIGCOMM '88 by Jacobson only, this revised version includes an additional appendix. The revised version is available at ftp://ftp.ee.lbl.gov/papers/congavoid.ps.Z. 1992. [Joh95] Stacy Johnson. Increasing TCP Throughput by Using an Extended Acknowledgment Interval. Master's Thesis, Ohio University, June 1995. Expires: May, 1999 [Page 33] draft-ietf-tcpsat-res-issues-05.txt November 1998 [Kes91] Srinivasan Keshav. A Control Theoretic Approach to Flow Control. In ACM SIGCOMM, September 1991. [KAGT98] Hans Kruse, Mark Allman, Jim Griner, Diepchi Tran. HTTP Page Transfer Rates Over Geo-Stationary Satellite Links. March 1998. Proceedings of the Sixth International Conference on Telecommunication Systems. [KM97] S. Keshav, S. Morgan. SMART Retransmission: Performance with Overload and Random Losses. Proceeding of Infocom. 1997. [MM96a] M. Mathis, J. Mahdavi, "Forward Acknowledgment: Refining TCP Congestion Control," Proceedings of SIGCOMM'96, August, 1996, Stanford, CA. Available from http://www.psc.edu/networking/papers/papers.html [MM96b] M. Mathis, J. Mahdavi, "TCP Rate-Halving with Bounding Parameters" Available from http://www.psc.edu/networking/papers/FACKnotes/current. [MSMO97] M. Mathis, J. Semke, J. Mahdavi, T. Ott, "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm",Computer Communication Review, volume 27, number3, July 1997. available from Available from http://www.psc.edu/networking/papers/papers.html [MV98] Miten N. Mehta and Nitin H. Vaidya. Delayed Duplicate-Acknowledgments: A Proposal to Improve Performance of TCP on Wireless Links. Technical Report 98-006, Department of Computer Science, Texas A&M University, February 1998. [Nic97] Kathleen Nichols. Improving Network Simulation with Feedback. Com21, Inc. Technical Report. Available from http://www.com21.com/pages/papers/068.pdf. [PADHV97] Vern Paxson, Mark Allman, Scott Dawson, Ian Heavens, Bernie Volz. Known TCP Implementation Problems, March 1998. Internet-Draft draft-ietf-tcpimpl-prob-03.txt. [Par97] Craig Partridge. ACK Spacing for High Delay-Bandwidth Paths with Insufficient Buffering, July 1997. Internet-Draft draft-partridge-e2e-ackspacing-00.txt. [Pax97] Vern Paxson. Automated Packet Trace Analysis of TCP Implementations. In Proceedings of ACM SIGCOMM, September 1997. [PN98] Poduri, K., and Nichols, K., Simulation Studies of Increased Initial TCP Window Size, February 1998. Internet-Draft draft-ietf-tcpimpl-poduri-00.txt (work in progress). [Pos81] Jon Postel. Transmission Control Protocol, September 1981. RFC 793. Expires: May, 1999 [Page 34] draft-ietf-tcpsat-res-issues-05.txt November 1998 [RF98] A Proposal to add Explicit Congestion Notification (ECN) to IPv6 and to TCP. K. K. Ramakrishnan and Sally Floyd. Internet-Draft draft-kksjf-ecn-01.txt, July 1998. (Work in progress). [SF98] Nihal K. G. Samaraweera and Godred Fairhurst, "Reinforcement of TCP error Recovery for Wireless Communication", Computer Communication Review, volume 28, number 2, April 1998. [SP97] Tim Shepard and Craig Partridge. When TCP Starts Up With Four Packets Into Only Three Buffers, July 1997. Internet-Draft draft-shepard-TCP-4-packets-3-buff-00.txt (work in progress). [Ste97] W. Richard Stevens. TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms, January 1997. RFC 2001. [Sut98] B. Suter, T. Lakshman, D. Stiliadis, and A. Choudhury. Design Considerations for Supporting TCP with Per-flow Queueing. Proceedings of IEEE Infocom `98 Conference, 1998. [Tou97] Touch, J., "TCP Control Block Interdependence," RFC-2140, USC/Informatino Sciences Institute , April 1997. [VH97a] Vikram Visweswaraiah and John Heidemann. Improving Restart of Idle TCP Connections. Technical Report 97-661, University of Southern California, 1997. [VH97b] Vikram Visweswaraiah and John Heidemann. Rate-based pacing Source Code Distribution, Web page http://www.isi.edu/lsam/publications/rate_based_pacing/README.html. November, 1997. [VH98] Vikram Visweswaraiah and John Heidemann. Improving Restart of Idle TCP Connections (revised). Submitted for publication. 7 Author's Addresses: Mark Allman NASA Lewis Research Center/Sterling Software 21000 Brookpark Rd. MS 54-2 Cleveland, OH 44135 mallman@lerc.nasa.gov http://gigahertz.lerc.nasa.gov/~mallman Spencer Dawkins Nortel P.O.Box 833805 Richardson, TX 75083-3805 Spencer.Dawkins.sdawkins@nt.com Dan Glover NASA Lewis Research Center 21000 Brookpark Rd. MS 54-2 Expires: May, 1999 [Page 35] draft-ietf-tcpsat-res-issues-05.txt November 1998 Cleveland, OH 44135 Daniel.R.Glover@lerc.nasa.gov Jim Griner NASA Lewis Research Center 21000 Brookpark Rd. MS 54-2 Cleveland, OH 44135 jgriner@lerc.nasa.gov John Heidemann University of Southern California/Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292-6695 johnh@isi.edu Tom Henderson University of California at Berkeley Phone: +1 (510) 642-8919 Email: tomh@cs.berkeley.edu URL: http://www.cs.berkeley.edu/~tomh/ Hans Kruse J. Warren McClure School of Communication Systems Management Ohio University 9 S. College Street Athens, OH 45701 Phone: 740-593-4891 Fax: 740-593-4889 hkruse1@ohiou.edu http://www.csm.ohiou.edu/kruse Shawn Ostermann School of Electrical Engineering and Computer Science Ohio University 416 Morton Hall Athens, OH 45701 Phone: (740) 593-1234 ostermann@cs.ohiou.edu Keith Scott Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive MS 161-260 Pasadena, CA 91109-8099 Keith.Scott@jpl.nasa.gov http://eis.jpl.nasa.gov/~kscott/ Jeffrey Semke Pittsburgh Supercomputing Center 4400 Fifth Ave. Pittsburgh, PA 15213 semke@psc.edu http://www.psc.edu/~semke Expires: May, 1999 [Page 36] draft-ietf-tcpsat-res-issues-05.txt November 1998 Joe Touch University of Southern California/Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292-6695 USA Phone: +1 310-822-1511 x151 Fax: +1 310-823-6714 URL: http://www.isi.edu/~touch Email: touch@isi.edu Diepchi Tran NASA Lewis Research Center 21000 Brookpark Rd. MS 54-2 Cleveland, OH 44135 dtran@lerc.nasa.gov Expires: May, 1999 [Page 37]