IPFIX working group Internet Draft EDITOR: B. Claise draft-ietf-ipfix-protocol-04.txt Cisco Systems Expires: January 2005 July 2004 IPFIX Protocol Specifications Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsolete by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document specifies the IPFIX protocol that provides network operators with access to IP flow information. In order to export IP flow information to the IPFIX collecting process, a common method of representing the flow data and a standard means of communicating them from an exporter to a collector required. This document describes how the IPFIX flow record data, options record data and control information (templates for example) are carried over a congestion-aware transport protocol from IPFIX exporting process to IPFIX collecting process. Claise, et. al Standard Track [Page 1] IPFIX Protocol Specifications July 2004 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Table of Contents 1. Points of Discussion.........................................3 1.1 Open Issues................................................3 1.2 Action Items...............................................6 2. Introduction.................................................6 2.1 IPFIX Documents Overview...................................6 3. Terminology..................................................7 3.1 Terminology Summary Table.................................12 4. Criteria for Flow Expiration and Export.....................13 4.1 Flow Expiration...........................................13 4.2 Flow Export...............................................13 5. Transport Protocol..........................................14 5.1 Transport Compliance and Transport Usage..................14 5.2 TCP.......................................................15 5.3 SCTP......................................................15 5.3.1 Congestion Avoidance...................................15 5.3.2 Reliability............................................15 5.3.3 MTU....................................................15 5.3.4 Exporting Process......................................15 5.3.4.1 Association...........................................15 5.3.4.2 Source ID.............................................16 5.3.4.3 Stream................................................16 5.3.4.4 Template..............................................17 5.3.5 Collecting Process.....................................17 5.3.6 Failover...............................................17 5.4 UDP.......................................................18 5.4.1 Congestion Avoidance...................................18 5.4.2 Reliability............................................18 5.4.3 MTU....................................................18 5.4.4 Port Numbers...........................................19 5.4.5 Exporting Process......................................19 5.4.5.1 Template..............................................19 5.4.6 Collecting Process.....................................19 5.4.7 Failover...............................................19 6. Message Layout..............................................20 7. IPFIX Message Format........................................21 7.1 Header Format.............................................21 7.2 Field Type Format.........................................22 7.3 Template FlowSet Format...................................23 7.4 Data FlowSet Format.......................................26 7.5 Options Template FlowSet Format...........................27 7.5.1 Options Template FlowSet Format........................28 Claise, et. al Standard Track [Page 2] IPFIX Protocol Specifications July 2004 7.5.2 Options Data Record Format.............................30 8. Specific Reporting Requirements.............................32 8.1 The Metering Process Statistics Option Template...........33 9. Export Packet "Export Time" Computation and Flow Record Time33 9.1 Microsecond Precision.....................................33 9.2 Millisecond Precision.....................................34 9.3 Nanosecond Precision......................................35 9.4 Multiple Precisions.......................................35 10. Linkage with the Information Model.........................35 10.1 Boolean..................................................35 10.2 Byte.....................................................35 10.3 UnsignedByte.............................................36 10.4 Short....................................................36 10.5 Reduced Size Encoding of Integral Types..................36 11. Variable Length Data Type..................................37 12. Template Management........................................38 13. The Collecting Process's Side..............................39 14. Security Considerations....................................40 14.1 IPsec Usage..............................................41 14.1.1 Selectors..............................................41 14.1.2 Mode...................................................41 14.1.3 Key Management.........................................42 14.1.4 Security Policy........................................42 14.1.5 Authentication.........................................42 14.1.6 Availability...........................................42 14.2 TLS Usage................................................42 14.3 Protection against DoS attacks...........................43 14.4 When IPsec or TLS is not an option.......................43 14.5 Logging an IPFIX Attack..................................44 15. IANA Considerations........................................44 16. Examples...................................................44 16.1 Message Header Example...................................45 16.2 Template FlowSet Example.................................45 16.3 Data FlowSet Example.....................................46 16.4 Options Template FlowSet Example.........................47 16.5 Data FlowSet with Options Data Records Example...........48 17. References.................................................48 17.1 Normative References.....................................48 17.2 Informative References...................................49 18. Acknowledgments............................................50 1. Points of Discussion 1.1 Open Issues This section covers the open issues, still to be resolved/updated in this draft: Claise, et. al Standard Track [Page 3] IPFIX Protocol Specifications July 2004 Issues in the Terminology section PROTO-1: Is flowSet the right term to use? - leave as is - Record Set - Record Array - Record Collection - Record List PROTO-2: Some discrepancies between data types, field type and Information Element terminology. - field type (IPFIX-PROTO) conflicts with field ID (IPFIX-INFO) - suggestion: use field type instead of field Id in IPFIX-INFO - rename 'type' to 'data type' and 'info elements' to 'fields' in IPFIX-INFO Issues in the Transport Protocol section PROTO-4: TCP section not yet covered. Starting point: draft-leinen- ipfix-tcp-00.txt. Ideally the same structure as SCTP and UDP should be preserved. Issues in the IPFIX Message Format section PROTO-16: relationship between several different scopes in one record PROTO-17: redefine scope values? - 1 System 2 IP interface 3 observation domain (SID) (preciously called line card) 4 reserved (previously used for cache) 5 template 6 metering process? 7 flow recording process? 8 exporting process? 9 observation point? PROTO-18: Can we have an optional length of 0 bytes for the scope section in the option template? PROTO-19: Do we really need different templates formats for flows and options? PROTO-20: Do we really need different record formats for flows and options? Claise, et. al Standard Track [Page 4] IPFIX Protocol Specifications July 2004 Issues in the Specific Reporting Requirement section PROTO-21: Do we need to define some mandatory content of the metering process statistics option template? - Maurizio suggested text on the mailing list Issues in the Export Packet "Export Time" Computation and Flow Record Time section PROTO-23: Finalize the time details Issues in the Linkage with the Information Model section PROTO-24: Section 11 "Linkage with the information model" must be completed with types used in [IPFIX-INFO] Issues in the Template Management section PROTO-25: The section 11 "Template Management" will have to updated according to the transport protocol. - For example, the point 2 of the section "Template Management". Remark: the template management will vary with TCP, SCTP, etc... Must have both sections updated: transport updated and template management sections (BTW, this is the same for the failover section). PROTO-29: The insertion of the new transport protocol sections has highlighted some inconsistency in the section 13 (template management) and the section 14 (The Collecting Process's Side). Issues in the IANA section PROTO-26: IANA considerations section to be updated: have a look at RFC 2434, which sets out guidelines for IANA Considerations. Also, searching the RFCs for 'IANA Considerations' brings up quite a few RFCs to look at as models. Issues - Miscellaneous PROTO-27: Correct the examples: no more FlowSet 0 and 1 Claise, et. al Standard Track [Page 5] IPFIX Protocol Specifications July 2004 1.2 Action Items This section covers the action items for this draft PROTO-29: number all the figures PROTO-30: Review the requirements draft to see what we miss, once it's an I-RFC 2. Introduction A data network with IP traffic, primarily consists of IP Flows passing through the network elements of the network. It is often interesting, useful or even a requirement to have access to information about these flows that pass through the network elements for administrative or other purposes. The IPFIX collecting process should be able to receive the flow information passing through multiple network elements within the data network. This requires uniformity in the method of representing the flow information and the means of communicating the flows from the network elements to the collection point. This document specifies the protocol to achieve these afore mentioned requirements. This document specifies in detail the representation of different flows, the additional data required for flow interpretation, packet format, transport mechanisms used, security concerns, etc. 2.1 IPFIX Documents Overview The IPFIX protocol provides network administrators with access to IP flow information. The architecture for the export of measured IP flow information out of an IPFIX exporting process to a collecting process is defined in [IPFIX-ARCH], per the requirements defined in [IPFIX-REQ]. [IPFIX-PROTO] specifies how IPFIX flow record data, options record data and control information is carried via a congestion-aware transport protocol from IPFIX exporting process to IPFIX collecting process. IPFIX has a formal description of IPFIX information elements (fields), their name, type and additional semantic information, as specified in [IPFIX-INFO]. Finally [IPFIX- AS] describes what type of applications can use the IPFIX protocol and how they can use the information provided. It furthermore shows how the IPFIX framework relates to other architectures and frameworks. Claise, et. al Standard Track [Page 6] IPFIX Protocol Specifications July 2004 The IPFIX protocol supports packet sampling. The methods of metering packet samples are out of scope of this specification. 3. Terminology The definition of the basic terms like IP Traffic Flow, Exporting Process, Collecting Process, Observation Points etc. are semantically identical with that found in the IPFIX requirements document [IPFIX-REQ]. Some of the terms have been expanded for more clarity when defining the protocol. Additional terms required for the protocol has also been defined. For the same terms defined in both this document and [IPFIX-ARCH], the definitions are identical with [IPFIX-ARCH]. The terminology summary table in Section 3.1 gives a quick overview of the relationships between some of the different terms defined. Observation Point The Observation Point is a location in the network where IP packets can be observed. Examples are a line to which a probe is attached, a shared medium such as an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) of a router. Note that one Observation Point may be a superset of several other Observation Points. For example one Observation Point can be an entire line card. This would be the superset of the individual Observation Points at the line card's interfaces. Observation Domain The set of Observation Points, which is the largest aggregatable set of Flow information at the Metering Process is termed an Observation Domain. Each Observation Domain presents itself as a unique ID to the Collecting Process for identifying the IPFIX Messages it generates. For example, a router line card composed of several interfaces with each interface being an Observation Point. Every Observation Point is associated with an Observation Domain. IP Traffic Flow or Flow Claise, et. al Standard Track [Page 7] IPFIX Protocol Specifications July 2004 There are several definitions of the term 'flow' being used by the Internet community. Within the context of IPFIX we use the following one: A flow is defined as a set of IP packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties. Each property is defined as the result of applying a function to the values of: 1. one or more packet header field (e.g. destination IP address), transport header field (e.g. destination port number), or application header field (e.g. RTP header fields [RFC1889]) 2. one or more characteristics of the packet itself (e.g. number of MPLS labels, etc...) 3. one or more of fields derived from packet treatment (e.g. next hop IP address, the output interface, etc...) A packet is defined to belong to a flow if it completely satisfies all the defined properties of the flow. This definition covers the range from a flow containing all packet observed at a network interface to a flow consisting of just a single packet between two applications with a specific sequence number. Flow Key Each of the fields which belong to 1. Packet header (e.g. destination IP address) 2. Property of the packet itself (e.g. packet length) 3. Derived from packet treatment (e.g. AS number) which is used to define a Flow is termed as Flow Key. Flow Type A function F which would take input as a set of Flow Keys and produce as output one or more Flows depending on the combination of values for the set of Flow Keys. Claise, et. al Standard Track [Page 8] IPFIX Protocol Specifications July 2004 Flow Record A Flow Record contains information about a specific Flow that was observed at an Observation Point. A Flow Record contains measured properties of the Flow (e.g. the total number of bytes of all packets of the Flow) and usually characteristic properties of the Flow (e.g. source IP address). Metering Process The Metering Process generates Flow Records. Input to the process are packet headers observed at an Observation Point and packet treatment at the Observation Point, for example the selected output interface. The Metering Process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining Flow Records. The maintenance of Flow Records may include creating new records, updating existing ones, computing Flow statistics, deriving further Flow properties, detecting Flow expiration, passing Flow Records to the Exporting Process, and deleting Flow Records. Exporting Process The Exporting Process sends Flow Records to one or more Collecting Processes. The Flow Records are generated by one or more Metering Processes. IPFIX Device A device hosting at least an Observation Point, a Metering Process and an Exporting Process. Typically, corresponding Observation Point(s), Metering Process(es) and Exporting Process(es) are co- located at this device, for example at a router. IPFIX Node An IPFIX node is a host that implements the IPFIX protocol which means it contains an Exporting Process or a Collecting Process or both. Collecting Process Claise, et. al Standard Track [Page 9] IPFIX Protocol Specifications July 2004 The Collecting Process receives Flow Records from one or more Exporting Processes. The Collecting Process might store received Flow Records or further process them, but these actions are out of the scope of this document. Collector The device which hosts one or more Collecting Processes. Flow Recording Process The Flows generated from the metering device(s) in an Observation Domain could be collected into one or more database before exporting. This functional block in addition to maintaining the Flow database(s) does Flow aggregation, maintain the aggregate statistics etc. This block is optional for an IPFIX device. Template Template is an ordered n-tuple (e.g. , TLV), used to completely identify the structure and semantics of a particular information that needs to be communicated from the IPFIX Device to the Collector. Each template is uniquely identifiable by some means (e.g. by using a Template ID). Control Information, Data Stream The information that needs to be exported from the IPFIX device can be classified into the following categories: - Control Information: This includes the Flow type definition, selection criteria for packets within the Flow sent by the Exporting Process and any IPFIX protocol messages (e.g. keepalives). The 'control' stream carries all the information needed for the end-points to understand the IPFIX protocol, and specifically for the receiver to understand and interpret the data sent by the sender. - Data Stream: Claise, et. al Standard Track [Page 10] IPFIX Protocol Specifications July 2004 This includes data records carrying the field values for the various observed Flows at each of the Observation Point. A sequence of such records may also be described as a Data Stream. IPFIX Message An IPFIX Message is a message originating at the Exporting Process that carries the IPFIX records of this Exporting Process and whose destination is the Collecting Process. An IPFIX Message is encapsulated within a transport layer header. Message Header The Message Header is the first part of an IPFIX Message, which provides basic information about the message such as the IPFIX version, length of the message, message sequence number, etc. Template Record A Template Record defines the structure and interpretation of fields in a Flow Data Record. Flow Data Record A Flow Data Record is a data record that contains values of the Flow parameters corresponding to a Template Record. Options Template Record An Options Template Record defines the structure and interpretation of fields in an Options Data Record, including defining how to scope the applicability of the Options Data Record. Options Data Record The Options Data Record is a data record that contains values and scope information of the Flow measurement parameters, corresponding to an Options Template Record. FlowSet Claise, et. al Standard Track [Page 11] IPFIX Protocol Specifications July 2004 FlowSet is a generic term for a collection of records that have a similar structure. In an IPFIX Message, one or more FlowSets follow the Message Header. There are three different types of FlowSets: Template FlowSet, Options Template FlowSet, and Data FlowSet. Template FlowSet A Template FlowSet is a collection of one or more Template Records that have been grouped together in an IPFIX Message. Options Template FlowSet An Options Template FlowSet is a collection of one or more Options Template Records that have been grouped together in an IPFIX Message. Data FlowSet A Data FlowSet is one or more records, of the same type, that are grouped together in an IPFIX Message. Each record is either a Flow Data Record or an Options Data Record previously defined by a Template Record or an Options Template Record. Information Element An Information Element is a protocol and encoding independent description of an attribute which may appear in an IPFIX Flow Record. The IPFIX information model [IPFIX-INFO] defines the base set of Information Elements for IPFIX. The type associated with an Information Element indicates constraints on what it may contain and also determine the valid encoding mechanisms for use in IPFIX. 3.1 Terminology Summary Table +------------------+---------------------------------------------+ | | Contents | | +--------------------+------------------------+ | FlowSet | Template Record | Data Record | +------------------+--------------------+------------------------+ | | | Flow Data Record(s) | | Data FlowSet | / | or | | | | Options Data Record(s) | Claise, et. al Standard Track [Page 12] IPFIX Protocol Specifications July 2004 +------------------+--------------------+------------------------+ | Template FlowSet | Template Record(s) | / | +------------------+--------------------+------------------------+ | Options Template | Options Template | / | | FlowSet | Record(s) | | +------------------+--------------------+------------------------+ A Data FlowSet is composed of an Options Data Record(s) or Flow Data Record(s). No Template Record is included. A Template Record defines the Flow Data Record, and an Options Template Record defines the Options Data Record. A Template FlowSet is composed of Template Record(s). No Flow or Options Data Record is included. An Options Template FlowSet is composed of Options Template Record(s). No Flow or Options Data Record is included. 4. Criteria for Flow Expiration and Export 4.1 Flow Expiration A Flow is considered as expired under the following conditions: 1. If the Metering Process can detect the end of a Flow. For example, if the FIN or RST bit is detected in a TCP [TCP] connection. 2. If no packets belonging to the Flow have been observed for a certain period of time. This time period SHOULD be configurable at the Metering Process. Note that if the time period is set to 0, the Metering Process will create a Flow for every single packet observed. 3. If the Metering Process experiences internal constraints, a Flow MAY be expired forcibly. For example, counters wrapping or low memory. 4.2 Flow Export A flow can be exported because it expired due to the reasons mentioned in Flow Expiration section. The exporting process decides when and whether to export an expired flow. For example: the Claise, et. al Standard Track [Page 13] IPFIX Protocol Specifications July 2004 exporting process exports a portion of the expired flows every 'x' seconds. For long-lasting Flows, the Exporting Process SHOULD export the Flow Records on a regular basis or based on some export policy. This periodicity or export policy SHOULD be configurable at the Metering Process. 5. Transport Protocol The IPFIX Protocol Specifications have been designed to be transport protocol independent. Note that the Exporter can export to multiple Collecting Processes, using independent transport protocols. 5.1 Transport Compliance and Transport Usage We must differentiate between what must be implemented (so that operators can interoperably deploy compliant implementations from different vendors) and what should or could be used in various operational environments. We must also make sure that ALL implementations can operate in a congestion-aware and congestion avoiding mode. SCTP [RFC2960] and SCTP-PR [RFC3758] MUST be implemented by all compliant implementations. UDP [UDP] MAY also be implemented by compliant implementations. TCP [TCP] MAY also be implemented by compliant implementations. SCTP-PR SHOULD be used in deployments where exporters and collectors are communicating over links which are susceptible to congestion. SCTP-PR is capable of providing any required degree of reliability. TCP MAY be used in deployments where exporters and collectors communicate over links which are susceptible to congestion, but SCTP-PR is preferred, due to its ability to limit back pressure on exporters and its message vs. stream orientation. Other non-congestion aware protocols (like UDP) MAY be used in deployments where exporters and collectors always communicate over dedicated links which are not susceptible to congestion. Claise, et. al Standard Track [Page 14] IPFIX Protocol Specifications July 2004 5.2 TCP EDITOR NOTE: to be completed. A good starting point is draft-leinen- ipfix-tcp-00.txt. TCP [TCP] 5.3 SCTP This section describes how IPFIX can be transported over SCTP [RFC2960] using the PR-SCTP [RFC3758] extension. 5.3.1 Congestion Avoidance The SCTP transport protocol provides the required level of congestion avoidance by design. 5.3.2 Reliability The SCTP transport protocol is by default reliable, but has the capability to operate in unreliable and partially reliable modes [RFC3758]. Using reliable SCTP streams (referred to hereafter as "streams") for the IPFIX export is not in itself a guarantee that all Data Records are delivered. If there is congestion on the link from the Exporting Process to the Collecting Process, or if a significant number of retransmissions are required, the send queues on the Exporting Process may fill up: the Exporting Process MAY either suspent export or discard IPFIX Messages. If Data Records are discarded the sequence numbers used for export MUST reflect the loss of data. 5.3.3 MTU SCTP provides the required IPFIX Message fragmentation service based on path MTU discovery. 5.3.4 Exporting Process 5.3.4.1 Association The Exporting Process MUST create at least one association (connection "bundle" in SCTP terminology) to the Collecting Process. However, the Exporting Process MAY create more than one association. Claise, et. al Standard Track [Page 15] IPFIX Protocol Specifications July 2004 The Collecting Process MUST NOT initiate the connection. 5.3.4.2 Source ID The IPFIX Message MUST contain a Message Header, which includes a source id (SID). The Exporting Process uses the SID to uniquely identify to the Collecting Process the Observation Domain that metered the Flows. If a Metering Process consists of a single Observation Domain, a single SID value MUST be used for all IPFIX Messages. If a Metering Process consists of multiple Observation Domains, a unique SID value for each Observation Domain MUST be used. 5.3.4.3 Stream An Exporting Process MUST request at least two outbound streams per association. The first stream (referred to as stream zero in the rest of the document), is used to send the Template FlowSet and the Options Template FlowSet. Stream zero MUST be fully reliable. Data FlowSets MUST NOT be sent on stream zero. Depending on the application requirement, the Exporting Process selects the mode (unreliable, partially reliable, or fully reliable mode) of the stream, used to send the Data FlowSets. Unreliable mode MAY be used where the application does not require reliable transmission and the use of a retransmission queue is impractical. An Exporter MAY use multiple streams to export Data FlowSet, in some cases different applications will have different requirements in terms of reliability. In such a case, the Observation Domain MUST use the same SID value on all of the multiple streams it uses. Data FlowSets from multiple Observation Domains MUST NOT be transmitted over the same stream; the Collecting Process should however verify that the SID values are the expected values. When Data FlowSets are exported over a partial reliable stream, they SHOULD be marked for retransmission as long as there is room in the SCTP send queues. However during times of congestion or other retransmission events, if the queue overflows, the oldest data record that has been transmitted and marked as partially reliable should be freed and marked to be skipped per the PR-SCTP [RFC3758] Claise, et. al Standard Track [Page 16] IPFIX Protocol Specifications July 2004 specification. The freed buffer space should then be re-used for the new Data FlowSets being exported. 5.3.4.4 Template The Templates FlowSet and Option Template FlowSet MUST be sent on stream zero with full reliability. New Template Records SHOULD be transmitted as soon as they are created on the Metering Process, and preferably before any associated Data Record is transmitted. The Collecting Process SHOULD accept Data Records without the associated Template Record. 5.3.5 Collecting Process The Collecting Process SHOULD listen for a new association request from the Exporting Process. The Exporting Process will request a number of streams to use for export. A Collecting Process MUST support a least two inbound streams per association. A Exporting Process and Collecting Process MAY ask for and support more than two streams. The Collecting Process SHOULD verify that the received IPFIX Messages inside one stream does not have differing SID values and silently discard any data that does NOT meet the collected values. The Exporting Process SHOULD NOT transmit messages inside one stream with multiple SID values. The correlated Flow Records are then treated like a normal export Flow. If the Collecting Process receives an IPFIX Message that it cannot decode, it MUST reset the SCTP association, discard the message and log the error. When an SCTP association is closed, the Collecting Process MUST discard all templates received over that association and stop decoding IPFIX Messages that use those templates. 5.3.6 Failover If the Collecting Process does not acknowledge the attempt by the Exporting Process to establish an association it will retry using the SCTP exponential backoff feature. The Exporter MAY log an alarm Claise, et. al Standard Track [Page 17] IPFIX Protocol Specifications July 2004 if the time to establish the association exceeds a specified threshold. If Collecting Process failover is supported by the Exporting Process a second SCTP association MAY be opened in advance. 5.4 UDP This section describes how IPFIX can be transported over UDP [RFC768] 5.4.1 Congestion Avoidance UDP has no integral congestion avoidance mechanism. Its use over congestion sensitive network paths is therefore deprecated. UDP MAY be used in deployments where exporters and collectors always communicate over dedicated links which are not susceptible to congestion. 5.4.2 Reliability UDP is not a reliable transport protocol, and cannot guarantee delivery of messages. IPFIX Messages sent from the Exporting Process to the Collecting Process using UDP may therefore be lost. UDP MUST NOT be used unless the application can tolerate some loss of Messages. The IPFIX Message sequence number in the packet header indicates the lost of IPFIX Messages to the Collecting Process. Templates sent from the Exporting Process to the Collecting Process using UDP as a transport MUST be resent at regular intervals in case previous copies were lost. Implementations MAY send templates using a reliable transport protocol, and send IPFIX Flow and Option Data Records using UDP as the transport protocol. 5.4.3 MTU The maximum size of exported messages MUST be configured such that the total packet size does not exceed the path MTU. Claise, et. al Standard Track [Page 18] IPFIX Protocol Specifications July 2004 5.4.4 Port Numbers The UDP destination port is set by manual configuration at both Exporting Process and Collecting Process. The UDP source port is allocated from the dynamic and/or private ports space. 5.4.5 Exporting Process The Exporting Process MAY duplicate the IPFIX Message to the several Collecting Process. 5.4.5.1 Template If sent using UDP as the transport protocol, Templates FlowSets and Option Template FlowSets MUST be re-sent at regular intervals. New Template Records SHOULD be transmitted as soon as they are created on the Metering Process, and before any associated Data Record is transmitted. The Collecting Process SHOULD accept Data Records without the associated Template Record. 5.4.6 Collecting Process If the Collecting Process receives an IPFIX Message that it cannot decode, it MUST discard the message and log the error. The Collecting Process must associate a lifetime with each Template received via UDP. If the template is not refreshed by the Exporting Process before that lifetime has expired, the Collecting process MUST discard the Template. The Collecting Process MUST NOT decode Flow or Option Data Records which have an expired Template. 5.4.7 Failover Because UDP is not a connection oriented protocol, the Exporting Process is unable to determine from the transport protocol that the Collecting Process is no longer able to receive the IFPIX Messages. Therefore, it can not invoke a failover mechanism. However, the Exporting Process MAY duplicate the IPFIX Message to the several Claise, et. al Standard Track [Page 19] IPFIX Protocol Specifications July 2004 Collecting Process. 6. Message Layout An IPFIX Message consists of a Message Header followed by one or more FlowSets. The FlowSets can be any of the possible three types: Template, Data, or Options Template. IPFIX Message: +--------+-------------------------------------------+ | | +----------+ +---------+ +----------+ | |Message | | Template | | Data | | Options | | | Header | | FlowSet | | FlowSet | | Template | ... | | | | | | | | FlowSet | | | | +----------+ +---------+ +----------+ | +--------+-------------------------------------------+ A FlowSet ID is used to distinguish the different types of FlowSets. FlowSet IDs lower than 256 are reserved for special FlowSets, such as the Template FlowSet (ID 0) and the Options Template FlowSet (ID 1). The Data FlowSets have a FlowSet ID greater than 255. The format of the Template, Data, and Options Template FlowSets will be discussed later in this document. The Exporter MUST code all binary integers of the Message Header and the different FlowSets in network byte order (also known as the big-endian byte ordering). Following are some examples of IPFIX Messages: 1. An IPFIX Message consisting of interleaved Template, Data, and Options Template FlowSets-A newly created Template is exported as soon as possible. So if there is already an IPFIX Message with a Data FlowSet that is being prepared for export, the Template and Option FlowSets are also interleaved with this information, subject to availability of space. IPFIX Message: +--------+--------------------------------------------------------+ | | +----------+ +---------+ +-----------+ +---------+ | |Message | | Template | | Data | | Options | | Data | | | Header | | FlowSet | | FlowSet | ... | Template | | FlowSet | | | | | | | | | FlowSet | | | | | | +----------+ +---------+ +-----------+ +---------+ | Claise, et. al Standard Track [Page 20] IPFIX Protocol Specifications July 2004 +--------+--------------------------------------------------------+ 2. An IPFIX Message consisting entirely of Data FlowSets-After the appropriate Template Records have been defined and transmitted to the Collecting Process, the majority of IPFIX Messages consists solely of Data FlowSets. IPFIX Message: +--------+----------------------------------------------+ | | +---------+ +---------+ +---------+ | |Message | | Data | ... | Data | ... | Data | | | Header | | FlowSet | ... | FlowSet | ... | FlowSet | | | | +---------+ +---------+ +---------+ | +--------+----------------------------------------------+ 3. An IPFIX Message consisting entirely of Template and Options Template FlowSets-The Exporter MAY transmit a message containing Template and Options Template FlowSets periodically to help ensure that the Collecting Process has the correct Template Records and Options Template Records when the corresponding Flow Data records are received. IPFIX Message: +--------+-------------------------------------------------+ | | +----------+ +----------+ +----------+ | |Message | | Template | | Template | | Options | | | Header | | FlowSet | ... | FlowSet | ... | Template | | | | | | | | | FlowSet | | | | +----------+ +----------+ +----------+ | +--------+-------------------------------------------------+ 7. IPFIX Message Format 7.1 Header Format 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version Number | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Export Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | Claise, et. al Standard Track [Page 21] IPFIX Protocol Specifications July 2004 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Message Header Field Descriptions Version Version of Flow Record format exported in this message. The value of this field is 0x000a for the current version. Length Total Length is the length of the IPFIX message, measured in octets, including message Header and FlowSet(s). Export Time Time in seconds since 0000 UTC 1970, at which the Export Packet leaves the Exporter. Sequence Number Incremental sequence counter of all IPFIX Messages sent from the current Observation Domain by the Exporting Process. This value MUST SHOULD be used by the Collecting Process to identify whether any IPFIX Messages have been missed. Source ID A 32-bit value that identifies the Exporter Process Observation Domain. Collecting Process SHOULD use the combination of the source IP address and the Source ID field to separate different export streams originating from the same Exporting Process. 7.2 Field Type Format Vendors need the ability to define proprietary Information Elements, because, for example, they are delivering pre-standards product, or the Information Element is in some way commercially sensitive. This section describes the Field Type format for both IETF specified Information Elements [IPFIX-INFO] and Vendor Specified Information Elements, both the Template FlowSet and the Option Template FlowSet. The Field Ids used to identify Information Elements are represented by the Field Type. When the Enterprise Field Type bit is set to 0, Claise, et. al Standard Track [Page 22] IPFIX Protocol Specifications July 2004 the corresponding Field Type will report an IETF specified Information Elements. When the Enterprise Field Type bit is set to 1, the corresponding Field Type will report a Vendor Specified Information Elements. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |E| Field Type | Field Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Enterprise Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Fig X: Field Type Where: E Enterprise Field Type. If this bit is zero, the Field Type identifies an IETF specified Information Element, and the four octet Enterprise Number field MUST NOT be present. If this bit is one, the Field Type identifies a Vendor Specified Information Element, and the Enterprise Number filed MUST be present. This definition is backwards compatible with Netflow v9 [NF RFC] Field Type A numeric value that represents the type of the field. Refer to [IPFIX-INFO]. Field Length The length of the corresponding Field Type, in bytes. Refer to [IPFIX-INFO]. Enterprise Number IANA enterprise number [PEN] of the authority defining the field type in this template record. 7.3 Template FlowSet Format One of the essential elements in the IPFIX format is the Template FlowSet. Templates greatly enhance the flexibility of the Flow Claise, et. al Standard Track [Page 23] IPFIX Protocol Specifications July 2004 Record format because they allow the Collecting Process to process Flow Records without necessarily knowing the interpretation of all the data in the Flow Record. A Template FlowSet MAY exclusively contain IETF defined Field Types. A Template FlowSet MAY contain Vendor Specified Information Elements from multiple vendors. The format of the Template FlowSet is shown in Figure D. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 2 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 1 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Enterprise Number 1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type N | Field Length N | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Enterprise Number 1.N | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 2 | Field Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 1 | Field Length 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type 2 | Field Length 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Enterprise Number 2.2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Type M | Field Length M | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Enterprise Number 2.M | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Padding (opt) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Claise, et. al Standard Track [Page 24] IPFIX Protocol Specifications July 2004 Figure D: Template Flowset The Template FlowSet Field Definitions are as follows: FlowSet ID FlowSet ID value of 2 is reserved for the Template FlowSet Length Total length of this FlowSet. Because an individual Template FlowSet MAY contain multiple Template Records, the Length value MUST be used to determine the position of the next FlowSet record, which could be any type of FlowSet. Length is the sum of the lengths of the FlowSet ID, the Length itself, and all Template Records within this FlowSet. Template ID Each of the newly generated Template Records is given a unique Template ID. This uniqueness is local to the Observation Domain that generated the Template ID. Template IDs 0-255 are reserved for Template FlowSets, Options FlowSets, and other reserved FlowSets yet to be created. Template IDs of Data FlowSets are numbered from 256 to 65535. Field Count Number of fields in this Template Record. Because a Template FlowSet usually contains multiple Template Records, this field allows the Collecting Process to determine the end of the current Template Record and the start of the next. Field Type A numeric value that represents the type of the field. Refer to [IPFIX-INFO]. Field Length The length of the corresponding Field Type, in bytes. Refer to [IPFIX-INFO]. Enterprise Number IANA enterprise number [PEN] of the authority defining the field type in this template record. Claise, et. al Standard Track [Page 25] IPFIX Protocol Specifications July 2004 Padding The Exporting Process MAY insert some padding bytes, so that the subsequent FlowSet starts at an aligned boundary. Padding MUST be composed of zero (0) bits. The padding length MUST be shorter than one Template Record. It is important to note that the Length field includes the padding bits. Because Template Flowsets are always 4-byte aligned by definition padding is only needed in case of other alignments e.g. on 8-byte boundaries. The FlowSet ID value of 0 is reserved for backwards compatibility of the Template FlowSet: the Template specified in this document is a superset of the Template specified in [NETFLOW9]. The Template specified in this document that would contain only IETF defined Field Types would be equivalent to the Template specification in [NETFLOW9], with the only exception that the FlowSet ID value is 0. 7.4 Data FlowSet Format The format of the Data FlowSet is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = Template ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 1 | Record 1 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 1 | Record 2 - Field Value 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Field Value 3 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 3 - Field Value 1 | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Padding (opt) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Note that not all Field Values do necessarily have a length of 16 bit. Data FlowSet Field Descriptions Claise, et. al Standard Track [Page 26] IPFIX Protocol Specifications July 2004 FlowSet ID = Template ID Each Data FlowSet is associated with a FlowSet ID. The FlowSet ID maps to a (previously generated) Template ID. The Collecting Process MUST use the FlowSet ID to find the corresponding Template Record and decode the Flow Records from the FlowSet. Length The length of this FlowSet. Length is the sum total of lengths of FlowSet ID, Length itself, all Flow Records within this FlowSet, and the padding bytes, if any. Record N - Field Value M The remainder of the Data FlowSet is a collection of Flow Data Record(s), each containing a set of field types and values. The Type and Length of the fields have been previously defined in the Template Record referenced by the FlowSet ID or Template ID. Padding The Exporting Process MAY insert some padding bytes, so that the subsequent FlowSet starts at an aligned boundary. Padding MUST be composed of zero (0) bits. The padding length MUST be shorter than one Flow Data Record. It is important to note that the Length field includes the padding bits. Interpretation of the Data FlowSet format can be done only if the Template FlowSet corresponding to the Template ID is available at the Collecting Process. 7.5 Options Template FlowSet Format The Options Template Record (and its corresponding Options Data Record) is used to supply information about the Metering Process configuration or Metering Process specific data, rather than supplying information about IP Flows. For example, the Options Template FlowSet can report the sample rate of a specific interface, if sampling is supported, along with the sampling method used. Claise, et. al Standard Track [Page 27] IPFIX Protocol Specifications July 2004 7.5.1 Options Template FlowSet Format A Options Template MAY exclusively contain IETF defined Field Types. A Options Template MAY contain Vendor Specified Information Elements from multiple vendors. The format of the Options Template FlowSet is shown in Figure E. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 3 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID | Option Scope Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option Length | Scope 1 Field Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope 1 Field Length | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Scope N Field Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope N Field Length | Scope N... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... Enterprise Number | Option 1 Field Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option 1 Field Length | Option 1 ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ...Enterprise Number | ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Option N Field Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option N Field Length | Padding (opt) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure E: Option Template FlowSet The Options Template FlowSet Field Definitions are as follows: FlowSet ID = 1 A FlowSet ID value of 3 is reserved for the Options Template. Claise, et. al Standard Track [Page 28] IPFIX Protocol Specifications July 2004 Length Total length of this FlowSet, including the padding bytes, if any. Each Options Template FlowSet MAY contain multiple Options Template Records. Thus, the Length value MUST be used to determine the position of the next FlowSet record, which could be either a Template FlowSet or Data FlowSet. Length is the sum total of lengths of FlowSet ID, the Length itself, and all Options Template Records within this FlowSet Template ID. Template ID Template ID of this Options Template. This value is greater than 255. Option Scope Length The length in bytes of any Scope fields definition contained in the Options Template Record (The use of "Scope" is described below). Option Length The length (in bytes) of any options field definitions contained in this Options Template Record. Scope 1 Field Type The relevant portion of the Exporting Process/Metering Process to which the Options Data Record refers. Currently defined values are: 1 System 2 Interface 3 Line Card 4 Cache 5 Template For example, the Metering Process can be implemented on a per-interface basis, so if the Options Template Record were reporting on how the Metering Process is configured, the Scope for the report would be 2 (interface). The associated interface ID would then be carried in the associated Options Data FlowSet. The Scope can be limited further by listing multiple scopes that all must match at the same time. Note that the Scope fields always precede the Option fields. Scope 1 Field Length Claise, et. al Standard Track [Page 29] IPFIX Protocol Specifications July 2004 The length (in bytes) of the Scope field, as it would appear in an Options Data Record. Scope N Enterprise Number IANA enterprise number [PEN] of the authority defining Scope N. Option 1 Field Type A numeric value that represents the type of field that would appear in the Options Template Record. Refer to [IPFIX- INFO]. Option 1 Field Length The length (in bytes) of the Option Field. Option N Enterprise Number IANA enterprise number [PEN] of the authority defining the Option N field type. Padding The Exporting Process MAY insert some padding bytes, so that the subsequent FlowSet starts at an aligned boundary. Padding MUST be composed of zero (0) bits. The padding length MUST be shorter than one Options Template Record. It is important to note that the Length field includes the padding bits. The FlowSet ID value of 1 is reserved for backwards compatibility of the Option Template FlowSet: the Option Template specified in this document is a superset of the Option Template specified in [NETFLOW9]. The Option Template specified in this document that would contain only IETF defined Field Types would be equivalent to the Option Template specification in [NETFLOW9], with the only exception that the FlowSet ID value is 1. 7.5.2 Options Data Record Format The Options Data Records are sent in Data FlowSets, on a regular basis, but not with every Flow Data Record. How frequently these Options Data Records are exported is configurable. See the Templates Management" section for more details. Claise, et. al Standard Track [Page 30] IPFIX Protocol Specifications July 2004 The format of the Data FlowSet containing Options Data Records follows. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = Template ID | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 1 - Scope 1 Value | Record 1 - Scope 2 Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... |Record 1 - Option Field 1 Value| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Record 1 - Option Field 2 Value| ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 2 - Scope 1 Value | Record 2 - Scope 2 Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... |Record 2 - Option Field 1 Value| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Record 2 - Option Field 2 Value| ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Record 3 - Scope 1 Value | Record 3 - Scope 2 Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... |Record 3 - Option Field 1 Value| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Record 3 - Option Field 2 Value| ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... | Padding (opt) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Options Data Records of the Data FlowSet Field Descriptions FlowSet ID = Template ID A FlowSet ID precedes each group of Options Data Records within a Data FlowSet. The FlowSet ID maps to a previously generated Template ID corresponding to this Options Template Record. The Collecting Process MUST use the FlowSet ID to map the appropriate type and length to any field values that follow. Length The length of this FlowSet. Claise, et. al Standard Track [Page 31] IPFIX Protocol Specifications July 2004 Length is the sum of the lengths of the FlowSet ID, Length itself, all the Options Data Records within this FlowSet, and the padding bytes, if any. Record N - Option Field M Value The remainder of the Data FlowSet is a collection of Flow Records, each containing a set of scope and field values. The type and length of the fields were previously defined in the Options Template Record referenced by the FlowSet ID or Template ID. Padding The Exporting Process MAY insert some padding bytes, so that the subsequent FlowSet starts at an aligned boundary. Padding MUST be composed of zero (0) bits. The padding length MUST be shorter than one Options Data Record. It is important to note that the Length field includes the padding bits. The Data FlowSet format can be interpreted only if the Options Template FlowSet corresponding to the Template ID is available at the Collecting Process. 8. Specific Reporting Requirements Some specific Options Templates and Options Templates Records are necessary to provide extra information about the Flow Records and about the Metering Process. The ipfixOption [IPFIX-INFO], always included in these specific Options Templates, defines the type of information sent in the Option Template / Option Template Record pair. For example, if the ipfixOption [IPFIX-INFO] value is METER_STATS, then the Option Template will specify information about the Metering Process statistics. The ipfixOption [IPFIX-INFO] MUST always be the first Data Type in the Option Template so that the Collector could quickly determine whether or not a specific Option Template is described. And if the ipfixOption [IPFIX-INFO] is present, which specific Option Template type it defines. The minimum set of Data Types is always specified in these Specific IPFIX Options Templates. Nevertheless, extra Data Types MAY be used in these specific Options Templates. Claise, et. al Standard Track [Page 32] IPFIX Protocol Specifications July 2004 8.1 The Metering Process Statistics Option Template The Metering Process Statistics Option Template defines the Metering Process Statistics with the export of the following Data Types [IPFIX- INFO]: ipfixOption The value MUST be METER_STATS observationDomain Source ID lostFlows flows not exported due to resource starvation lostFlowsPacket Packets in the lost flows lostFlowsBytes Bytes in the lost flows droppedPacketCount Packets dropped by Metering Process at the Observation Point droppedByteCount Bytes dropped by Metering Process at the Observation Domain time; When this record was generated The minimum set of Data Type in the Metering Process Statistics Option Template is: ipfixOption, observationDomain, lostFlows, time 9. Export Packet "Export Time" Computation and Flow Record Time 9.1 Microsecond Precision For a Data FlowSet with Flow Records requiring microsecond precision, the Export Packet "Export Time" field MUST be calculated so that each Flow Records flowStartUsec [IPFIX-INFO] and flowEndUsec [IPFIX-INFO] would contain a 32 bit signed microsecond offset from the "Export Time" base timestamp. Hereafter some pseudo code to calculate the Export Time in one pass, which would return an absolute duration of 35 minutes for all Flow Records contained in the Data FlowSet. Flow Records MUST be exported in different Export Packet if the absolute duration can not fit in those 35 minutes. // pseudo code for microsecond offset in IPFIX encoded Flow Records. // struct flow{ uint32 tv_sec; uint32 tv_usec; uint32 numbytes; ... // other information elements... }; Claise, et. al Standard Track [Page 33] IPFIX Protocol Specifications July 2004 struct flow flowtable [MAX_TABLE_SIZE]; int lastflowindex = -1; writeflows() { if (lastflowindex < 0) return; // simply take the second field from the first available flow // and make this the base time for this collection of flows. uint32 base_sec = flowtable[0].tv_sec; writeheaderToSocket(base_sec); // put 32-bit second value in header for (int i=0; i<=lastflowindex; i++){ int32 offset = (flowtable[i].tv_sec - base_sec) * 1000000 + flowtable[i].tv_usec; writeint32ToSocket(offset); // put the 32-bit time offset in the record. // write other information elements... } } A two pass approach calculation for the optimum (center) "Export Time" base timestamp would allow an absolute duration of 71 minutes for all Flow Records contained in the Data FlowSet. The two pass approach MAY be used. The "Export Time" base timestamp calculation requires that at the Export Packet exporting time the Exporting Process MUST run down the list of Flow Records in the Data FlowSet message and adjust the Flow start and Flow end timestamps. 9.2 Millisecond Precision For a Data FlowSet with Flow Records requiring a millisecond precision, the same principles as in section 10.1 "Microsecond Precision" will be used. The only difference will be that the Flow start and the Flow end SHOULD now be represented respectively by the flowStartMsec [IPFIX- INFO] and flowEndMsec [IPFIX-INFO]. As a consequence of the millisecond precision, the absolute duration of all Flow Records is now of about 49 days. The Export Header "Export Time" base time SHOULD be calculated with the algorithm described in the Section 10.1 "Microsecond Precision". In order to reduce the load on the Exporter, the Export Header "Export Time" MAY be the time in seconds Claise, et. al Standard Track [Page 34] IPFIX Protocol Specifications July 2004 since 0000 UTC 1970 at which the Export Packet leaves the Exporter and not the calculated optimum value anymore as described in section 10.1 "Microsecond Precision". Alternatively, for a Data FlowSet with Flow Records requiring a millisecond precision, the microsecond mechanism as described in section 10.1 MAY be used as such. The Flow Record MAY use the flowStartUsec [IPFIX-INFO] and flowEndUsec [IPFIX-INFO] rounded at a millisecond precision. 9.3 Nanosecond Precision For a Data FlowSet with Flow Records requiring a nanosecond precision, all Flow Records will contain Flow start flowStartNsec [IPFIX-INFO] and flowEndNsec [IPFIX-INFO]. The Export Header "Export Time" will be of no use on the Collector side in this case as the flowStartNsec [IPFIX-INFO] and flowEndNsec [IPFIX-INFO] both have a nanosecond precision already. Both flowStartNsec [IPFIX-INFO] and flowEndNsec [IPFIX-INFO] use the NTP time format which is represented as a 64-bit value which contains a 32-bit specification of seconds since 1900 and a 32-bit "fraction" field. Refer to the NTP specification, RFC1305, section 3.1 "Data Formats". 9.4 Multiple Precisions When Flow Records requiring different precisions must be exported, the Exporting Process SHOULD split the Flow Records in different Data FlowSet according to the precision: millisecond, microsecond or nanosecond. 10. Linkage with the Information Model The information model associates each IPFIX Data Type with a well defined type, such as hexBinary, long, unsignedInt, etc. This document defines how fields of a given type are encoded. 10.1 Boolean A boolean field shall be encoded in a single byte with the value of 0 indicating false and any other value indicating true. 10.2 Byte Claise, et. al Standard Track [Page 35] IPFIX Protocol Specifications July 2004 A byte value shall be encoded as a single byte representing a value between -128 and 127. The value is represented in two's complement notation. 10.3 UnsignedByte An unsigned byte value shall be encoded as a single byte representing a value between 0 and 255. 10.4 Short A short is a 16-bit datum that encodes an integer in the range [- 32768,32767]. The short is represented in two's complement notation. The most and least significant bytes are 0 and 1, respectively. 10.5 Reduced Size Encoding of Integral Types Information Elements containing integral types in the information model MAY be encoded using fewer bytes than those implied by their type in the information model definition [IPFIX-INFO], based on the assumption that the smaller type is sufficient to carry any value the Exporter may need to deliver. This reduces the network bandwidth requirement between the Exporter and the Collector. Note that the information model Data Types definition [IPFIX-INFO] will always define the maximum encoding size for each Data Type. For instance the information model [IPFIX-INFO] defines byteCount as an unsignedLong type, which would require 64-bits. However if the exporter will never locally encounter the need to send a value larger than 4294967295, it may chose to send the value instead as an unsignedInt. For example, a core router would require an unsignedLong byteCount while an unsignedInt might be sufficient for an access router. This behavior is indicated by the Exporter by specifying a type size smaller than that associated with the assigned type of the field. In the example above the Exporter would place a length of 4 vs. 8 in the template. Reduced sizing MAY only be applied to the following integral types: short, unsignedShort, int, unsignedInt, long, unsignedLong. In each case the downcasting must be to a smaller integral type which MUST have the same signed vs. unsigned properties. Claise, et. al Standard Track [Page 36] IPFIX Protocol Specifications July 2004 Specifically unsignedLong may be downcast to unsignedInt, unsignedShort or unsignedByte. A long may be downcast to an int a short or a byte. The other downcasts follow the same pattern. 11. Variable Length Data Type The IPFIX template mechanism is optimized for fixed length Information Elements [IPFIX-INFO]. Where an Information Element has a variable length the following mechanism MUST used to carry the length information. In the Template FlowSet the length is recorded as 65535. This reserved length value notifies the Collecting Process that length of the Information Element will be carried in the Information Element itself. In most cases the length of the Information Element will be less than 256 bytes. The following length encoding mechanism optimizes the overhead of carrying the Information Element length in this majority case. If the length of the Information Element is less than 255 bytes, the length is carried in the first byte of the Information Element. This is shown on Figure A. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length (< 255)| Information element | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... continuing as needed | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure A: Variable Length Information Element (length < 255 bytes) If the length of the Information Element is greater or equal than 256 bytes, the first byte of the Information Element is 255, and the length is carried in the second and third bytes of the Information Element. This is shown in Figure B. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 Claise, et. al Standard Track [Page 37] IPFIX Protocol Specifications July 2004 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 255 | Length (256 to 65535) | IE | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ... continuing as needed | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure B: Variable Length Information Element (length 256 to 65535) bytes 12. Template Management Flow Data records that correspond to a Template Record MAY appear in the same and/or subsequent IPFIX Messages. The Template Record is not necessarily carried in every IPFIX Message. As such, the Collecting Process MUST store the Template Record to interpret the corresponding Flow Data Records that are received in subsequent data messages. A Collecting Process that receives IPFIX Messages from several Observation Domains from the same Exporter MUST be aware that the uniqueness of the Template ID is not guaranteed across Observation Domains. The Template IDs must remain constant for the life of the Metering Process and the Exporting Process. If the Exporting Process or the Metering Process restarts for any reason, all information about Templates will be lost and new Template IDs will be created. Template IDs are thus not guaranteed to be consistent across an Exporting Process or Metering Process restart. A newly created Template record is assigned an unused Template ID from the Exporter. If the template configuration is changed, the current Template ID is abandoned and SHOULD NOT be reused until the Metering Process. If a Collecting Process should receive a new definition for an already existing Template ID, it MUST discard the previous template definition and use the new one. If a configured Template Record on the Exporting Process is deleted, and re-configured with exactly the same parameters, the same Template ID COULD be reused. The Exporting Process sends the Template FlowSet and Options Template FlowSet under the following conditions: Claise, et. al Standard Track [Page 38] IPFIX Protocol Specifications July 2004 1. After a Metering Process restarts, the Exporting Process MUST NOT send any Data FlowSet without sending the corresponding Template FlowSet and the required Options Template FlowSet in a previous message or including it in the same IPFIX Message. It MAY transmit the Template FlowSet and Options Template FlowSet, without any Data FlowSets, in advance to help ensure that the Collector will have the correct Template Record before receiving the first Flow or Options Data Record. 2. In the event of configuration changes, the Exporting Process SHOULD send the new template definitions at an accelerated rate. In such a case, it MAY transmit the changed Template Record(s) and Options Template Record(s), without any data, in advance to help ensure that the Collector will have the correct template information before receiving the first data. 3. If the Template Records and Options Template Records are sent using a transport protocol that is not fully reliable they MUST be refreshed on a regular basis by the Exporting Process which MUST re-send all the Template Records and Options Template Records to the Collecting Process. 13. The Collecting Process's Side The Collecting Process receives Template Records from the Exporting Process, normally before receiving Flow Data Records (or Options Data Records). The Flow Data Records (or Options Data Records) can then be decoded and stored locally on the devices. If the Template Records have not been received at the time Flow Data Records (or Options Data Records) are received, the Collecting Process SHOULD store the Flow Data Records (or Options Data Records) and decode them after the Template Records are received. A Collecting Process device MUST NOT assume that the Data FlowSet and the associated Template FlowSet (or Options Template FlowSet) are exported in the same IPFIX Message. The Collecting Process MUST NOT assume that one and only one Template FlowSet is present in an IPFIX Message. The life of a template at the Collecting Process is limited to a fixed refresh timeout. Templates not refreshed from the Exporting Process within the timeout are expired at the Collecting Process. Claise, et. al Standard Track [Page 39] IPFIX Protocol Specifications July 2004 The Collecting Process MUST NOT attempt to decode the Flow or Options Data Records with an expired Template. At any given time the Collecting Process SHOULD maintain the following for all the current Template Records and Options Template Records: Note that the Observation Domain is identified by the Source ID field from the IPFIX Message. Template IDs are unique per Exporting Process and per Observation Domain. If the Collecting Process receives a new Template Record (for example, in the case of an Exporter restart) it MUST immediately override the existing Template Record. The Collecting Process MUST note the Field ID of any Information Element that it does not understand and MAY discard the Information Element from the Flow Record. The Collecting Process MUST note the size and position of any Vendor Specified Information Element that it does not understand and discard the Information Element from the Flow Record. The Collector MUST accept padding in the Data FlowSet and Options Template FlowSet, which means for the Flow Data Records, the Options Data Records and the Template Records. Refer to the terminology summary table in Section 3.1. The IPFIX protocol has a sequence number field in the Export Header which increases with each IPFIX message. A Collector may detect out of sequence, dropped, or duplicate messages by tracking the sequence number. A collector SHOULD provide a logging mechanism for tracking out of sequence messages. Such out of sequence messages may be due to congestion on the network link between the Exporter and Collector, Collector resource exhaustion where it can not process the IPFIX messages at their arrival rate, Exporter resource exhaustion where it can not transmit messages at their creation rate, out of order packet reception, duplicate packet reception, an Exporting Process reset, or an attacker injecting false messages. 14. Security Considerations Because IPFIX can be used to collect billing information and network forensics, confusing or blinding IPFIX must be seen as a prime Claise, et. al Standard Track [Page 40] IPFIX Protocol Specifications July 2004 objective during a sophisticated network attack. If an attacker is in a position to inject false messages into an IPFIX message stream this will allow them to send forged flow records, options, or templates. Forged templates may impair the Collectors ability to process any further Flow Records. Forged Flow Records would have a direct effect on the application using the Flows, for example a billing system may generate incorrect billing information. Forged options may be able to alter the meaning of flow records, for example if the sample rate is changed. The IPFIX messages themselves may contain information of value to an attacker, and thus care must be taken to confine their visibility to authorized users. IPFIX messages can be secured using IPsec. Alternatively if IPFIX runs on top of SCTP or TCP TLS [TLS] can be used. 14.1 IPsec Usage To secure messages between the Exporter and the Collector an IPFIX implementation MAY use IPsec. To ensure interworking between Exporters and Collectors from different vendors, the following IPsec profile MUST be supported. This profile is derived from [USEIPSEC]. 14.1.1 Selectors IPFIX runs between manually configured pairs of hosts on the following transport ports (TBD). The appropriate selector would be Exporter-Collector pairs and port number. Note that, if the Exporter is a router, a non-interface ("loopback") address should be used. 14.1.2 Mode IPsec MUST be run in transport mode. The AH and ESP MUST be supported by an IPFIX implementation of IPsec. The Authentication Header (AH) [RFC2402] MUST be used if authentication is required. The Security Protocol (ESP) [RFC2406] must be used if the is a threat to the IPFIX message content, or if it is confidential. Claise, et. al Standard Track [Page 41] IPFIX Protocol Specifications July 2004 Normally in situations where the ESP was required the AH would also be required. If ESP only is used, the sender's IP address MUST be checked against the IP address asserted in the key management exchange. 14.1.3 Key Management In many networks, manual key management will be sufficient, and this reduces the complexity of the Exporter, albeit at a cost of greater configuration complexity. Manual key management MUST be supported. If a replay attack is considered likely, an automated key management the IKE [IKE] key management system SHOULD be used. 14.1.4 Security Policy Connections should be accepted only from the designated peer. 14.1.5 Authentication Given the number of IPFIX capable Exporters that are likely to be deployed by large ISPs, there will be circumstances where shared key mechanisms are not adequate. Where an automated key management system is used, certificate-based IKE SHOULD be supported. 14.1.6 Availability It is accepted that IPsec will not be universally available in IPFIX Exporters, and that where it is available, there may be issues of throughput, which may itself raise security issues. In such circumstances the other security measures described in this draft provide some threat mitigation. 14.2 TLS Usage The IPFIX Exporter initiating a connection acts as a TLS client according to [TLS], and an IPFIX Collector that accepts a connection acts as a TLS server. If mutual authentication is required the IPFIX node acting as TLS server MUST request a certificate from the IPFIX node acting as TLS client, and the IPFIX node acting as TLS client MUST be prepared to supply a certificate on request. Claise, et. al Standard Track [Page 42] IPFIX Protocol Specifications July 2004 14.3 Protection against DoS attacks An attacker may directly mount a DoS attack by generating large amounts of traffic. If TCP is used for transport, then the flow to the collector would back off due to congestion and eventually stall, blinding the IPFIX system. An attack could then proceed without further observation. SCTP-PR will have a different pathology under such an attack. Stale data at the head of the queue will get flushed giving some visibility of the attack. In case of UDP, IPFIX would reduce to some sort of sampling meaning that some forensics may be left. To avoid blinding of the IPFIX system some mechanism for service differentiation can be used to prioritizes IPFIX traffic over user traffic. An alternative is to use a dedicated network for the transport of IPFIX messages. By sending the IPFIX messages over a dedicated network, IPFIX message loss induced by user traffic congestion is minimized. However an attacker may trigger the generation of excessive IPFIX messages, and to avoid information loss during such an attack the IPFIX network must be adequately sized. 14.4 When IPsec or TLS is not an option The use of IPsec or TLS might not be an option because of performance issues. Without IPsec or TLS an IPFIX entity has no means to authenticate an IPFIX entity other than the Source IP address. Useful protection is gained by allocating Exporter and Collector IP addresses from ranges that are excluded from use by user traffic and preventing spoofing attacks by proper ingress filtering. Where large numbers of exporters, proxies and collectors are used in a network, it may be tempting for the administrator to not impose source IP address restrictions but this leaves a proxy or collector open to the reception of invalid information. Using an open proxy or collector is therefore to be deprecated. If IP address spoofing can not be prevented some level of protection against an insertion attack is required. With a modern implementation of TCP with good ISN randomization [XXX-REFERENCE] or SCTP insertion such attacks are difficult without the ability to snoop the packet flow [XXX-SCTP-BLIND-SPOOFING-REFERENCE]. UDP is vulnerable to insertion attacks however, randomization of the IPFIX sequence number might mitigate this problem. In all these cases, the sequence number space is relatively small giving only limited Claise, et. al Standard Track [Page 43] IPFIX Protocol Specifications July 2004 protection. Therefore a 64 bit cookie [L2TPv3] SHOULD be included as an element within all messages. The use of a dedicated network prevents IPFIX messages from being inspected by an attacker. 14.5 Logging an IPFIX Attack A Collector may detect problems by tracking the IPFIX sequence number and therefore SHOULD provide a logging mechanism for tracking out of sequence messages. [EDITORÆS NOTE: Double check whether this is already specified in an earlier section.] Such out of sequence messages may not only be caused by network congestion or Exporter/Collector resource exhaustion but also by an attacker injecting false messages. Note that an attacker may be able to exploit the behavior of the Collector when it receives an out of sequence message. For example a Collector that simply reset the expected sequence number upon receipt of a later message would easily be temporarily blinded by deliberately injecting messages with a much larger sequence number. [EDITORÆS NOTE: the security section may need be adapted to the revised transport section] 15. IANA Considerations IANA will need to set up a registry of Flowset IDs, field types, scope and option codepoints. In compiling the registry of field types IANA must set asside a range value for vendor use. It is proposed that the range <0..32767> be administered by IANA for IETF defined IEs, and that the range <32768..65535> be allocated for private use by vendors. Similarly the scope and option codepoints need to be split between IANA administered and private ranges. 16. Examples Let's consider the example of an IPFIX Message composed of a Template FlowSet, a Data FlowSet (which contains three Flow Data Records), an Options Template FlowSet and a Data FlowSet (which contains 2 Options Data Records). Claise, et. al Standard Track [Page 44] IPFIX Protocol Specifications July 2004 IPFIX Message: +--------+---------------------------------------------. . . | | +--------------+ +-----------------------+ |Message | | Template | | Data | | Header | | FlowSet | | FlowSet | . . . | | | (1 Template) | | (3 Flow Data Records) | | | +--------------+ +-----------------------+ +--------+---------------------------------------------. . . . . .+-------------------------------------------------+ +------------------+ +--------------------------+ | | Options | | Data | | . . .| Template FlowSet | | FlowSet | | | (1 Template) | | (2 Options Data Records) | | +------------------+ +--------------------------+ | . . .--------------------------------------------------+ 16.1 Message Header Example The Message Header is composed of: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version = 0x000a | Length = 152 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Export Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 16.2 Template FlowSet Example We want to report the following Field Types: - The source IP address (IPv4), so the length is 4 - The destination IP address (IPv4), so the length is 4 - The next-hop IP address (IPv4), so the length is 4 - The number of bytes of the Flow - The number of packets of the Flow Claise, et. al Standard Track [Page 45] IPFIX Protocol Specifications July 2004 Therefore, the Template FlowSet will be composed of the following: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 0 | Length = 28 bytes | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 256 | Field Count = 5 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP_SRC_ADDR = 0x0008 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP_DST_ADDR = 0x000C | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP_NEXT_HOP = 0x000F | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IN_PKTS = 0x0002 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IN_BYTES = 0x0001 | Field Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 16.3 Data FlowSet Example In this example, we report the following three Flow records: Src IP addr. | Dst IP addr. | Next Hop addr. | Packet | Bytes | | | Number | Number --------------------------------------------------------------- 198.168.1.12 | 10.5.12.254 | 192.168.1.1 | 5009 | 5344385 192.168.1.27 | 10.5.12.23 | 192.168.1.1 | 748 | 388934 192.168.1.56 | 10.5.12.65 | 192.168.1.1 | 5 | 6534 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 256 | Length = 64 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 198.168.1.12 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10.5.12.254 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 5009 | Claise, et. al Standard Track [Page 46] IPFIX Protocol Specifications July 2004 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 5344385 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.27 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10.5.12.23 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 748 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 388934 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.56 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10.5.12.65 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 192.168.1.1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 5 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 6534 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Note that padding was not necessary in this example. 16.4 Options Template FlowSet Example Per line card (the router being composed of two line cards), we want to report the following Field Types: - Total number of IPFIX Messages - Total number of exported Flows The format of the Options Template FlowSet is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 1 | Length = 24 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Template ID 257 | Option Scope Length = 4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Option Length = 8 | Scope 1 Field Type = 0x0003 | Claise, et. al Standard Track [Page 47] IPFIX Protocol Specifications July 2004 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Scope 1 Field Length = 2 | TOTAL_EXP_PKTS_SENT = 41 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Length = 4 | TOTAL_FLOWS_EXP = 42 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Field Length = 4 | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 16.5 Data FlowSet with Options Data Records Example In this example, we report the following two records: Line Card ID | IPFIX Message| Export Flow ------------------------------------------ Line Card 1 | 345 | 10201 Line Card 2 | 690 | 20402 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FlowSet ID = 257 | Length = 16 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 1 | 345 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 10201 | 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 690 | 20402 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 17. References 17.1 Normative References [IPFIX-ARCH] Sadasivan, G, Brownlee, N. "Architecture Model for IP Flow Information Export" draft-ietf-ipfix-arch-02.txt", October 2003 [IPFIX-INFO] Calato, P, Meyer, J, Quittek, J, "Information Model for IP Flow Information Export" draft-ietf-ipfix-info-02, November 2003 [IPFIX-AS] Zseby, T, Boschi, E, Penno, R, Brownlee, N, Claise, B, "IPFIX Applicability", draft-ietf-ipfix-as-02.txt, July 2004 [UDP] Postel, J., "User Datagram Protocol" RFC 768, August 1980 Claise, et. al Standard Track [Page 48] IPFIX Protocol Specifications July 2004 [TCP] "TRANSMISSION CONTROL PROTOCOL DARPA INTERNET PROGRAM PROTOCOL SPECIFICATION" RFC 793, September 1981 [RFC1889] Schulzrinne, H., Casner, S., Frederick, R., Jacobson, V., "RTP: A Transport Protocol for Real-Time ApplicationsÆÆ, RFC 1889, January 1996 [RFC2402] Kent, S., Atkinson, R., "IP Authentication HeaderÆÆ, RFC 2402, November 1998 [RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security Payload (ESP)ÆÆ, RFC 2406, November 1998 [RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol", RFC 2960, October 2000 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., Conrad, P. "Stream Control Transmission Protocol (SCTP) Partial Reliability Extension", RFC 3758, May 2004 17.2 Informative References [IPFIX-REQ] Quittek, J, Zseby, T, Claise, B, Zander, S, "Requirements for IP Flow Information Export" draft-ietf-ipfix-reqs- 15.txt, June 2003 [IPFIX-AS] Zseby, T, Penno, R, Brownlee, N, Claise, B, "IPFIX Applicability", draft-ietf-ipfix-as-01.txt, October 2003 [IPFIX-EVAL] Leinen, S, "Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX)", draft-leinen-ipfix-eval-contrib- 02.txt, January 2003 [NETFLOW9] Claise, B, et al "Cisco Systems NetFlow Services Export Version 9", draft-claise-netflow-9-07.txt, December 2003 [PEN] IANA Private Enterprise Numbers registry http://www.iana.org/assignments/enterprise-numbers [USEIPSEC] S. Bellovin, Guidelines for Mandating the Use of IPsec, draft-bellovin-useipsec-02.txt, October 2003, work in progress. Claise, et. al Standard Track [Page 49] IPFIX Protocol Specifications July 2004 [IKE] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)", RFC 2409, November 1998. [TLS] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC 2246, January 1999. [L2TPv3] J. Lau et al. Layer Two Tunneling Protocol (Version 3) draft-ietf-l2tpext-l2tp-base-11.txt, October 2003, work in progress. [XXX-REFERENCE] [XXX-SCTP-BLIND-SPOOFING-REFERENCE] 18. Acknowledgments We would like to thank the following persons for their valuable technical feedback: Juergen Quittek, Sebastian Zander, Dave Plonka, Jeff Meyer, Maurizio Molina, Carter Bullard, Randall Stewart, Peter Lei, Tal Givoly and many more. Authors Addresses Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 E-mail: bclaise@cisco.com Mark Fullmer OARnet 2455 North Star Rd. Columbus, Ohio 43221 Phone: +1 (614) 728-8100 Email: maf@eng.oar.net Reinaldo Penno Nortel Networks 2305 Mission College Blvd Claise, et. al Standard Track [Page 50] IPFIX Protocol Specifications July 2004 Santa Clara, CA 95054 Phone: +1 408.565.3023 Email: rpenno@nortelnetworks.com Paul Calato Riverstone Networks, Inc. 5200 Great America Parkway Santa Clara, CA 95054 USA Phone: +1 (603) 557-6913 Email: calato@riverstonenet.com Ganesh Sadasivan Cisco Systems, Inc. 170 W. Tasman Dr. San Jose, CA 95134 USA Phone: +1 (408) 527-0251 Email: gsadasiv@cisco.com Stewart Bryant Cisco Systems, Inc. 250, Longwater, Green Park, Reading, RG2 6GB, United Kingdom Phone: +44 (0)20 8824-8828 Email: stbryant@cisco.com Claise, et. al Standard Track [Page 51]