Skip to main content

Determinism: from the OSI and TCP/IP Ethernet stack to TSN

· 11 min read
PhD candidate, LyRIDS (ECE) × CEDRIC (Cnam)

A control loop on a factory floor and an e-mail server can share the same cable only if the network can promise the first one something the second never needs: that every packet arrives on time. This post starts from the ordinary Ethernet and TCP/IP stack, shows where its timing breaks down, and explains how Time-Sensitive Networking (TSN) extends it without replacing it.

OSI modelExamples, data unitTCP/IP modelDeterministic extensionL7 ApplicationHTTP, OPC UA, MQTTdataL6 Presentationencoding, TLSdataL5 Sessiondialogue controldataL4 TransportTCP, UDPsegment, datagramL3 NetworkIP, routingpacketL2 Data linkEthernet MAC, bridgingframeL1 PhysicalEthernet PHY, cablebitsApplicationTransportInternetLinkunchangedapplications seea bounded networkIETF DetNetIP, MPLSIEEE 802.1 TSNtime, shaping,redundancy, config
Figure 1. The seven OSI layers, the four TCP/IP layers they correspond to, and where the deterministic extensions act. Upper layers are unchanged; IETF DetNet extends the internet layer, and IEEE 802.1 TSN extends the link layer that Ethernet implements.Source: own drawing, after the layer descriptions and the OSI to TCP/IP comparison in the Wikipedia articles OSI model and Internet protocol suite, and RFC 1122.
Abbreviations used in this post (24)
AbbreviationStands forMeaning
CSMA/CDCarrier-Sense Multiple Access with Collision DetectionThe access method of the original shared-medium Ethernet: listen before sending, detect collisions, back off for a random time and retry. The random back-off makes the delay unpredictable.
DetNetDeterministic NetworkingThe IETF architecture and working group for bounded-latency, low-loss flows over IP and MPLS (RFC 8655).
FRERFrame Replication and Elimination for ReliabilityIEEE 802.1CB: frames are numbered, sent over disjoint paths, and duplicates are removed.
GCLGate Control ListThe repeating timetable of gate states used by the Time-Aware Shaper (and by stream gates in PSFP).
gPTPgeneralized Precision Time ProtocolThe profile of PTP defined by IEEE 802.1AS, in which every bridge takes part in time distribution.
IEEEInstitute of Electrical and Electronics EngineersThe professional association whose Standards Association publishes the IEEE 802 LAN/MAN standards.
IETFInternet Engineering Task ForceThe body that develops Internet standards, published as RFCs.
IPInternet ProtocolThe network-layer protocol of the Internet (IPv4, RFC 791; IPv6, RFC 8200).
ITInformation TechnologyComputing and networking for information processing, such as office, business and cloud systems.
LANLocal Area NetworkA network covering a limited area, such as a building or a vehicle; IEEE 802 standardises LANs.
MACMedium Access ControlThe lower sublayer of the data link layer, which frames data and controls access to the medium. A MAC address identifies a network interface.
OSIOpen Systems InterconnectionThe seven-layer reference model for communication protocols (ISO/IEC 7498-1).
OTOperational TechnologyHardware and software that monitor and control physical processes, such as controllers, drives and sensors in a factory.
PDUProtocol Data UnitThe unit of data at one layer: a frame at the data link layer, a packet at the network layer, a segment (TCP) or datagram (UDP) at the transport layer.
PHYPhysical layerLayer 1: the transmission of bits over a medium such as copper, fibre or radio.
PTPPrecision Time ProtocolIEEE 1588, a protocol that synchronises clocks over a packet network using timestamped message exchanges.
QoSQuality of ServiceThe treatment a network gives to a flow or class, in terms of bandwidth, delay, jitter and loss.
R-TAGRedundancy tagThe 802.1CB tag that carries the sequence number of a replicated frame.
RFCRequest for CommentsThe document series in which IETF standards and related documents are published.
TASTime-Aware ShaperIEEE 802.1Qbv scheduled traffic: a gate per queue, opened and closed by a repeating gate control list.
TCPTransmission Control ProtocolA reliable, connection-oriented transport protocol with congestion control (RFC 9293).
TSNTime-Sensitive NetworkingThe set of IEEE 802.1 standards that give Ethernet bounded latency, time synchronisation and high reliability.
UDPUser Datagram ProtocolA connectionless transport protocol without congestion control (RFC 768).
VLANVirtual LANA logical LAN within a bridged network, identified by a 12-bit VLAN ID in the 802.1Q tag.

1. Definition: determinism​

A network is deterministic for a flow when it can guarantee that every packet of the flow arrives within a precise, predictable and bounded time. In practice that means three properties, each stated as a bound rather than an average:

PropertyGuarantee
Bounded latencyNo packet takes longer than a known maximum from source to destination
Bounded jitterThe variation of that latency stays within a known range
No congestion lossNo packet is dropped because a queue overflowed

Fast is not the same as deterministic. A best-effort network can have a very low average delay and still, occasionally, delay a packet by orders of magnitude. For a closed control loop, one late packet can be as bad as a lost one [1]. Why deterministic guarantees? develops this contrast between average and worst case.

2. The OSI reference model: seven layers​

The OSI model, published by ISO in 1984, describes communication in seven abstraction layers, from the transmission of bits to the data of a distributed application. Each layer serves the one above it and is served by the one below [2]:

LayerRoleEthernet / Internet exampleData unit
7 ApplicationServices to applicationsHTTP, MQTT, OPC UAdata
6 PresentationRepresentation of dataencoding, encryption (TLS)data
5 SessionDialogue between applicationsopening and closing sessionsdata
4 TransportEnd-to-end delivery between processesTCP, UDPsegment, datagram
3 NetworkAddressing and routing across networksIPpacket
2 Data linkDelivery between neighbours on one networkEthernet MAC, bridging (IEEE 802.1)frame
1 PhysicalBits on the mediumEthernet PHY (IEEE 802.3), cable, fibrebits
OSI reference model with IEEE 802 covering the data link and physical layers and IEEE 802.1 at the top of the data link layer
Figure 2. Where IEEE 802 sits in the OSI model: IEEE 802 standards cover the physical and data link layers, with IEEE 802.1 at the top of the data link layer. Ethernet (IEEE 802.3) and TSN (IEEE 802.1) are both here.Source: J. Farkas, “IEEE 802.1 TSN TG Overview,” DetNet–TSN workshop, IEEE 802.1, 11 November 2018, slide 4; cropped, otherwise unchanged.

3. From OSI to the TCP/IP model​

The protocols actually used on the Internet follow a simpler model. The Internet protocol suite, or TCP/IP model, defined in RFC 1122 and RFC 1123, has four layers named by their scope rather than numbered by function [3][4]:

TCP/IP layerScopeCorresponding OSI layers
ApplicationProcess-to-process data for applications7, 6 and most of 5
TransportHost to host: TCP (reliable) or UDP (unreliable)4 (and part of 5)
InternetBetween independent networks: IPpart of 3
LinkWithin one network segment: for example Ethernet2, with functions of 1

The OSI model is still the common vocabulary, but the TCP/IP suite became the standard for networking because its protocols were simpler and could be implemented independently [2]. Both models rest on encapsulation: each layer wraps the data of the layer above in its own header [3].

Application data wrapped in a UDP header, then an IP header, then an Ethernet frame header and footer as it descends the stack
Figure 3. Encapsulation down the TCP/IP stack: application data gets a UDP header, then an IP header, and finally an Ethernet frame header and trailer before it reaches the wire.Source: UDP encapsulation.svg by Cburnett, colourised by Kbrose, from the Wikipedia article “Internet protocol suite”, Wikimedia Commons, CC BY-SA 3.0.
Two hosts connected through two routers: application and transport layers run end to end between the hosts, the internet layer runs in every router, and the link layer is per link
Figure 4. Two hosts connected through two routers. Application and transport layers run end to end, between the hosts only; the internet layer runs in every router; the link layer, such as Ethernet, runs separately on each link.Source: IP stack connections.svg by Kbrose, from the Wikipedia article “Internet protocol suite”, Wikimedia Commons, CC BY-SA 3.0.

Figure 4 shows the key point for determinism: the link layer is where frames wait. Every switch or router on the path queues frames before sending them on the next link. Whatever happens in those queues decides the delay.

Ethernet Type II frame: 14-byte MAC header with destination MAC, source MAC and EtherType, a 46 to 1500 byte payload, and a 4-byte CRC
Figure 5. An Ethernet Type II frame: a 14-byte MAC header (destination address, source address, EtherType), 46 to 1500 bytes of payload, and a 4-byte check sequence.Source: Ethernet Type II Frame format.svg by Wikimedia Commons contributors, from the Wikipedia article “Ethernet frame”, Wikimedia Commons, Public domain.

Ethernet, standardised by the IEEE 802 project from 1980 onward, covers the physical layer and the lower part of the data link layer (IEEE 802.3), while bridging and VLANs are specified by IEEE 802.1 [5][6]. Its history explains why it was not deterministic to begin with:

  • Shared medium. The original Ethernet put every station on one coaxial cable. Access was governed by CSMA/CD: a station listens, transmits, and on a collision backs off for a random time and tries again. The random back-off makes the delay of a frame unpredictable, and heavy load makes collisions more frequent [5].
  • Switched, full-duplex Ethernet. Bridges and switches split the network into separate links, and full-duplex links remove collisions altogether [5]. This is the Ethernet used today. But a switch still stores frames in queues at each output port. When frames from several inputs head for the same output, they wait, and the waiting time depends on all the other traffic.
  • Priorities are not enough. IEEE 802.1Q gives eight priority levels, but a frame already being transmitted cannot be interrupted, and nothing ties transmission to a common time. A critical frame can still wait behind a long frame or a burst of other traffic [7].

The layers above add their own variability. TCP retransmits lost segments and slows down on congestion, which is right for file transfer but useless for a control message that is only valid for a few milliseconds. UDP adds no retransmission, but no guarantee either. The stack as a whole is best effort: it delivers, usually quickly, but promises no bound.

5. Why industry needs a deterministic Ethernet​

Industrial systems carry two kinds of traffic with opposite needs [1]:

OT (operational technology)IT (information technology)
ExamplesMachine control, sensor valuesE-mail, firmware updates
What mattersFixed delay, low latency, low jitterThroughput
If lateThe data is useless, and the machine may failNothing serious; best effort is enough
BandwidthUsually smallUsually larger

The traditional answer was two networks: one for OT, often a fieldbus or a proprietary variant of Ethernet, and one for IT. That doubles cabling and equipment, and the proprietary OT networks do not interoperate with each other or follow the speed increases of standard Ethernet [1][8].

6. TSN: an extension of standard Ethernet​

Time-Sensitive Networking sits on top of standard Ethernet. It is a set of IEEE 802.1 standards that lets one Ethernet network carry IT and OT data on a shared connection, while each kind of traffic still gets what it needs [1][10]. In Cisco's description, the IEEE TSN task group develops specifications that provide deterministic services over layer 1 and layer 2. These are implemented in Ethernet switches such as the Cisco Industrial Ethernet 4000 Series, and are being extended towards wireless access such as 5G [9].

TSN does not replace any layer. As Figure 1 shows, applications, TCP/IP and the Ethernet frame format stay the same; TSN adds functions to the bridges and the MAC. Each standard covers one function, and system designers combine the ones they need [1]. The three described in [1] cover the three gaps of Section 4:

A common time: 802.1AS. TSN timing comes from the Precision Time Protocol (PTP, IEEE 1588), which distributes time from a grandmaster clock, such as a GPS-disciplined source, through a tree of master and slave clocks. The generalized PTP of 802.1AS is a stricter profile: it requires a peer-to-peer delay measurement on every link, expects all devices to run at the same frequency (syntonised), and in newer revisions brings back one-step timestamping [1]. Devices are synchronised to within nanoseconds [1]. See the 802.1AS post.

Scheduled transmission: 802.1Qbv. A time-aware shaper at each egress port gives each of the eight queues a gate. A gate control list opens and closes the gates on a repeating schedule, so time slots can be reserved for OT traffic and others left to IT traffic [1].

Two cycles of an 802.1Qbv schedule, each with a short time slice for VLAN priority 3 followed by a longer slice for all other priorities
Figure 6. An 802.1Qbv schedule: in every cycle, one time slice is reserved for the time-critical priority and the rest is left to all other traffic.Source: Figure 1- Example IEEE 802.1Qbv schedule.svg by Ulgorash, from the Wikipedia article “Time-Sensitive Networking”, Wikimedia Commons, CC BY-SA 4.0.

Before each protected slot, the hardware keeps a guard band: it does not start a frame that could not finish before the slot begins, and holds it for the next slot of its class instead [1]. Traffic is mapped to queues by priority, protocol, destination port or traffic type [1]. See the TAS post.

Redundancy without retransmission: 802.1CB. Frames of a stream are replicated and sent over two paths, typically both ways round a ring. A redundancy tag carries a sequence number, and the receiver keeps the first copy of each number and discards the duplicates. If a link breaks, no frame is lost and no retransmission delay occurs, and the hardware does the work instead of application software [1]. See the FRER post.

The full set of tools, including credit-based shaping, preemption, cyclic queuing, policing and configuration, is described in the TSN toolbox.

7. Why it prepares a system for the future​

  • One converged network. IT and OT share cables, switches and management, instead of running two networks side by side [1].
  • Standard Ethernet underneath. TSN rides on IEEE 802.3, so it benefits from each new Ethernet speed and from multi-vendor, standard hardware rather than a proprietary fieldbus [1][8].
  • Guarantees that can be proven. Bounded latency and no congestion loss are properties of the configuration, which can be checked before a stream is admitted. That is the role of network calculus in this series.
  • Beyond one LAN. Where a deterministic path must cross routers, IETF DetNet provides the same service at the internet layer (Figure 1 and the DetNet post).

8. Summary​

Layer (TCP/IP)Ordinary behaviourDeterministic extension
Application, transportTCP retransmits and adapts; UDP gives no guaranteeUnchanged: they receive a bounded service
InternetIP routing, best effortIETF DetNet
Link (Ethernet)Switched queues, eight priorities, no common timeIEEE 802.1 TSN: 802.1AS time, 802.1Qbv schedules, 802.1CB redundancy, and more
PhysicalIEEE 802.3 PHYUnchanged, plus frame preemption (802.3br)

References​

  1. J. Steinheider (NXP Semiconductors), "The Fundamentals of Time-Sensitive Networking," All About Circuits, industry article, 19 January 2021. Article
  2. Wikipedia, "OSI model," sections on the layer architecture and the comparison with the TCP/IP model. Article
  3. Wikipedia, "Internet protocol suite," sections on key architectural principles and layers. Article
  4. R. Braden (Ed.), RFC 1122, "Requirements for Internet Hosts – Communication Layers," IETF, October 1989. RFC Editor
  5. Wikipedia, "Ethernet," sections on standardisation, shared medium, and bridging and switching. Article
  6. IEEE, IEEE 802.3, Standard for Ethernet. IEEE SA
  7. J. Farkas, "TSN Basic Concepts," DetNet–TSN workshop, 11 November 2018. Slides
  8. J. Woods, "IEC/IEEE 60802: The Case for the Converged Network in the Factory of the Future," IEEE SA TSN webinar series, April 2023, slide 8. Slides
  9. Cisco Systems, "Wi-Fi 6 and Private LTE/5G Technology and Business Models in Industrial IoT," white paper, 2019, section on IEEE standards. PDF
  10. IEEE 802.1, Time-Sensitive Networking (TSN) Task Group. Web page