As a result of the Covid-19 pandemic, this year’s IFIP 2020 which was supposed to be in Paris became a completely virtual event and our Product Manager, Nicolas Keukeleire presented “Current and Planned Commercial Deployments of Multipath Transport Protocols”.

Here is a summary of the talk alongside some of the key slides that he shared.

What is MPTCP?

MPTCP was developed as part of the Trilogy project financed by the EU ¹ . It has been standardised with RFC 6824 in 2013 and more recently RFC 8684 in 2019. The implementation of MPTCP in the Linux kernel received the Sigcomm Networking Systems Award in 2019. MPTCP is an evolution of TCP allowing one TCP session to be conveyed on several paths (several TCP subflows) between a client and a server. 

Figure 1 shows the location of Multipath TCP in the transport layer between the existing sockets and regular TCP connections. 

Figure 1. The location of MPTCP within the TCP/IP model

How does MPTCP work?

>MPTCP connections are established in the same way as TCP connections except for a couple of things:

1. An MP_Capable option is used to indicate that this is going to be an MPTCP connection and
2. We have another MPTCP specific option, MP_Join which we can use to create secondary paths.

Figure 2. The familiar TCP three way handshake but with the addition of the MP_Capable and MP_Join options.

MPTCP uses two mechanisms to manage physical connections:

1. A Path Manager which decides when secondary connections should be opened or closed and
2. A Scheduler which decides which path each packet should take (yes, this is done on a packet by packet basis)

How is MPTCP used?

One early, very famous use of MPTCP, is Apple’s digital assistant SIRI which has used MPTCP since 2013 (iOS 7) to enable Siri to use either Wi-Fi or Cellular whichever is available and providing the lowest latency and packet loss. Apple implemented MPTCP in Maps and Music in 2019.

Figure 3. Apple uses MPTCP to reduce latency and improve handovers between Wi-Fi and Cellular networks

Samsung is another player in the MPTCP ecosystem implementing it inside phones used in South Korea by KT, SK Telecom and LG+ since 2015.  The goal here was to aggregate Wi-Fi and Cellular to use them at the same time to reach gigabit speeds.

Figure 4. Samsung uses MPTCP to aggregate  Wi-Fi and Cellular networks to reach gigabit speeds

Since 2016, Tessares and Swisscom have been enabling MPTCP on customer premises equipment (CPE), either DSL home gateways or LTE modems. Here the goal is to aggregate speeds from fixed  and cellular networks but in such a way as to prioritise the fixed network. A Hybrid Access Gateway or HAG is located in the operator network to recombine the outgoing traffic and to direct incoming traffic from the Internet across the two networks.

Figure 5. Tessares and Swisscom have embedded MPTCP capabilities in CPE to enable fixed and cellular hybrid networks since 2016.

The role of the Hybrid Access Gateway

 The HAG is a transparent proxy, placed on path from the Internet to the CPE’s. Using the HAG transparently means that we do not have to make any modifications to apps, access devices or content servers. To the Internet, the requests for a user with an ISP assigned, public IP address look as though they have come from the CPE, not the HAG.

Fig. 6 shows what this looks like for the three way handshake. C@ represents the IP address of the access device (phone, laptop, whatever), CPE@ represents the public IP address assigned to the CPE.  S@ represents the IP address of the web / content server. Notice that the HAG itself is not directly addressable.

Figure 6. The three way handshake showing how the transparent HAG is on path and not directly addressable 

Making best use of network resources

We mentioned that the Tessares implementation of MPTCP prioritises DSL.  This is because cellular networks are an expensive resource shared with higher priority, mobile users.  The cellular network is used but at a lower priority. The goal is to fill the fixed network pipe first (as much as possible) before using the cellular network. The Path Manager decides when to open the cellular connection and the Scheduler decides which of the available networks to use.

Figure 7. The Path Manager and the Scheduler are used to maximize the use of the fixed network, only overflowing to cellular when necessary and when resources are available.

Commercial deployments to date

So far there have been seven commercial deployments including the one by Swisscom. Tessares have deployed at incumbents in six countries: Belgium, The Netherlands, Lithuania, Finland, Malta and Croatia.

Figure 8. Map of MPTCP in Europe

Looking at one Hybrid Access Gateways at one of the deployments, we can see how much of the inbound traffic has been carried over the fixed network and how much over the cellular network. In this weekday example, peak downlink throughput came out as follows:

DSL: ~5.5 Gbps

LTE: ~4.5 Gbps

Total: ~10 Gbps

Figure 9. Network throughput on a single HAG showing distribution of downlink over the two networks. Source: Source: N. Keukeleire, B. Hesmans and O. Bonaventure, “Increasing Broadband Reach with Hybrid Access Networks,” in IEEE Communications Standards Magazine, vol. 4, no. 1, pp. 43-49, March 2020.

Looking at another deployment, this time what we would call a high speed use case where MPTCP has been used to ‘top-up’ DSL 3,000  lines of < 50 Mbps  to the advertised speed of 75 Mbps.

This time the downlink traffic has been distributed as follows:

DSL: ~8 Gbps

LTE: ~2.5 Gbps

Total: ~10.5 Gbps

Figure 10.  Distribution of traffic over the course of a week showing proportion of LTE and DSL downlink in a hybrid access network

The impact of the Covid-19 lockdown

Interestingly, when we were researching this talk, we noticed an interesting change in traffic patterns before and after the Covid-19 pandemic lockdown in one European country. Whereas before the lockdown, there was a definite weekday / weekend difference in the daily traffic pattern, after the lockdown begins this differentiation disappears as the weekday traffic sees a 25% average increase.

Figure 11. Difference in weekly traffic pattern both before (Top) and after (bottom) the Covid-19 pandemic lockdown.

Role of Multipath Transport in 5G

Multipath has an important role to play in 5G networks. Since Release 16, the 3GPP introduced a new feature ATSSS or Access Traffic Steering, Switching and Splitting. This will be the mechanism, at UPF level (User Plane Function see fig. 13),  by which 5G and Wi-Fi will be combined in the near future:

 Figure 12. 3GPP Rel 16 describes how MPTCP will be used to combine 5G with unlicensed spectrum like Wi-Fi

Steering is about choosing the best network depending on the user’s location and network conditions. Switching enables seamless handovers from cellular to Wi-Fi (and back). Splitting refers to splitting the traffic over multiple paths for higher speeds.

ATSSS has two components, one in the User Equipment (UE) and one in the 5G Core. The UE will natively use MPTCP while the User Plane Function will have a TCP-> MPTCP proxy similar to the Hybrid Access Gateway discussed earlier.

Figure 13. The high level architecture of the ATSSS

The 3GPP has defined the use of the 0-RTT TCP Convert Protocol² for the MPTCP proxy, which addresses the proxy directly, the proxy then carries out Network Address Translation. Two of the authors contributing to the design of this protocol are from Tessares (Olivier Bonaventure & Benjamin Hesmans).

Figure 14. ATSSS makes use of the 0-RTT TCP Convert protocol for the MPTCP proxy

Release 16 is limited to TCP traffic (most of the traffic on the Internet including a lot of OTT video) but with Release 17, the 3GPP is looking to support non-TCP traffic such as UDP / QUIC.

QUIC is a relatively new, reliable and secure transport protocol based on UDP. Its first use case was to replace TCP+TLS in the HTTP/3 stack but there are multiple extensions in development.

Figure 15. TCP and QUIC in the network model 

• Datagram Extension – allows unreliable QUIC connections
• Tunnel Extension – Leverages the datagram extension to tunnel packets inside a QUIC connection
• Multipath Extension – Multipath QUIC enabling one QUIC session to be conveyed over multiple network paths  

Conclusion

The main multipath milestones we can look forward to are:

• The upstreaming of MPTCP into the mainstream Linux kernel
• The standardization of MPQUIC at the IETF, which will start by the end of this year
• The possible integration of MPQUIC into 3GPP Release 17

There is a clear momentum around multipath protocols, driven by the multiple applications it offers.

For more information, please contact us.

1. https://cordis.europa.eu/project/id/216372
2. https://datatracker.ietf.org/doc/draft-ietf-tcpm-converters/
3. https://datatracker.ietf.org/doc/draft-pauly-quic-datagram/
4. https://datatracker.ietf.org/doc/draft-piraux-quic-tunnel/
5. https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/
6. https://github.com/multipath-tcp/mptcp_net-next/wiki