是网络拥塞控制方面的一个review
作者最后总结重点强调了几点趋势：

• Recent works on this topic have seemingly converged on the use of either offline or online learning algorithms to replace heuristics designed for specific types of network environments, such as datacenter or wireless networks
• It is conceivable that deep reinforcement learning (DRL) algorithms can be used to perform online training without relying heavily on the use of heuristic algorithms.Indeed, DRL algorithms have already been proposed for automated traffic optimization in datacenter networks [15]
• One of the recent machine learning techniques, transfer learning, may become quite promising to bridge such a gap.

---

Abstract

• TCP and its variance may not perform well because of its high latency(relatively)
• present a survey of some of the most prominent research results on this topic in the recent years, with an editorial commentary on potential research directions in the future

Introduction

• BDP（bandwidth-delay product (the product of
  the bottleneck bandwidth and the round-trip propagation time) ） varies because bandwidth and propagation vary by a few orders of magnitude.
  but CC still have to be able to operate effectively over the entire range of the BDP
• two ways of solving the problem of TCP
  • redesign CC protocols maintaining TCP-friendliness without backward compatiblity like eXplict Control Protocal( XCP )
  • “patch” the protocol while maintaining backward compatibility with respect to fairness to the traditional TCP variants such as TCP-Reno.

Redesign CC protocals in DCN(the first method)

minimize queuing delays

• the focus of most of the new papers:
  • fast convergece to the steady state
  • reduce queuing delays as much as possible (propagation delay in DCN is really low for now) to reduce end-to-end delay (Ideally ,queuing delays should be close to zero )
  • traditional TCP can’t balance them two.
• HULL ( high-bandwidth ultra-low latency ) via Alizadeh et al[3]
  • avoid queuing formation by reduceing sending rates in advance at the eexpense of small loss in urilizing boottleneck bandwidth.
    • traditional active queue management approach is ECN
    • we can set a lower threshold of congestion signals to lower queuing delays
  • combine a phantom queue ( bandwidth headroom ) and ECN (instead of using the index of queue oocupacy ratio to set the ECN mark )
  • essence of methods of this kind is to set ECN marks based on link utilizations before queues are formed
  • use method of packet pacing implemented by a simple leaky bucket regulator to smooth out highly bursty traffic
    • recommended that packet pacer is implemented in hardware within NIC (high scheduling granularity at the sub-micronsecond level )
  • It is clear that HULL is designed specifically for datacenter networks, taking advantage of the feasibility of implementing hardware packet pacers in NICs,as well as ECN marking in intermediate switches.
• DX via Lee et al[4]
  • record round-trip time at the receiver end to find even one packet queue in the network
  • compare the average queue latency of the current to avoid spurious congestion signals
    • also considers head room
• Fastpass

Accelerating Convergence

• congestion control protocols we discussed so far, such as HULL, are reactive, may take a long time to calculate the fair-share rate to flows at the steady state, leading to a long tail with respect to the flow completion time. To meet the demand of ever increasing bandwidth in datacenter networks, it is imperative for congestion control protocols to converge as rapidly as possible.
• the need to accelerating convergence --> use fewer iterations to converge -.we should know more about the network status
• PERC [6]
  • employs a message passing algorithm to interact and share information between flows and links, instead of using a centralized rate allocator
  • however,requires that intermediate switches should be programmable, in order to calculate the fair shares of flows explicitly
• ExpressPass [7]
• Flowtune[8]
• PERC via Jose et al
  • a distributed algorithm instead of using a centralized rate allocator
  • employs a message passing algorithm to interact and share information between flows and links
  • a flow in PERC carries its demand in the packet headers so that it can be conveyed to each link along its path
  • With the knowledge about all active flows in the network, the optimal fair-share rate allocation can then be computed directly, which allows PERC to converge in fewer iterations.
  • requires that intermediate switches should be programmable, in order to calculate the fair shares of flows explicitly.(It is a distributed system after all.)

redisign CC protocals for the Internet

• much more challenging than CC protocal for DCN
• ‘luxury’ to resort to a centralized design
• ultimate design objectives is the same as forementioned
• BBR (Bottleneck Bandwidth and Round-trip propagation time) via Cardwell (Communications of the ACM 2017)
  • simple ,heuristic ,proposed by Google
  • has already been incorporated in to recent Linux Kernel
• Remy [ offline ]
• PCC
• PCC Vivace
  • more TCP-friendly, converges faster, and reacts more swiftly to changes in network conditions.
• Pantheon and Indigo [ offline ]
  • Indigo employs offline training using Pantheon’s abundant supply of network emulators.
  • observes the network states and adjusts the congestion window instead of changing teh sending rates(e.g. BBR,Remy,Pcc or Vivace )[所以后者都是属于pacing嘛]
    • cwnd observes the network states and adjusts the congestion window
  • network state characteristics
    • a moving average of the queuing delay (which, in a similar vein as BBR, measured as the difference between the current and the minimum RTT);
    • a moving average of the sending rate;
    • a moving average of the delivered rate (again similar to BBR)
    • the current congestion window size
  • use a LSTM RNN to store mapping from states to actions and train such a RNN using an offline training phase.
    • Once trained and deployed, the mapping will be fixed.

Conclusion

• offline ones may come with problems that the training model differs greatly form the online one
• online ones require elaborate heuristics to strike a balance
• DRL can be used to online training like AuTO
• offline training requires great training time ,which much exceeds the complete time of dataflows
  • transfer learning may solve the problem by using a pre-trained model(using data in other conditions )

collect

• resurgence
• editorial
• product
  often refer to the multiply result
• subpar(aka. sub par)：lower than average
  par : standard,average
• de facto adj.in fact
• plateu n. plain land
• in a similar vein as 和…风格类似

to be continued:

• CUBIC
  a TCP variant that is designed for high-BDP environments by growing the congestion window more aggressively beyond the saturation point. Since it was proposed,CUBIC has become a remarkable success, making its way into the Linux kernel since version 2.6.19 as the default TCP implementation, as well as the Windows 10 kernel.
• backward compatiblity特性
• XCP and Fast TCP (in introduction )
• why does it say Reno maintains backward compatiblity with respect to fairness
• plateau
• what does the tilt font mean in papers
• what’s fair-share rate mentioned many times (like in PERC)
• optimal operating point 最优操作点
• Pantheon:train ground for Internet congetsion-control research 【了解一下这个平台】