Highly Automated Operations within Massively Scalable Data Centers. The Border Gateway Protocol Case

Some operators operate DCs (Data Centers) housing several hundred thousand servers, known as MSDCs (Massively Scalable Data Centers), HSDCs (Hyper-Scale DCs) or Large Scale DCs. This type of DC implies particular management and operating constraints. Operators are therefore reviewing DC network architecture to take better account of the specific features of MSDCs, while guaranteeing simplicity of infrastructure management.

Management simplicity is characterized by the industrialization and automation of the operations required to run an MSDC.

Indeed, DC deployments require a high level of automation in the production of DC resources (e.g. dynamic resource production processes such as virtual machines designed to host certain virtualized network functions). The operation of this type of infrastructure also needs to be simplified, given the complexity of the configuration processes involved in setting up connectivity services based on the use of virtual resources that can be massively distributed within a DC or between DCs, while guaranteeing network availability and stability at all times.

Various routing policies can be implemented within a DC. These policies can be based on the IS-IS (Intermediate System to Intermediate System) protocol, or on L2-level techniques. Border Gateway Protocol (BGP) is also used in the latest DC infrastructures. From this point of view, BGP is used in the same way as an IGP (Interior Gateway Protocol), such as IS-IS, to calculate, select and establish routes for internal DC traffic.

This article discusses the implementation of BGP routing policies in large-scale data centers (MSDCs). In particular, the article describes common practices for deploying BGP in MSDCs, as well as the mechanisms needed to automate MSDC operations.

Thanks to the mechanisms described in this article, the following operations are completely distributed and automated:

• automatic distribution and allocation of AS (Autonomous System) numbers, while optimizing the volume of signaling traffic typical of configuration procedures. The involvement of a DC controller is not systematically required for each of the nodes that make up the DC;

• automatic establishment of BGP sessions ;

• standardization and automatic dissemination of the parameters governing the BGP process, in compliance with the constraints and recommendations of the DC infrastructure operator;

• automatic recognition of new switching resources introduced into the DC infrastructure as part of its evolution (e.g. automatic DC extension, without controller intervention);

• ...