SDN – Dhiman Deb Chowdhury

Facts behind the Myth of Whitebox Open Networking: A Reality Check

I begin with an assumption that you are somewhat familiar with “whitebox” term as it relates to networking gears and cognizant of its trend. For those needing a brush up, please read my earlier article https://www.linkedin.com/pulse/bare-metalwhite-box-solutions-open-hyperscale-data-center-chowdhury/ .

The term “whitebox” and “open networking” to some extent synonymous, both innately suggest networking gears that are “open” meaning adheres to “open” standards and ecosystems. The former is a byproduct of open networking concept and refers to the disaggregation model in which hardware and software are separated. This decoupling of hardware and software created opportunities for a software ecosystem to flourish: well, almost!

As with any technologies advents, early days are little bumpy. It is no different for whitebox and open networking ecosystems. There are few constraints but it has come long way since its introduction in 2011. Yet, some are cautious and doubtful of its reliability: perhaps confused by choices, constraints and complexities.

In this article, I will simplify and explain fact behind the myth of “whitebox”. This article is educational in nature and intended to help readership make right decision choosing whitebox in their network transformation.

Making the right Choice

Let’s be candid! Choices are always good if we are prudent. During my presentation at Amsterdam OCP summit, someone asked me a question referring to the often confusing software ecosystems and choices thereof; as to what method one can take to make right choice for hardware and software?

Responding to this question outright in few sentences may be difficult but the method of whitebox selection could be simple: first, determine your network transformation needs and develop a checklist. It should include mandatory, good to have and nice to have feature lists. Secondly, match this checklist against hardware and software features of given whitebox solutions.

Figure 1. Whitebox product choices.

In figure 1, I depicted a diagrammatical representation of various choices available in a given hardware: herein, Agema products from Delta (http://www.agemasystems.com ). It shows four different choices for customers: bare metal (1), whitebox (2), bare metal with open source ONL/ONLP (3) and bare metal with OFDPA (4). If you do not have software resources and not planning to have experimental undertaking, please stick with whitebox. The “baremetal” switch is generally good for NOS (Network Operating Systems) vendors those who provide protocol stack software solutions e.g. IPV4, BGP, MPLS etc. If you have source code from particular NOS vendor and like to qualify this on a hardware platform go with bare metal. Those who are more incline towards experimental undertaking may go for Opensource route: 3 & 4 (refer to figure 1). The bare metal switch with support for ONL (Open Network Linux) is ideal for those who likes to work with ONL ecosystems. Please visit https://opennetlinux.org/ for more details. The OFDPA (Open Flow Data Plane Abstraction) option is good for people who prefers decoupling of data plane and control plane. This software is a contribution from Broadcom® and may not be fully supported in some of their silicons. Please visit https://github.com/Broadcom-Switch/of-dpa for further details and consult with hardware vendors for specific platform qualification in OFDPA.

Choosing the Right Hardware (Networking Gears)

One thing you can be sure of that hardware are stable and reliable if you are buying from the vendor known for quality products. The same goes for software as well. More importantly, networking hardware that are available in whitebox disaggregation model now offer some of the “state of the art” silicons. To some extents, these hardware are more advanced than those available from typical networking vendors. For example, you may now able to purchase 100GbE platform that includes programmable chips. Not that it is important for your consideration when comes to traditional network transformation but those interested in experimental use of HW level APIs may find it tempting. Additionally, whitebox networking gears also offer performance and scalability choices.

Let consider the diagram below: the figure depicts choices of whitebox hardware from 1GbE to 30Tbps.

Figure 2. Whitebox Networking hardware choices.

Products are now available in standalone and chassis format. More importantly, some of these networking gears are versatile depending upon NOS availability. For example, you could use profile based deployments of a “Top-of-Rack” switch in many network scenarios including data centers and Service provider networks with ability to perform traffic engineering if so is desired. However, there is a caveat to it, not all hardware capabilities are utilized by NOS vendors. Hence, be prudent with your selection.

Networking Software Choices

The term “NOS” (Network Operating Systems) refers to networking software solutions that offers many networking protocols and application. Some NOS are more generic type having more emphasis on traditional networking protocols while others are focused on flow based or API based software solutions. Being prudent of choosing a NOS is thus important: if you are confused, ask respective hardware vendor as to which NOS best suits their platform. For example, if you are looking for a platform that should support let’s say 1 Million IPV4 route, ensure that the hardware are consideration comes with TCAMs and NOS vendor are capable of supporting the routes without performance compromise. Similarly, if you are having microburst scenarios in your data center and are looking for ways to minimize application performance issues, you may need deep buffer switch. The same goes for service providers. Therefore, understanding your networking needs andt jotting them down may help you make the right selection. Always, keep your mandatory checklist ready and use it as reference to match against hardware and software capabilities.

If you are not considering to decouple the forwarding (e.g. packet forwarding) and control plane (e.g. flow controls and/or flow based forwarding decisions), stick with traditional protocol suits. This is not to suggest that you do not need APIs/protocols for programmability and easier provisioning. You definitely do, but ensure what benefit it derives. I suggest that you always look for ZTP (Zero Touch Provisioning) capabilities in software irrespective of which NOS you choose. The ZTP will help you provision many switches at once reducing time and overhead cost associated with manpower.

Figure 3. Network Operating System (NOS).

In the figure above shows typical NOS availability in a given whtiebox platform but there are more vendors available than those depicted above. Some are startups while others are well-known networking giant. Best way to find out is to contact respective hardware vendor.

Building the Network

Let’s be real! Transforming networks begins with few essentials: CAPEX/OPEX, Bandwidth consideration and business problem you are trying to solve. First, clear your mind of buzzwords, go to basics. Let’s consider you are building a data center network infrastructure or augmenting thereof. The important question is what fundamental architecture you are following and protocol suits you are looking for. Let’s say, you are building the data center network based on BGP and wanted to keep each cluster separated. I am depicting a typical setup considering the above deployment scenarios in figure 4 below. The physical network setup is based on spine/leaf architecture (known as “clos” named after Charles Clos). You may able to redesign such architecture to a five-stage folded clos to accommodate future demand.

The diagram shows Compute, Storage and Machine Learning Clusters are separated. It so done to help you easily visualize the product needs in each area. For example, you may be planning to use separate switch for WAN and DCI leaving DC setup isolated from abnormalities of WAN or DCI routers.

Figure 4. Data center Spine/Leaf Architecture.

Similarly, you may want to keep your machine learning cluster in separate set of deep buffer switches minimizing application performance degradation due to micro burst. Whatever the network transformation decision you make, please ensure that you are choosing right networking gears considering specific deployment scenarios. As such, you cannot go wrong with your selection of whitebox solutions.

Same goes other network scenarios, whether service provider and/or enterprise.

The possibilities

I have thus far kept this article focused on traditional network transformation without referring to the possibilities of programmability, service orientation and artificial intelligence (AI) use cases in network transformation. While these discrete technological advents are having different triggers and origins, whitebox are making it easier to realize such technological bonanza. This can be a forward looking statement but such assertion is not without basis. For example, Open Networking paved the way for whitebox to be realized and at the same time it created the possibility of network programmability. Since whitebox products support open standards/ecosystems and are affordable, many researchers and startups are working to bring the programmability, service orientation and AI to networks. Considering such developments in the marketplace, I have created a timeline and presented herein below. The diagram depicts how each technological advents are connected and potential timeline for each advent to be realized.

Figure 5. Whitebox to cognitive networks timeline.

For example, Open Networking model created whitebox and helped developed software ecosystems. Such works made it easier to realize programmability and decoupling of forwarding/control plane (SDN) in whitebox platforms. Overtime, the ensuing software ecosystems and works thereof paved the way for further development works towards intent-based Network. This service oriented networking concept is still premature and the works are piecemeal at best, nonetheless, it is a near term possibility.

Similarly, Self-organized networks and Cognitive networks are some of the other new possibilities that may come to the fore by 2020. One potential is that Whitebox software ecosystem may soon offer deep learning agent at switches and AI engine at controller making the network more state aware. Hence, possibilities are endless with open systems though limitation may plague some deployment if end user is not careful. The bottom-line, if an end-user is prudent in their selection of whitebox, it will offer great potential in their network transformation.

Network Transformation is in the offing: Does disaggregation model help?

One of the fascinating aspects in my job is understanding how some of world’s largest networks will deploy Open networking platforms. It is this continuous learning that keeps me passionate about what I do. When I first use “open networking” term, people wondered what it may entails. Some thought of it is all about using “open source” software and the other thought of it as another connotation of confusing technical term that simply add to plethora of jargons. Interestingly, open networking is none of the said. It is about disaggregating hardware and software in a way that allows relatively easier integration of each sub systems. In doing so, the model innately provides three distinct benefits to customers:

Eliminates Vendor lock-in: There is no need to be enchained by vendor or vendors. With state of the art merchant silicon available off the shelf, benefits of technology can be realized without being locked into a specific vendor device.
Cost Reduction: As more and more vendors join the marketplace, price of hardware and software becomes suddenly more affordable. This innately reduces CAPEX and OPEX for customers.
Choice of HW and Software: The disaggregation model in which open networking conception is born innately provides a choice for customers on hardware and software.

Figure 1‑1. Open Networking: Disaggregation model.

Central to open networking is the disaggregation model that allows hardware and software vendors to deliver their capability on what they do best. For example, hardware vendor can bring best breed of technology from modularization of their platform to make use of state of the art market silicons and on the other hand, software vendors to offer best NOS (Network Operating Systems) or software means for data plane and control plane separation. In doing so, the marketplace has created an apparently robust eco-chain. Some will disagree here but the fact remains that the eco-chain is improving and some parts of it is more realistic now then perhaps a year before. To be more specific, I have elaborated the eco-chain in the figure below. Rather than referring it to open networking eco-chain, I denoted it as “SDN eco-chain” since the central to this initiative is about making network more programmable and service aware.

Figure 1‑2. Typical SDN Eco-chain.

The programmability is the core element of SDN (Software Defined Networking) for which pundits advocated the decoupling of control and data plane. However, programmability can be done in many ways: through ZTP (Zero touch provisioning), APIs (e.g. Netconf) and OpenFlow protocol (a more conventional notion of decoupling).

To my experience, majority of network deployments thus far refrained from purely decoupling model but there are exceptions and distinct differences in deployment. For example, FBOSS that runs as agent daemon uses thrift API instead of openflow to communicate route information with controller. Details of this opensource decoupling software is available at https://github.com/facebook/fboss .

Service Providers are more interested about CAPEX and OPEX reduction with choice and flexibility that open networking offers. For example, a provider edge switch that offers flexibility of advance MPLS networks with APIs for model based network configuration and monitoring could best fit their immediate need of transforming networks as oppose to overhauling the network in one go using decoupling of data and control plane. It is in this phased transformation, open networking offers best value but it is not confined by the said as it is encompassing to traditional networks and decoupling of data and control plane.

I conjectured even a bigger picture for “Open networking” as it is the fundamental conduit of future network transformation. For it, disaggregation model is paving the way for a more sophisticated network transformation to occur. A vision towards that end would be “Intelligent Networks” one that is provisionable in click, intent-based and self-healed, to name few attributions.

Figure 1‑3. Networking Technology Trend.

Affordability, openness and choices are big factors in network transformation especially for service providers since disaggregation model is giving them ability to deliver more service at a fraction of cost that would otherwise be. Moreover, open networking is offering best of both world, the “traditional” and “NextGen”. For example, in a given network deployment scenario customer may prefer to choose their own NOS depending upon southbound API requirements and L2/L3 switching products. A NOS that combines both APIs and decoupling protocols such OpenFlow is known hybrid NOS. The goal here would be two folds, first allow traditional network protocols such BGP to be run at NOS level forming traditional routing architecture and secondly, network topology is passed to Openstack for visibility, orchestration and management. The following picture depicts how such nextgen network abstraction can coexist with traditional networks.

Figure 1‑4. Cloudification of Network.

Telemetry at each network device can be obtained through Broadview™, an opensource software contributed by Broadcom®. Such deployment not only allows physical networks to be virtualized and cloudified, it is done so without compromising backward compatibility of physical network and the traditional network routing architecture therein. The depiction herein is forward looking and implementation of such network is at the discretion of its designer and availability of suitable hybrid NOS. However, this notion of network cloudification is possible only due to advances in open networking and the eco-chain that supports it.

This article is provided herein for educational purpose only implying the importance of “Open Networking” and how disaggregation model helps in realizing NextGen network transformation.

SDN API and Protocols: ForCES (Forwarding and Control Element Separation)

This article is part of my “Network Virtualization 101” series. For other articles in this series, please visit http://www.dhimanchowdhury.com or my blog athttp://www.dhimanchowdhury.blogspot.com .

SDN (Software Defined Networking) is much of a buzz word now a days; for some it is simply decoupling of control and data plane but for others, it is an encompassing abstraction for cloudification. If you scratching your head, you are not alone. Well, simplistic purview may be decoupling but the sole purpose is network programmability. So as it goes, the work of network programmability developed into the notion of decoupling as a means to attain network programmability. Thus relating SDN to that degree of granularity is important to understand why industry is pondering over so many opensource projects and why protocol such as OpenFlow are gaining momentum so quickly as de-facto south bound protocol. The popularity of Opendaylight and Ryu controllers surely made OpenFlow a well-known name and most sought after as southbound protocol for network programmability. But does this mean, OpenFlow is the best solution there is for southbound protocol?

In this article, I am introducing somewhat forgotten yet an important framework and protocol for Control and Data plane separation (decoupling), “ForCES (Forwarding & Control Element Separation)”. The ForCES is an undertaking of IETF (Internet Engineering Task Force) and defined through a number of RFCs (Request for Comments). In my previous article at https://www.linkedin.com/pulse/network-virtualization-101-nve-overlay-sdn-dhiman-chowdhury?trk=pulse_spock-articles , I discussed about the historical significance of ForCES. It is important to note, the notion of decoupling originated from the scholarly works of ForCES led to the research work of Ethane project in 2007 and OpenFlow in 2008. The following diagram depicts the timeline for each.

Figure 1: Timeline depicting historical significance of ForCES.

In the early experimentations, scholars proposed the notion of separating control and data plane for networking processor (Olsson, et al., n.d.) [1], which pave the way for programmable networks or programmability in network devices. Today the context is further expanded to network design and architecture of NextGen networks. It is a good idea and in fact essential for future of networking. However, there are major predicaments to such abstraction e.g. network latency and scalability issues especially when control element is centralized and separated from data plane. This concern bogged down the wide spread deployment of some of the commonly used protocols such as OpenFlow which relies mainly on wildcard matches to push “packet flows” rules to the networking device. Such approach requires that wildcard matches are applied through TCAMs in the switch.

TCAM (Ternary Content Addressable Memory) and OpenFlow Issues

The TCAM (Ternary Content Addressable Memory) is a specialized high speed memory that allows searches of entire content within single clock-cycle. However, TCAM size are often limited in network devices due high cost of external TCAMs and thus such approach limits the number of flow table entries can be applied on a network device raising the question for scalability (Braun & Menth, 2014) [2].

All is not bad, whitebox are good news for OpenFlow implementation as some of these products are implementing server like capabilities packed in 1 RU box with increase CPU capacity, memory and TCAM sizes (Bifulco & Matsiuk, 2015) [3]. For example, Agema’s AGC7648 family of product supports two external TCAMs supporting more than 1.25 Millions IPV4 routes lookup. The AGC7648 combines a relatively powerful CPU with internal 750k UFT (Unified Forwarding table) on the switching chip, two external TCAMs and 6Gbytes of external port buffer. Enough horse power packed in 1 RU size box. In such platform, OpenFlow controller has better chance implementing significant number of flows than products without external TCAMs. However, question of latency and performance are still a big challenge for OpenFlow based Deployment despite the claim that OF-TTP (OpenFlow Table Type Patern) resolves scalability and performance issues.
Bifulco & Matsiuk (2015) argue that combination of software based switching function (which they denote as shadow switch) [3] and TCAM will highly improve throughput in OpenFlow deployment and thus the proposal addresses the concern for performance.

Why ForCES?

Some scholars consider wildcard matches that requires significant TCAM size on networking devices is a handicap approach. Bifulco & Matsiuk, 2015 [3] think such drawback can be improved through a combination of software switching function and TCAM to increase overall throughput. An alternative to OpenFlow in networking programmability field, Haleplidis et al (2015) [4] argue is ForCES. Though it remain experimental at Academic world for elastic routing architecture, it lack of presence in commercial world does not make it bad alternative. It may even surprise you knowing that some of world’s largest service providers, and to my knowledge, few mega data centers are deploying ForCES framework. There are few startups have either developed ForCES based controller and/or working towards developing it.

To this abstraction, you may find this news interesting:https://www.sdxcentral.com/articles/news/verizon-uses-radisys-mojatatu-sdn-nfv/2016/06/ .

The ForCES provides a framework and defines open API/protocols that clearly separate control and forwarding planes. Although many such API/protocols has been developed and/or proposed (.e.g. OpenFlow and RestAPI), the real strength of ForCES lies with its model which enables the description of new data-path functionality without changing the protocol between the control and forwarding plane (Haleplidis et al, 2015) [4]. In contrast, OpenFlow requires implementation of defined protocol at controller and Forwarding plane (e.g. OpenFlow agent at networking device). ForCES extend the notion of elastic routing architecture with belief that data plane packet processing may need additional functionalities programmed in the future while the switches are deployed in the network. This abstraction innately differentiate it from other existing SDN protocols/APIs such as OpenFlow.

In my suggestion, ForCES serves as a good framework upon which to develop elastic routing architecture for networking devices considering forwarding elements and control elements part of a system. In that way, centralized controller and networking devices are capable of exchanging capabilities and be part of an elastic routing architecture system.

ForCES Framework

IETF formed ForCES Working group in 2001during the time of dedicated network processor and that led to its main objective of standardizing open programmable interfaces for off-the-shelf Network processor devices. However, the framework was later influenced by programmable network and the IEEE P1520 initiatives [Tourrilhes, 2014] [5].

Figure 2: ForCES Timeline: a brief overview.

Several RFCs were written since then as depicted in the timeline above. The following table shows the list of ForCES RFCs and their purposes.

Table 1: ForCES RFC Information.

In ForCES framework, the elastic routing architecture or network element (NE) comprises of two parts FE (Forwarding Element) and CE (Control Element). Both of these elements are implemented through ForCES Agent which includes a set of protocols and models.

Figure 3: ForCES Framework.

The ForCES framework allows developers to define their own FE abstraction models for implementation. This does not constrained CE to control and manage ForCES-modeled FEs. More importantly, ForCES framework is also protocol agnostic meaning a vendor may choose to use a traditional networking protocol to communicate between CEs and FEs. In theory, CE can implement routing protocols like OSPF and BGP and packets that cannot be handled by FEs are redirected to CEs (Haleplidis et al, 2015) [4].

ForCES Protocol

RFC5810 suggests two essential elements of ForCES Protocol for communication between FEs and CEs: PL (Protocol Layer) and TML (Transport Mapping Layer). The Protocol Layer (PL) is infact ForCES protocol which relies on TML like TCP and other underlying transport protocols for communication between FEs and CEs.

Figure 4: ForCES Protocol Implementation.

The implementation of PL and TML may differ between control plane and management plane and in high availability scenarios. Figure 5 shows various interface point that are referenced in RFC5810 and behavior of PL/TML are also suggested accordingly in the RFC. A further discussion about those reference interfaces are available in RFC 3746.

FEs relies on underlying hardware abstraction layer for utilizing hardware resources to process packets. The handling of packets are directed either by a single controller or multiple controllers over ForCES Interface depending upon network deployment. The ForCES interface (herein PL and TML: please refer figure 4) may deploy either TCP or UDP for communication between FE and CE. For example, a ForCES implementation may choose to use TCP port 6653 (similar to OpenFlow) for communication between FE and CE.

Figure 5: ForCES interfaces in a high availability scenario as defined in RFC5810.

One important distinction between OpenFlow and ForCES is the use of logical functional block (LFB) in FE which allows several functions including packet processing and capability exchange. Rather than treating network device as dumb (as in OpenFlow implementation), ForCES allows capability exchange to occur between CE and FE as depicted in figure 6. As such, FE could reach out to CE, indicating its ability or inability to process a packet. In cases where FE is unable to process a packet perhaps due to lack of underlying hardware support, e.g. VxLAN or MPLS, CE will perform the function on behalf of the FE.

Figure 6: Implementation of LFB and capability exchange between FE and CE.

Such implementation of capability and state exchanges are important consideration for NextGen architecture and could serve as an ideal framework upon which a true elastic routing architecture can be developed.

In summary, I would recommend developers and vendors to explore further on ForCES specific framework for the development of NextGen elastic routing architecture, one that can be implemented in a single chassis (as in ForCES) as well in disaggregation model in which CE is implemented in server base controller and FE is implemented in networking device. More importantly, learning from ForCES should be applied towards developing a flexible routing architecture that make use of traditional routing and transport protocol as well allows network devices and controller to exchanges capability parameters. The work of ForCES worth the praise and really an outstanding framework for elastic routing architecture development.

Reference:

[1] Olsson, R., Hagsand, O., Laas, J. & Gorden, B., n,.d. Control and forwarding-plane separation of an open-source router. Uppsala University and KTH. Available online athttp://www.inter.uadm.uu.se/digitalAssets/21/21239_opensourcerouting.pdf .

[2] Braun, W. & Meth, M., 2014. Software-Defined Networking Using OpenFlow: Protocols, Applications and Architectural Design Choices. Future Internet 2014, 6, 302-336; doi:10.3390/fi6020302.

[3] Bifulco R., & Matsiuk, A., 2015. Towards Scalable SDN Switches: Enabling Faster Flow Table Entries Installation. SIGCOMM ’15 August 17-21, 2015, London, United Kingdom.

[4] Haleplidis, E., Salim, J.H., Halpern, J.M., Hares, S., Pentikousis, K., Ogawa, K., Wang, W., Denazis, S. & Koufopavlou, O., 2015. Network Programmability With ForCES. IEEE COMMUNICATION SURVEYS & TUTORIALS, VOL. 17, NO. 3, THIRD QUARTER 2015.

[5] Tourrilhes, J., Sharma, P., Banerjee, S. & Pettit, J., 2014. SDN and openflow evolution: A standards perspective. Computer, vol. 47, no. 11, pp. 22–29, Nov. 2014

Network Virtualization 101: SDN APIs and Protocols

This article is part of my SDN/network virtualization series “Network Virtualization 101”. To view other articles in this series and/or learn more about SDN and network virtualization, please visithttps://www.linkedin.com/today/author/0_1J4WrxUu5ZhIszQwsUUcg_?trk=prof-sm or http://www.dhimanchowdhury.blogspot.com .

From the simplistic purview, SDN (Software Defined Networking) provides the means of dynamism in “networking infrastructure” solutions by facilitating programmable network control, centralized management and orchestration. For network control, it decouples control plane (think of it in terms of a set of conventional applications/protocol suits e.g. Firewall, NAT, load balancer and BGP etc) from data plane (in other words, networking equipment having only packet forwarding mechanism without conventional protocol suits) and allows centralized control over networking infrastructure. There are few variations to this approach, for example, a networking equipment with hybrid NOS (network operating system) may choose to implement both control plane protocols (conventional protocols suits) and SDN APIs/protocols in the same platform. On the other hand, some networking equipment may only implement agent such as “Indigo” allowing the device to solely depend on SDN controller for flow instructions. We will discuss such implementation later in this article.

Figure 1. Typical SDN (software Defined Networking) setup.

An SDN controller connects to Switch/routers or similar networking equipment through downstream communication interface using either API or protocol such as “OpenFlow” and “OVSDB”. This downstream communication interface (a shared boundary between two computing endpoints) is known as southbound Interface. Similar to southbound Interface, upstream communication from the controller to other server base tools such as “OpenStack” are realized through Northbound Interface. For example, “Neutron” server elements of OpenStack can be connected to “OpenDaylight” SDN controller using RESTful API as shown in the figure below. Let’s not get overwhelm by this, just keep the concept in mind as we explore SDN APIs and protocols in the subsequent sections. A detailed discussion on OpenStack and OpenDaylight connectivity will be presented in succeeding articles of this series.

Figure 2. OpenDaylight controller uses RESTful API to connect OpenStack Neutron server at Northbound Interface.

We discussed that both northbound and southbound interfaces use either API and/or protocols. A simplistic purview is that both are abstractions: API is top-down while protocol is bottom-up. An API (Application Programming Interface) is more suitable for client/server setup where API acts as the messenger between the two entities while tightly binding them together. We could further understand API by applying the analogy of restaurant order processing where “waitress” is a go-between customer and the kitchen to process the order and serve food. Here, API is synonymous to “Waitress” function from the operational perspective. Conversely, the protocol is a set of rules that governs communication between to networking end points. In order to establish a successful communication, both endpoints must speak the same language.

While some protocols and APIs may be newly introduced for SDN purposes, many other protocols/APIs referred are implemented or in use elsewhere. For example, OpenFlow, OVSDB and ForCES are new sets of protocols for the southbound interface (SBI) whereas BGP and XMPP are not new. BGP for example, a de-facto standard for how various routers within the internet “network mesh” communicate with each other.

The following table attempts to simplify the understanding of various SDN APIs/protocols and their intended use. Readership will find this table useful for SDN controller connectivity with network endpoint devices.

Table 1. Typical examples of SDN protocols and APIs.

[Please note, though I have listed Opencontrail and OpenVirtex as SDN controllers, those are more of a network hypervisor or network-level virtualization controller. We will explore further on the network hypervisor type controllers and other NOS type SDN controllers in the later part of this series.]

Among the typical SDN controllers listed in table 1, OpenDaylight provides a wide range of SBI protocols/plugins including LISP, SNMP and other Wireless Access point and IOT (Internet of Things) protocols. As of writing this article, there has been proposal or talks MP-BGP EVPN as standard control plane protocol for VxLAN. If MP-BGP EVPN considered a standard interface for VxLAN configuration, it may find its to some of the typical controllers as SBI protocol.

In the next article, I will discuss various SDN protocols and APIs starting with ForCES (Forwarding Control Element Separation) framework that was proposed by IETF as an alternative to OpenFlow. Some of the largest telecom service providers in the world are exploring the potential of deploying ForCES master-slave protocol framework. Find out why… please stay tuned for next articles.

Network Virtualization 101: NFV & VNF

In the previous articles ( http://www.dhimanchowdhury.blogspot.com ), I discussed how the need to share resources led to gradual development of technologies concerning network programmability and virtualization. I am hopeful that readership is now conversant on the notion of network virtualization (NV) and SDN and how network programmability technologies such as SDN coexist with topological virtualization of physical network. I connoted that both SDN and NV decouple network abstractions, the difference is that SDN separates data plane from control plane providing centralize network control and programming mechanisms while NV decouples physical infrastructure from logical topologies.

Figure 1. A diagrammatical representation of the interworks of NV and SDN.

With this distinction in mind, we embark on yet another discussion about how the needs for network programmability (i.e. SDN) and virtualization (i.e. NV) find common ground in NFV (Network Function Virtualization) architectural framework.

In October, 2012 some of major network/telecom operators got together at Darmstadt in Germany to devise a framework for NFV (Network Function Virtualization) (Haleplidis et al., 2015) penciling ideas from the technological advances in system virtualization, SDN and NV. It is an architectural framework that make use of virtualization technologies to define virtual elements of entire classes of node (compute, storage and network) functions into building block that can be connected or chained to deliver communication services. The framework is devised under the stewardship of ETSI (European Telecom Standard Institute) and various workgroups within ETSI entrusted with the responsibility to define different aspects of NFV framework (e.g. network elements and APIs etc).

The need for network/telecom operators to devise a white paper on the NFV framework comes from the growing challenges to accommodate customer demands for improve services while reducing CAPEX (Capital Expenditure) and OPEX (Operational Expenditure). The NFV framework aims to address those challenges by virtualizing network functions (NF) [e.g. load balancer, firewall, PE Router, CG-NAT etc] and allowing network/telecom operators to deploy those NFs at will on high end devices. One of the major challenges in NFV deployment is the orchestration and management of network functions (NF) such as load balancer, firewall and other control functionalities. A workgroup within ETSI NFV undertaking, namely “MANO” (Management and Orchestration) was formed to develop specification on management and orchestration framework for NFV architecture (figure 2). We will discuss framework of MANO later in this article but for now think of NFV framework as the main architectural construct or reference model for which VNF, NF and Management/Orchestrations are the elements or subsets.

Figure 2. Timeline on NFV related research works at ETSI (source: ETSI, 2015; ETSI, 2014).

One more important subset of NFV framework is the NFVI (Network Function Virtualization Infrastructure). The NFVI specifies various physical resources and how abstraction of those resources are virtualized, e.g. virtual compute, virtual network and virtual storage. Applying our understanding of system virtualization (hypervisor/VM concept) [if you need further clarification, please visit my blog: http://www.dhimanchowdhury.blogspot.com/2015/07/network-virtualization-101-prelude.html ], the NFVI framework can be understood as hardware resources (compute, storage and network) virtualized by hypervisor for which VM is virtual instances and VNFs (Virtual Network Functions) are implemented as application on top of VMs. Please refer to the NFV reference model as depicted in figure 3. The NFV reference model comprises of three main subsets: NFVI, VNFs and NFV Management and Orchestration. The element management system (EMS) is commonly used in a typical telecom network to manage network element. In my good old Nortel days, we used element manager to retrieve configuration information in various networking and telecom gears while pulling device information through NMS. Once a device is found in the network using NMS network map, you could double click on the device to open up element manager to configure and manage the device in a way that is not possible through simply using MIBs as you would through the NMS. I hope this imparts you the usefulness and purpose of EMS. In a telecom network EMS brings many values, for example, you could integrate different vendors and their element manager to a NMS and manage the entire group of systems through a single NMS.

Figure 3. The NFV reference Model.

The OSS (Operational/Operation Support System) and BSS (Business Support Systems) are back-office software applications to supports activities related to services. For example, OSS includes software toolsets that are related to management and operation aspects of network services while BSS toolsets are used for billing, order management and CRM etc. For telecom professional these terms and software toolsets are nothing new but for rest of the readership this brief explanation is useful.

Now that we understand some key terms and functional elements that connects with NFV framework, let us explore essential components of NFV reference model. Please note that NFVI specifies hypervisor based virtualization but it is also possible to use containers (a notion mainly associated with LinuX Container) to support same VNF instances as in hypervisor/VM based system. The notion of container was born in 2005, out of necessities for webscale services at Google Data center. Google was experimenting with hypervisor based virtualization at the time but soon realized that performance penalty for such virtualization is too high and was not elastic enough to support a web-based services at scale (Bottomley, 2014). During the time, a group of engineers were working on Linux around a concept based on cgroup called process container who were later hired by Google to advance experimentation for webscale services. Some of these Cgroup technology later found its way into Linux Kernel and the LinuX Container (LXC) project was born in January 2008.
To date, implementation of containers are mainly done at server level system virtualization but works are underway to realize some benefits of container based virtualization at networking gears. As such, implementation of VNFs in networking switching gears may well be done using container virtualization techniques within next two years.

The VNFs (Virtual Network Functions) are simply network node or network functions (NF) running as virtual instance on top of VMs (virtual machines) (Vilalta, et al., 2014). For example, a VNF can be load balancer, NAT/firewall and many other similar network/network node functions. Today, much of the VNF implementations are done at server level but it is also possible to implement VNFs on networking gears. NFV framework (ETSI, 2014) did not constrained use case scenarios of VNFs and works are already underway to have server like experience in networking gears making it possible to implements VNFs in various use case scenarios, such as Residential, CDN, Fixed Access Networks, Mobile Station and Mobile Core & IMS deployments. This implies that even future residential or CPE gateways will have VNFs implemented on them allowing service providers to deploy service at will.

The Management and configuration of VNFs and other subsets of the NFV reference model (as depicted in figure 3) is done through MANO (Management and Orchestration) toolsets. The MANO architecture presents three elements: VIM (Virtual Infrastructure Manager), VNF manager and Orchestrator. The VIM is used for the management and allocation of virtual resources, e.g. add/remove and modification of resources related to compute, network and storage. The Openstack constitutes a close reality to ideal VIM but requires further works related to various plugins and integration of automation/configuration chef and puppet tools to realize its deployment in service provider network.

The VNF managers handle the configuration, lifecycle management, and element management of the virtualized network functions. Within Openstack initiative, the “Tracker” project aims to deliver VNF manger and NFV orchestration capabilities for NFV platforms. For further information on “Tracker”, please visit https://wiki.openstack.org/wiki/Tacker .

Figure 4. The diagrammatical representation of Openstack “Open NFV Orchestration” “Tracker” project focus. (Openstack, 2015).

Apart from the Openstack Tracker project, many other vendors also offer application solutions but challenges remain to develop a comprehensive toolsets due to complexities of unified transport networks systems.

I hope this article provided a brief overview of NFV reference model: readership should follow the work of ETSI at http://www.etsi.org/technologies-clusters/technologies/nfv to advance their understanding and stay in touch with the latest works in this field. In the next article, we will explore various APIS of SDN and understand how SDN and NFV are deployed in a given network scenario.

Please stay tune and follow me. If you are interested, please join me in the free live webinar on “Open Networking” at https://lnkd.in/bhydy6x .

Reference

[Bottomley, 2014] Bottomley, J., 2014. What is All the Container Hype? Parallels.

[Haleplidis et al., 2001] Haleplidis, E., Salim H. J., Denazis, S. & Koufopavlou, O, 2015. Towards a Network Abstraction Model for SDN. Network System Management (2015) 23:309-327.

[ETSI, 2014] ETSI, 2014. Network Functions Virtualisation – White Paper #3. ETSI NFV portal. Available online at https://portal.etsi.org/Portals/0/TBpages/NFV/Docs/NFV_White_Paper3.pdf

[ETSI, 2015] ETSI, 2015. NFV activity report 2014. ETSI NFV Portal. Available online at https://portal.etsi.org/TBSiteMap/NFV/ActivityReport.aspx

[Openstack, 2015] Openstack, 2015. Tacker. Available online at https://wiki.openstack.org/wiki/Tacker .

[Vilalta, et al., 2014] Vilalta, R., Muñoz, R., Casellas, R., Martínez, R., López, V., & López, D., 2014). Transport PCE network function virtualization. In European Conf. Optical Communications, Cannes, France.

Posted by Dhiman Chowdhury at 4:36 PM

Labels: Cloud Architecture, Cloud Computing, Data Center Networking, Hybrid Cloud, NFV, Software Define Data Center

Thursday, September 3, 2015

Network Virtualization 101: The NV

In previous article, I have discussed about historical perspective of Network virtualization ( http://www.dhimanchowdhury.blogspot.com/2015/07/network-virtualization-101-nve-overlay.html ): how the need to share resources and induce flexibility and programmability in network environment led to the series of research undertakings, e.g. MBone project (Almeroth, C.K., 2000) – an experimental backbone project for carrying IP “multicast” that was developed in 1994. If you have not read that article, I suggest that you do: it will help you understand the development in network virtualization and benefits thereof.

Figure 1. Timeline of Network Virtualization.

This article is third in the “Network Virtualization 101” series.

In the previous article, I presented a review of network programmability works in four stages rather than in chronological order: three stages were discussed in earlier section and the fourth “network virtualization” (NV) is presented herein as follows.

The notion of “network virtualization” (NV) can be understood as decoupling physical topology from logical topology (e.g., Overlay networks) and as such, implementation does not require SDN. Similarly, the common notion of SDN (separation of control plane and data plane) does not require network virtualization. This distinction is important since many may be confuse by the symbiotic nature in which network virtualization and SDN relate (Feamster, N., Rexford, J. & Zegura, E., 2014):

SDN as enabling technology: with the advent of cloud computing, service providers faced the challenge to share and isolate resources to multiple tenants in a way that make best use of available network infrastructure. A common method to share such isolation at VM level is to use overlay networks through protocols such as VxLAN and NVGRE. While VxLAN and/or NVGRE does not require SDN to implement but having the capability to provision the network for VxLAN and/or NVGRE from centralized server surely helpful. Another example of SDN as an enabling is Nicira’s network virtualization platform or NVP. The NVP framework implements Open vSwitch (a virtual switching platform), a controller and South bound API to facilitate network transport. The Open vSwitch is hardware agnostic and can be implemented in servers without the need for networking gears.
Slicing or virtualizing an SDN: A hybrid switch for example implements both traditional protocol suits and OF (OpenFlow) agents and other flow control APIs. With appropriate arbitration mechanisms, network flow can be logically separated from other logical instances of network. Similarly, Flowvisor (a special purpose controller that works as transport proxy between OF agent and OF controller) allows slicing of network resources and delegates control of each slice to a different controller (Flowvisor, 2014).

From historical perspective, the work on Network virtualization can be predated to the early days of MBone experiment. The MBone otherwise known as “Multicast Backbone” is a virtual network built on top of internet. It was invented by Van Jacobson, Steve Deering and Stephen Casner in 1992 as part of an undertaking by IETF (Internet Engineering Task Force). In the early 1990s, majority of the routers in the internet did not support IP Multicasting and packets were transported through IP unicast. As a result one-to-many communication was difficult. The solution was MBone or “Multicast Backbone” in which multicast function provided by workstation running a daemon process known as “mrouted” (Almeroth, 2000). Workstation running “mrouted” process is known as “mrouter” (essentially a multicast router). These mrouters are than placed in special group of LAN or single LAN that are multicast capable. The “mrouted” process received unicast-encapsulated multicast packets on an incoming interface and then forwarded packets over the appropriate set of outgoing interfaces. Connectivity among these machines was provided using point-to-point, IP-encapsulated tunnels. Each tunnel is connected two endpoints via one logical link. The routing decisions were made using DVMRP (Distance Vector Routing Protocol) as shown the figure below.

Figure 2. MBone topology during the early years of its deployment.

The DVMRP since then replaced by PIM (Protocol Independent Multicast) helping MBone to be integrated with internet than its initial attempts. For many years, network equipment supported the creation of virtual networks, e.g. VLAN which allows the creation of multiple logical networks on top of physical topology. But such network virtualization is limited to L2 network segments and impedes on the deployment of new technologies traversing across the network. To overcome this, researchers and practitioner resorted to running overlay networks which allows endpoint nodes to run their own control plane and forward data traffic and control-plane message across the networks traversing multi hop L3 networks. The MBone (for multicast) and 6Bone (for IPV6) are example of such overlay network virtualization. In the previous article, I discussed example of overlay network architecture and some of the protocol used for tunneling and will explore this further in the succeeding articles about network virtualization configurations. The complete survey of network virtualization is cumbersome, however, the historical perspective I am presenting herein, though brief, is important in the research of network programmability (SDN) and in the gradual development of programmable and dynamic network systems. It is to be noted both SDN and network virtualization is tightly coupled despite their distinctions. Programmable network (i.e. SDN) often presume “network virtualization” as an integral part to share network infrastructure for multi-tenant services supporting logical network topologies that differ from physical network. The early overlay network that is essential in evaluating and understanding “network virtualization” often used dedicated nodes running special protocols. The notion of such early day overlay network soon expanded to include any host computer that run special application in hope of supporting peer to peer file sharing application (e.g. Napster; Wikipedia, 2015). The research on peer to peer networking reignited interest and research works in the development of robust overlay network technologies. An example of such work is “Resilient Overlay Networks” (Andersen et al., 2015) in which a small number of network nodes form overlay network detecting network failure and recovering quickly from network issues and performance problem. Since overlay network does not require any special equipment (unlike Active network; please refer to my previous article), researchers began building experimental infrastructure like Planetlab (Peterson et al., 2002) to support wider research works on network virtualization. Interestingly, PlanetLab itself was a form of “programmable router/switch” active networking, but using a collection of servers rather than the network nodes, and offering programmers a conventional operating system (i.e., Linux) (Feamster, N., Rexford, J. & Zegura, E., 2014). The project of GENI (GENI, 2015) took this notion of programmable virtual network infrastructure to next level supporting much large scale national experimental for research in networking and distributed system.

Figure 3. GENI – The vast experimental virtualized network infrastructure project (GENI, 2015). Figure (Courtsey, GENI, 2015).

If you are interested in experimenting your concept in a virtualized network infrastructure, please visit http://groups.geni.net/geni/wiki/GENIConcepts and join the project.

Considering the project like GENI, one can easily perceive the potential of network virtualization. Some researchers argued that network virtualization is key to next generation internet architecture. In the first article of this series, discussing about NVE (Network Virtualization Environment) I explored the theoretical connotations from various scholars regarding the need for next generation internet and service provider network in which multiple network architectures can coexist at the same time (each optimized for different applications or requirements, or run by different business entities), and evolve over time to meet changing needs (Feamster, N., Rexford, J. & Zegura, E., 2014; Carapinha & Jiménez, 2009; Chowdhury & Boutaba, 2009; Chowdhury & Boutaba, 2008).

I hope this brief overview of NV (Network Virtualization) is helpful in understanding the difference and dependencies between network virtualization and SDN and the importance of network virtualization in future network design. You will find the basic understanding helpful in the succeeding articles about network architecture and configurations.

In the next article, I will extend the notion of Network Virtualization to VNF (Virtual Network Function) and NFV (Network Function Virtualization). Please stay tune and follow me at linkedin (https://www.linkedin.com/in/dhiman1 ), twitter @dchowdhu ( https://twitter.com/dchowdhu ) and google plus (https://plus.google.com/u/0/+DhimanChowdhury/posts ). You may also subscribe to all these feeds through Agema System’s linkedin page at https://www.linkedin.com/company/agema-systems-inc?trk=top_nav_home

Reference

[Almeroth, C.K., 2000] Almeroth, C.K., 2000. The Evolution of Multicast: From the MBone to Interdomain Multicast to Internet2 Deployment. IEEE Network. Available online at http://www.cs.ucsb.edu/~almeroth/classes/F05.276/papers/evolution.pdf .

[Andersen et al., 2001] Andersen, D. G., Balakrishnan, H., Kaashoek, M. F. & Morris, R., 2015. Resilient Overlay Networks. In Proc. 18th ACM Symposium on Operating Systems Principles (SOSP), pages 131–145, Banff, Canada, Oct. 2001.

[Carapinha, J. & Jiménez, J. 2009] Carapinha, J. & Jiménez, J. 2009. VISA ’09 Proceedings of the 1st ACM workshop on Virtualized infrastructure systems and architectures. The ACM Digital Library.

[Chowdhury, K.M.M.N. & Boutaba, R., 2008 ] Chowdhury, K.M.M.N. & Boutaba, R., 2008. A Survey of Network Virtualization. Technical Report CS-2008-25. University of Waterloo.

[Chowdhury, K.M.M.N. & Boutaba, R., 2009 ] Chowdhury, K.M.M.N. & Boutaba, R., 2009. Network Virtualization: State of the Art and Research Challenges. IEEE COMMUNICATIONS MAGAZINE.

[Clark et al., 2006] Clark, D., Lehr, B., Bauer, S., Faratin, P., Sami, R. & Wroclawski, J., 2006. Overlay Networks and the Future of the Internet. Communications & Strategies, no. 63, 3rd quarter 2006, p. 109.

[Feamster, N., Rexford, J. & Zegura, E., 2014] Feamster, N., Rexford, J. & Zegura, E., 2014. The Road to SDN: An Intellectual History of Programmable Networks. ACM Queue, 2014.

Flowvisor, 2014. Flowvisor. Atlassian Confluence Open Source Project: University of Stanford. Available online at https://openflow.stanford.edu/display/DOCS/Flowvisor

[Feamster et al., 2004] Feamster, N., Balakrishnan, H., Rexford, J., Shaikh, A. & van der Merwe, J., 2004. The Case for Separating Routing from Routers. SIGCOMM’04 Workshops, Aug. 30-Sept. 3, 2004, Portland, Oregon, USA.

[GENI, 2015] GENI: Global Environment for Network Innovations. Available online at http://www.geni.net/.

[Peterson, et al., 2002] Peterson, L., Anderson, T., Culler, D. & Roscoe, T., 2002. A Blueprint for Introducing Disruptive Technology into the Internet. Planet Lab. Proceedings of the First ACM Workshop on Hot Topics in Networks (HotNets-I), Princeton, NJ, October 2002.

[Wikipedia, 2015]. Wikipedia, 2015. Napster. Wikipedia: The free Encyclopedia. Available online athttps://en.wikipedia.org/wiki/Napster