Network Management Challenge
Networks, connectivity and mobile access are evolving, growing exponentially in volume and scope, with spectrum extending from large corporate and telecom networks to connectivity between thousands of “constrained devices”, including Machine-to-Machine (M2M) automation networks and sensor-based, data gathering and analysis networks, described as the Internet of Things (IoT).
The computing networks increasingly take shape of large multi-campus installations, providing applications, storage and connectivity services, often via multi-tenant cloud architectures.
All networking sectors require Internet & mobile connectivity, demanding security, instant remote configurations, monitoring and network management. The challenge is in carrying these concepts across different networking architectures, levels of computing, connectivity and memory resources while retaining some commonalities in application APIs to provide an effective foundation for automation and sophisticated management applications of networking infrastructure, services and information.
Private, Carrier & Constrained Device Networks
The size and complexity of today’s networks are evident, from volumes of specialized function boxes in carrier networks, and the complexity of network applications to connectivity intelligence in just about any device that produces information or requires receiving configuration and command data in IoT networks.
Applications are far-reaching and expanding, from business, services and carrier applications to industrial, energy, medical, building automation, smart cities, transportation, dynamic consumer analysis, logistics sector, to name a few; the mandate is creating intelligent eco-systems providing desired data and application services and automatically responding to events, reporting systems status to management applications or adjusting environment to evolving conditions, all in line with a pre-configured policy.
Software-Defined Networks & Network Function Virtualisation
Responding to complexity, to provide reliable service and manageability, networks are changing, adopting new architectures. These are taking cues from modern computing environments with regards to virtualization. This capability improves server and network storage availability, it provides for rapid provisioning of services in demand, and it can adapt and scale network connectivity and capacity to changing applications and use patterns.
The virtual networking architecture’s promises to make “the dream of wire once and use forever” possible … 🙂 … It is the approach that re-makes the network connectivity on-demand, making it software configurable, in an application & usage adaptable fashion.
SDN and NFV Industry Initiatives
-
Software Defined Networks (SDN) Project at the ONF (Open Networking Foundation) – a complementary approach, conceives the decomposition of networking equipment into a Control Layer and a Forwarding, Datapath Layer, where the Control Layer offers software APIs that deal with the real time management of flows in the forwarding plane (OpenFlow protocol) and provides for remote configuration of datapath resources, via remote configuration protocols, allowing for increasing bandwidth and connectivity where and when it’s needed.
-
Network Function Virtualisation (NFV) Project at the ETSI (European Telecommunications Standards Institute) – potentially simplifies the physical network infrastructure by moving away from many specialized devices to fewer standard Servers, Storage, Core & Edge Switch devices while implementing network functions in software, an attractive proposition for carriers.
-
Linux Foundation (LF) Open DayLight Project – a collaborative SDN / NFV Open DayLight project is supported by numerous networking vendors and industry groups.
-
Linux Foundation (LF) Open Platform OPNFV Project – an open source OPNFV Project focused on creation of an NFV reference platform accelerating new products and services.
-
Interface to the Routing System Working Group – the I2RS group of IETF (the Internet Engineering Task Force) is developing an SDN strategy that aims to balance the routing decision between central controllers and field proven routing protocols, running across the network’s hardware devices.
Management & Orchestration (MANO) Initiatives
Service, Network & Device Configuration Models & Protocols
Multiple vendors and several next generation open networking architecture projects have selected the IETF’s YANG data modeling language and NETCONF, RESTCONF and / or gRPC based configuration protocols as their technology base for automation of network service configurations, device configurations and operational management. An excellent overview of these concepts is provided in the informational rfc6244, An Architecture for Network Management Using NETCONF and YANG.
Similarly, the management protocols of IoT and M2M constrained devices space, Constrained Application Protocol and CoAP Management Interface, were selected for efficient access to data contained in the YANG Datastores of such devices. These web protocols are simplified versions of the web HTTP transport protocol and RESTCONF programmatic interface. The CoAP / CoMI protocols have been designed to use the minimum resources of their networking and embedded environments, solving a unique system architecture and design challenges.
The industry drive is toward open, vendor-neutral, service and device YANG data models, software APIs and tools that would enable automation of configuration and network management, thus help increase reliability and flexibility of networking services, while lowering operational costs and enabling new business models for network operators.
The list below is a summary of relevant standards and industry wide development projects:
- YANG – IETF Modeling Language Standard – defined by rfc6020 – YANG 1.0, new features defined by rfc7950 – YANG 1.1, with updates by rfc8342 – Network Management Datastore Architecture (NMDA). The NMDA’s use requires implementation of YANG Library, rfc8525 by the YANG Datastore Server
- NETMOD IETF – IETF Network Modeling Working Group developing vendor-neutral, open YANG data models used to configure and manage the networks.
- YANG Catalog – a structured depository of YANG modules, related metadata, embedded search and validation tools and data model-driven open source management tools (YDK – YANG Development Kit) from IETF, Broadband Forum, other SDOs, multiple industry groups and vendors.
- OpenConfig – an informal consortium of network operators focused on development of vendor-neutral, open YANG models used to configure & manage the networks; open source management tools, such as PYANG based plugin converting YANG models into hierarchy of Python config application classes, focused on OpenConfig YANG models (PyangBind).
- other Model-Driven Configuration Applications – open source projects focused on automation of configurations and operations, please see YANG Resource links
- NETCONF & RESTCONF – IETF Network Configuration protocols – NETCONF defined by rfc6241, with extensions to support rfc8342 Network Management Datastore Architecture (NMDA) described in rfc8526. The HTTP-based configuration protocol RESTCONF is defined by rfc8040 with NMDA extensions described in rfc8527. The NMDA’s use requires implementation of YANG Library, rfc8525 by the YANG Datastore Server.
- NETCONF IETF – IETF Network Configuration Working Group developing NETCONF and RESTCONF configuration protocols.
- gRPC – an open source RPC framework using Protobuf encoding over HTTP/2 transport supporting bi-directional streaming model, underpinning of gNMI / gNOI configuration and control automation protocols.
- gNMI & gNOI – configuration protocol – gRPC based Network Management Interface (gNMI) & set of operational microservices management commands – gRPC based Network Operations Interface (gNOI) – open source technologies, providing for automation of configuration and operations of the network, enabling the Streaming Telemetry publish – subscribe monitoring model.
- CoAP / CoMI – network management interface standard for constrained devices of IoT and M2M space. The CoAP (Constrained Application Protocol – rfc7252) and CoMI (CoAP Management Interface – draft-ietf-comi) are adaptations of HTTP / RESTCONF web protocols and are used to access the management data resources specified in the YANG Datastores with focus on minimal payloads required for operating in constrained IoT networks and device’s embedded environments.
Please also review these YANG / NETCONF online resources.
The specific goals of the networking industry efforts are to automate the delivery of Service Definition Models, then resulting Network and Device Configuration & Operational Models, with automation based on higher level abstractions focused on “what and not how” is required, please review rfc8309 – Service Models Explained for more details.
Achieving such automation goals requires that the networking gear supports vendor-neutral, open YANG data models, implemented across different vendors equipment, thus making the network fully compatible with model-aware configuration and management automation tools.
Today the YANG models are being developed by IETF standardization working groups, equipment vendors, other SDO’s and a number of networking industry consortiums, such as the OpenConfig project, with all groups making progress toward improved automation. The intended architecture would provide Standard YANG Modules, augmented by Vendor and User (network operators groups) Modules and Extensions thus enabling configuration and operational automation – please review rfc8199 – YANG Module Classification.
Currently, the YANG and NETCONF standards are represented by stable YANG 1.0 rfc6020 and NETCONF rfc6241 documents. The networking community implemented and successfully uses these standards, even as some differences emerged in depicting the “intended vs operational configuration” by YANG modules created by different organizations. In fact this work prompted the new advancements that better provide for the goal of automated configuration and management processes in a YANG model based and software API driven network, while providing a continuity of present implementations.
The new standard for Network Management Datastore Architecture (NMDA), rfc8342 offers a solution for a more realistic device configuration handling. It provides a notion of multiple datastores that reflect the reality of transitions and differences between the initially configured values and those operationally used, while such values transition from “configured” to “intended” then to “applied” before settling on “operationally used configuration”. These transitions are effected by local factors, also by learned and system provided updates. The main architectural advantage of NMDA approach is simplification of the YANG model schema, the configuration object appears in a tree once but it might have different values in different Datastores (for example Intended or Operational Datastore).
The NDMA updates certain aspects of the latest YANG 1.1 modeling standard, rfc7950 and the NETCONF standard, rfc6241 and calls for the use of YANG Library, rfc8525. It also forces changes to standard YANG models documents, thus making them more compatible with automated configuration and operation processes.
The OpenConfig project represents the approach where differences between “intended vs operational configuration” are handled by the YANG model schema, where configuration object may appear in a YANG model’s tree twice, in a “configuration” branch and also in its “operational” branch. The end goal is the same, to provide an open YANG data models aiming to define the configuration and operational state of networks and devices for automation of common networking services.
The main thing to note here is that IETF’s, OpenConfig’s and other SDO’s YANG models are configuration protocol independent. Time will show if one of these approaches prevails or both architectures will co-exist.
The IETF’s NETCONF standardization program produced a number of additional YANG Datastore Server capabilities that enable automatic configurations and monitoring of large and practical networks of varied devices. Briefly:
- draft-ietf-netconf-subscribed-notifications – capability to subscribe to publisher’s (NETCONF equipped, YANG Datastore Server) events streams , extending the capability defined in the rfc5277, NETCONF Event Notifications
- rfc8071 – Call Home – a capability of YANG Datastore Server to initiate a secure connection to a remote NETCONF or RESTCONF Client to receive and act on a pre-determined configuration, based on any number of criteria.
- subscription to YANG DataStores – draft-ietf-netconf-yang-push-22 – push technology aiming at Telemetry and Monitoring applications. The NETCONF equipped, YANG Datastore Servers supporting this NETCONF standard’s draft extension provide a publish / subscribe interfaces allowing a subscriptions against the updates to any datastore implemented by the device’s Server. A complete description of Requirements for Subscription to YANG Datastores is contained in informational rfc7923.
All groups committed to YANG standard modeling language promote the use of NETCONF and RESTCONF configuration protocols, with OpenConfig project (and multiple vendors) also supporting the gRPC based Network Management Interface (gNMI) configuration protocol, along with its gRPC Network Operational Interface (gNOI) extension. All three protocols can be used for configurations, gRPC based protocol also enables automation of control and monitoring of the network by providing the Streaming Telemetry, a configurable push model for collection of network operational data. This capability is enabled by gRPC’s use of HTTP/2 transport that supports the bi-directional data streaming model.
These protocols use XML / JSON / gProtoBuf encoding over SSH, HTTPS or HTTP/2 transport and use RPCs to model the YANG configuration, operational data and actions for delivery to remote devices. Regardless of characteristics of vendor’s NFV / SDN architecture implementations, YANG / NETCONF / RESTCONF / gRPC/gNMI/gNOI technologies promise a scalable and extensible network configuration and management automation framework.
For constrained devices of IoT and M2M space the CoAP / CoMI protocols run on top of UDP, binding to Datagram Transport Layer Security (DTLS) layer for security, and use a set of encoding rules for the Concise Binary Object Representation (CBOR) to identify YANG Datastore’s management objects in order to achieve minimum possible payloads.
While adoption of YANG and the associated configuration and control protocols seems assured going forward, the support of the existing networking installations and management platforms in the transition period requires the availability of present SNMP monitoring and CLI configuration capabilities in the YANG model based devices as well.
NuDesign Product Direction
NuDesign provides its OEM customers with software applications, code generation tools and development libraries oriented on building of secure management, configuration and monitoring agents for servers and embedded systems.
The company has invested into migration from legacy SNMP and CLI only interfaces, to supporting integrated and synchronized, YANG based, configuration, control and monitoring solutions that adopted NETCONF / RESTCONF / SNMP / CLI / Web protocols, with gRPC based Network Management Interface (gNMI) protocol as a future add-on option*.
NuDesign’s ndCONF Agent Development Studio current release is based on the NETCONF rfc6241 and YANG 1.0 rfc6020 (with NMDA rfc8342, YANG 1.1 rfc7950 and YANG Library, rfc8525 upgrades on the roadmap). It contains the extensible YANG Datastore Server, with datastore extension providers built and instrumented from the source YANG modules, using NuDesign Studio’s code generator ndCONF Builder.
The YANG Datastore Server can be deployed with any combination of NuDesign’s protocol specific Access Agents (NETCONF, CLI, SNMP and / or RESTCONF), providing an OEM device with a well integrated ndCONF Agent for remote configuration and monitoring applications, exposing required multiprotocol interfaces to provisioning and management applications (with future gRPC / gNMI Access Agent enabling telemetry monitoring applications).
The ndCONF’s product architecture and the development roadmap also provide for extensions to NETCONF / YANG standards, such as support for NETCONF publish / subscribe push technologies for Telemetry and Monitoring applications, as well as support of automated delivery of configuration data via implementation of Call Home technology.
NuDesign’s YANG based tools produce modern management agents that can deploy any YANG model, standard or vendor’s based: IETF’s, OpenConfig, other standard bodies, while enabling additions of vendor-specific features via their own sub-agents or via YANG permitted augmentations of standard models.
The approach aims to protect the customer’s management software investment, it secures continuity and integration between generations of management applications and networking product families, it provides for co-existence and transition of existing CLI configurations and SNMP monitoring infrastructure to a new generation of management applications that support NETCONF / RESTCONF / gRPC YANG based automated configuration and operational processes, regardless of origins of their networking gear.
The company also continues evolving its present secure SNMPv3 products for Windows and Linux Servers and embedded devices for use in present server and embedded markets.
* gRPC / gNMI / gNOI are a subject of an internal engineering project, being evaluated for their applicability to configuration applications and Streaming Telemetry, operational network data gathering and monitoring for various analytics applications.
Learn more…
To review and for selection of NuDesign’s YANG based NETConf, SNMP, CLI & RESTCONF development tools please review NETCONF & YANG Development information.
To review and for selection of NuDesign’s MIB based SNMP & CLI development tools please review SNMP & CLI Development information.
