:klwh 3dshu 1hwzrun3urfhvvru3urjudpplqj0rghov 7kh.h\wr$fklhylqj)dvwhu7lphwr0dunhw dqg([whqglqj3urgxfw/lih the design of the network equipment powering the internet revolution has undergone profound changes over the last decade. today, with network equipment vendors racing to provide the new converged voice/video/data communications infrastructure, designers require both speed and flexibility to deliver within the highest time-to-market pressures the industry has ever seen. powerful new network processors are challenging traditional network device design methodolo- gies by enabling software implementations of virtually all key communications functions at hard- ware speeds. key to this revolution is the programming models that enable designers to implement the communications processing tasks on these processors. this paper explores these models, and their effects on delivering on the promise of a new and better network device design process. ,qgxvwu\,pshudwlyhv 7lphwr0dunhw$qg7lphlq0dunhw just as the internet revolution is forever changing the face of public communication networks, the way products that make up these networks are designed is also changing. network equip- ment developers have consistently faced a difficult trade-off: performance requirements demand hardware implementations of data forwarding functions, while new features, such as advanced quality of service (qos), require flexibility that only software can deliver. designers have been forced to revisit the fundamental hardware/ software trade-off with each new product (or even line card) they develop, sacrificing software reuse between product lines and product generations along the way. the result has been longer time-to-market, higher develop- ment costs, and shorter product lifetimes. companies trying to compete in internet time can no longer afford this type of product development. the network processor, a new type of semiconductor device, is changing the dynamics of the speed versus flexibility trade-off by enabling virtually all communications functions to be soft- ware programmable without sacrificing hardware speeds. these processors eliminate the high-risk, long development cycles of custom hardware by enabling advanced product features to be delivered completely in software, even long after initial product introduction. this allows network equipment vendors to concentrate precious development resources on delivering advanced services to their customers, rather than just the latest feeds and speeds . the best network processors form the foundation of a communications platform that contains the key elements required to radically transform the network device design process. for example, motorolas smart networks platform combines advanced network processor tech- nology, standard programming interfaces, communications software components (from c-port and motorola alliances) and a comprehensive development environment. this enables network equipment vendors to quickly bring to market a wide array of different products based on the same hardware and software architecture. the result is significantly faster time-to- market for new products, and dramatically longer time-in-market (through the use of software upgrades to deliver new, advanced services that extend the product life cycle). see figure 1. )ljxuh asics versus network processors product life cycles point product world (asics) open platform world (network processors) point product development point product lifetime product develop. open platform product lifetime s/w s/w s/w s/w s/w time time %\ 'dylg+xvdn &3ruw&irxqghudqg &klhi7hfkqlfdo2iilfhu dqg 5rehuw*rkq &3ruw9lfh3uhvlghqw 0dunhwlqj f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
2 but not all network processor architectures can support the platform model. the communications platform requires more than a reasonable merchant silicon point-product alternative to asic design. with so much of the platform value riding on the programmability of the devices, the network processor programming model is a key metric by which these solutions must be evaluated. 7kh1dwxuhri&rppxqlfdwlrqv3urfhvvlqj7dvnv to evaluate network processor programming models, the nature of the tasks to be programmed must be understood. there are two broad categories of communications tasks (see figure 2): ? )ruzduglqj3odqhwdvnv consisting of operations on forwarding path communications data that occur in real-time. these constitute the core device operations, and hence are performance critical. in a switch or router, these are the functions that receive, process, and transmit packets into and out of the device. ? &rqwuro3odqhwdvnv consisting of less time-critical control and management functions that determine general device operation. in a switch or router, these functions control routing table maintenance, port states, and higher-level management. in traditional designs, the forwarding plane functions are divided between fixed-function hardware (usually custom asics) and software running on a general-purpose cpu. control plane functions are implemented in software either on the same cpu or another, dedicated host cpu. )ljxuh communications processing tasks policy applications network management signaling topology management queuing / scheduling data transformation classification data parsing media access control physical layer forwarding plane control plane network processors are specifically designed to bring program- mability to the forwarding plane functions (layer 2 and higher of the iso model) required by the lan and wan devices that make up todays networks. these forwarding functions include: ? 0hglddffhvvfrqwuro implementation of low-layer protocols, such as ethernet, sonet framing, atm cell processing, and so on. these protocols define how the data is represented on the communications channel, and the rules governing how that channel is accessed. paradoxically, this is the area of the greatest standardization among network devices (due to standards-based protocol definitions), and also the area of greatest diversity (due to the wide and ever growing variety of protocols). these include: ethernet (with three different flavors at 10mbps, 100mbps and 1000mbps), sonet supporting both data packets and atm cells at a wide range of standard rates (oc-3, oc-12 oc-48, and so on), legacy t/e-carrier interfaces from the existing public voice infrastructure, and a variety of emerging optical interfaces all must coexist and interact. ? 'dwdsduvlqj parsing cell or packet headers containing addresses, protocol information, and so on. in the past, parsing functions were fixed based on the type of device being constructed (for example, lan bridges, by definition, only needed to look at the layer 2 ethernet header). today, switching devices need the flexibility to gain access to and examine a wide variety of information at all layers of the iso model in real time and on a conditional packet-by-packet basis. ? &odvvlilfdwlrq identifying a packet or cell against a set of criteria defined at layers 2, 3, 4, or higher of the iso model. once data is parsed, it must be classified in order to determine the required action. actions might include such basic functions as a filtering/forwarding decision, as well as advanced qos and accounting functions based on a specific end-to-end traffic flow. this is an area of rapidly changing requirements. ? 'dwdwudqvirupdwlrq modification or translation of data within or between protocols. the variety of low-layer transport protocols is matched only by the diversity of protocol combinations and services. transformation requirements can range from address translation within a given protocol (such as ip) to full protocol encapsulation or conversion (such as between ip and atm). ? 7udiilfpdqdjhphqw including the queuing, policing, and scheduling of data traffic through the device according to defined qos parameters, based on the results of classification and established policies. these functions are key to supporting convergence of voice, video, and data in next-generation networks. f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
may 2, 2001 network processor programming model choices 3 today, each of these functions presents the challenge of a wide diversity of possible implementations, rapid evolution based on continuing innovation, strong interdependencies between func- tions, and a need for interworking between the diverse proto- cols. delivering programmability and integration of these functions represents a major evolution in network device design. 1hwzrun3urfhvvru3urjudpplqj0rgho &krlfhv the computing world has always debated about what is the best processor hardware architectures: cisc versus risc, single cpu versus multi-cpu, coprocessors versus faster clocks, and so on. however, it is the software that determines the success of computing platforms, both in terms of perfor- mance and programming ease. the limited success of symmetric, parallel computing architectures proved that raw computing power was not the decisive factor, but rather how that power could be harnessed by software . the same is true for network processors the decisive factor is how the programming model serves the platform requirements of fast, simple, and flexible programmability. there are two primary metrics for evaluating network processor programming models. the first is the level of programmability offered, both in terms of which functions can be programmed (see figure 2), as well as the extent that these various func- tions can be programmed. while the physical space, cost, and power benefits of high functional integration into a single processor is well understood, there is a forgotten benefit to the programming model. processor architectures that assume a bag of parts approach provide programmability for a subset of the forwarding plane functions, limiting the ability of programmers to effectively deal with the diversity within each level and the often complex interactions between them. like- wise, providing appropriate programmability within each level is crucial to accommodating these interactions. hence a fully inte- grated, fully programmable network processor architecture is a major prerequisite for an effective programming model. the second, and most important metric, is the actual program- ming method for the processor. perhaps the largest struggle in traditional network system design with asics has been a hardware first architectural mentality, with software engi- neers designing around a less-than-ideal hardware/software partitioning. network processor programming models need to turn this around, providing a hardware processor platform that serves the requirements of the software functions and, in the end, the software designers themselves. the key criteria is a simple programming paradigm, using well known methods, without sacrificing product performance. the network processors available today fall into three broad categories of programming methods, with a spectrum of capa- bilities within each. these categories are discussed below. 0lfurfrgh(qjlqh3urjudpplqj these devices implement virtually all the forwarding plane func- tions in custom designed, low-level microcode machines. all tasks including data parsing, search algorithms, data transfor- mation, queuing, and scheduling algorithms must be specifi- cally programmed by the designer. these machines maximize performance through multi-threading, which generally requires the microcode writer to consider everything from memory access times to thread interactions when optimizing each func- tion. also, these architectures implement multiple instances (typi- cally 6 to12) of these machines in parallel, with fixed hardware schedulers assigning incoming data to a given machine based on availability. this is similar to traditional symmetric multipro- cessing computing models, which places additional constraints on the microcode writer to assure proper interactions and ordering of forwarded data. microcodes strength lies in the efficiency of the code once it is written. the code can be compact and fast. however, the downside of programming in low level microcode, compre- hending everything from memory access latencies to multipro- cessing dependencies, is the lack of portability of these code designs to other products based on same processor and to new, faster versions or new generations of processors. this code tends to be one time use , which when combined with the inherent difficulty of writing microcode, might make these processors suitable for point product designs, but seriously compromises the value of software programmability for an overall communications platform. */3urjudpplqj another programming model focuses on leveraging proprietary search and pattern-matching algorithms to the communications processing task, specifically for parsing and classification. a number of these algorithms use custom fourth generation languages (4gls) to describe the parsing and pattern-matching requirements for a number of applications. these 4gls provide a concise method of programming the classification function, and processors that implement these algorithms provide a partial solution to this piece of the commu- nications processing task. f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
4 the algorithms implemented by these processors typically trade-off memory size for search speed, which may or may not be an issue for the system design. there are, however, larger impacts on the programming model against the two main criteria outlined above. first, these processors focus almost exclusively on the parsing and classification tasks, providing only one piece of a bag of parts solution. the designer must either build the required external hardware (and associated soft- ware) around this part, or, if available, use other piece parts provided by the processor vendor (sometimes configurable with microcode as described above). in either case, the programming domain is disjoint, compromising what functions are actually programmable and the depth of that programma- bility. even if the other functions are ignored, using a proprietary description language for the classification requires new skills and tools, not just for the coding tasks but for debug, analysis, and maintenance. good tools can mitigate some of this cost, but the inconsistency between the other forwarding plane func- tions and the control plane functions will remain. 6wdqgdug/dqjxdjh3urjudpplqj the standard language programming model leverages existing languages (such as c and c++ with their inherent benefits such as readily available skilled programmers and industry standard programming tools), usually combined with special coprocessors, to implement the various communica- tions processing tasks. these use multiple embedded risc cores as a key processing element to support the execution of standard c/c++ programs imple- menting the desired behavior. note that the use of risc cores in a network processor does not auto- matically mean that the processor was designed to support a higher-level programming language paradigm. many risc-based network processors implement proprietary instruction sets (or proprietary extensions), which, while expedient from a hardware design perspective, force program- mers to write all or significant portions of their code in risc assembly language. similarly, the processing capacity may not be adequate to support reasonable implementations in a higher-level language. thus, programming these processors can be just as complex as writing in low-level microcode. to effectively support the requirements of a communications platform, the programming model must support the ability to write effective programs in a higher-level standard language. the means the risc cores need to have enough horsepower within a rich coprocessing architecture to support an api abstraction layer that insulates the operating code from low level chip implementation details without sacrificing perfor- mance. this is the key to providing a simple programming model environment and extending the life of the software. the implementation of the coprocessing architecture is critical, as the coprocessors must off-load the risc processor from the communications tasks that are notoriously poor in standard cpus (such as the bit manipulation typically required in parsing and data transformation tasks). 7kh)rxqgdwlrq $1hwzrun3urfhvvru'hvljqhgiru &rppxqlfdwlrqv7dvnv c-ports c-5 network processor (np) is an example of a network processor designed from the ground up to provide a simple and robust programming model. the c-5 np provides complete programmability for each of the forwarding plane tasks using standard c/c++ programming, enabling universal applications in a wide variety of network devices. the c-5 np combines multiple risc cores, specialized coprocessors, and microcode engines within a single integrated circuit to offer a full range of programmability at high performance. figure 3 shows a block diagram of the c-5 np . )ljxuh c-5 np software-optimized architecture phy phy phy phy phy phy phy phy buses (60gpbs bandwidth) cluster cluster cp-1 cp-0 cp-2 cp-3 cp-12 cp-13 cp-14 cp-15 external host cpu (optional) oc-3 phy interface examples: oc-12 oc-48 gigabit ethernet 10/100 ethernet external prom (optional) control logic (optional) fabric processor boundary buffer mgmt unit channel processors sdram c-5 np sram sram table lookup unit fabric processor executive processor queue mgmt unit pci serial prom channel processor 32-bit risc core serial data proc serial data proc f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
may 2, 2001 making programming more simple through a communications api 5 &kdqqho3urfhvvruv )oh[leoh%xloglqj%orfnv the fundamental building blocks of the c-5 np are the 16 embedded channel processors (cps). each cp consists of a dedicated risc cpu and dual serial data processors (sdps). the cp structure combines the best attributes of specialized configurable state-machine architectures with a fully program- mable risc core. cps can be assigned to physical interfaces, aggregated together to support higher-bandwidth i/o streams, or assigned internally as a dedicated internal coprocessor. the sdps handle data encoding/decoding, framing, formatting, parsing, error checking (crcs), and data movement. the sdps also control programmable external pin logic, allowing them to implement virtually any layer 1 interface including connection to t/e-carrier framers, 10/100 ethernet phy (rmii), gigabit ethernet phy (gmii or tbi), oc-3 phy, oc-12 phy, and oc-48 framers/phy. at layer 2, the sdps can be independently config- ured to support ethernet, pos, hdlc streams, atm, frame relay, fibrechannel, or virtually any format including various encapsulations such as mpls. the programmability of the sdps support the diversity of media access control interfaces, as well as first-order parsing requirements, and can support the mix-and-match requirements of different implementations on a port-by-port basis. this efficiently supports the needs of various interworking applications. the sdps are programmed in microcode, which is provided by c-port for the vast majority of applications (all flavors of ethernet, ip and atm over sonet, t/e carrier serial data streams, and so on). all the tools necessary for equipment vendors to program the sdps (including assembler and simu- lator support) are available. support for mac level diversity is available without any user coding. the cps risc core, programmed in c or c++, is available to focus on higher-level tasks such as final switching / forwarding decision making, scheduling, statistics gathering, or other tasks required for higher-level services. the risc core in each cp operates at the core clock rate of the c-5 np , has dedicated internal instruction and data memory, and implements an industry standard instruction subset, avoiding the issues asso- ciated with proprietary instructions. with the sdps off-loading the bit level tasks from the risc core, the capacity of the risc machine can be dedicated to the tasks that benefit the most from high-level language implementations. 'hglfdwhg&rsurfhvvruv the c-5 np also provides five coprocessors optimized for common tasks and used by the cps. these coprocessors handle shared tasks including table lookup, queue manage- ment, buffer management, fabric interfacing, and supervisory processing. each unit is highly configurable and offers perfor- mance and capabilities that, if packaged as stand-alone devices, would be considered best-in-class communications compo- nents. for example, the table lookup unit (tlu) enables a wide range of traffic classification functions and supports multiple, different search algorithms. 'hvljqhgiru+ljkohyho3urjudpplqj the cps, supported by the coprocessors, provide the funda- mental building blocks from which multiple applications can be supported through high-level programming. for example, the cps can take on different personalities to support atm, ethernet/ip , ppp/ip , frame relay, channelized hdlc, or even proprietary protocols through a combination of microcode in the sdps and c/c++ code running on the risc core. the data paths through the cps can be configured for external connec- tion (to phys) or looped back internally, for use as an applica- tions coprocessor . although there are 16 cps per c-5 np , each cp is independently programmable, avoiding the limitations typical of traditional symmetric multiprocessor designs. with the flexibility provided by the cp architectures, it is a straight forward task to write software for the cp to perform a given function. 0dnlqj3urjudpplqj0ruh6lpsoh7kurxjkd |&rppxqlfdwlrqv}$3, hardware flexibility is usually accompanied with complexity driven by the number of possible functional permutations. by adapting the concept of standard application programming interfaces (apis) to communications processing, this complexity can be put at the service of the programmer. the c-5 np supports c-ware application programming interfaces (apis), a set of open, efficient interfaces that abstract common functions from the underlying hardware. see figure 4. f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
6 )ljxuh c-ware apis for programmers accustomed to tweaking hundreds of lines of assembly code to squeeze out the last bit of performance from a cpu, the concept of using an api in forwarding plane code would appear odd. however, the c-5 np computing power (over 3,000 mips total) was sized from the beginning to accommo- date any overhead imposed by an api. this, combined with standard c/c++ programming, is the key to delivering on a simple programming model. in an effort to leverage the power of this concept throughout the industry, a group of network processor, software, and equipment vendors (with c-port, ibm, and lucent as charter members) initiated the common programming interface (cpix) forum (www.cpixforum.org). by defining a common framework and api, network processor vendors and communications soft- ware vendors can offer more portable and flexible solutions for network equipment designers. 3urjudpplqj(qylurqphqw5htxluhphqwv the use of a true communications platform in network device design changes the typical design process. a much larger percentage of the intellectual property of a product is delivered in software, hence the network processor development tools environment is critical to project success. in addition to the basic programming model, other factors influence the speed at which products can be brought to market. these factors include: ? 6riwzduhuhihuhqfhghvljqdydlodelolw\ most network processor vendors provide examples of forwarding plane software for some number of functions. the extent, quality, and breadth of these applications (as well as available implementations from software partners) can help make or break a project schedule. buffer mgmt. unit queue mgmt. unit table lookup unit executive proc. cp15 buffering services queuing services table lookup services fabric services kernel services pdu services protocol services fabric proc. cp1 cp0 c-5 network processor ? 5rexvwvlpxodwlrqhqylurqphqw most network processor vendors provide extensive simulation environments that allow completion of forwarding plane code development and performance characterization before hardware integration. a key differentiator is the speed and accuracy of the network processor simulation. those based on a full software implementation can be as accurate as a hardware model (for example, based on verilog/vhdl models), but orders of magnitude faster, allowing more simulation bandwidth. ? 'hyhorsphqwv\vwhpdydlodelolw\ a hardware development system, offering the ability to execute software on the real network processor, is also generally available from most vendors. while not a replacement for a good simulator (a simulator can always be better instrumented than real hardware), it is invaluable for starting final integration in advance of prototypes. a system that can be assembled to closely match the target system configuration (types of physical interfaces, and so on) is a great asset. ? 2wkhuvriwzduhwrrov software tools, such as compilers, debuggers, performance analyzers, and so on are also key elements of the software development environment. seamless integration of these tools across both the simulation and hardware development platforms is an often overlooked, but important, aspect of accelerating time-to-market. ? +rvwsurfhvvrulqwhjudwlrq as described earlier, the control plane functions are supported in a traditional embedded cpu. the hardware integration of this processor with the network processor is straight forward, but the software integration requires some considerable thought. hence a software and hardware development environment that comprehends the host processor, including drivers for the leading real-time operating systems, host-level apis, and some number of fully integrated applications, should be a key consideration. for example, c-port provides a complete communications development environment, consisting of a full software toolset (including simulator), and a development system. the develop- ment system consists of network processor modules, physical interface modules (for ethernet, gigabit ethernet, oc-3, oc-12, and so on), and a host processor module based on a powerpc cpu running the vxworks rtos. the vast majority of an appli- cation can be integrated and tested prior to integration with the target product hardware design, significantly reducing the time and risks of the product integration phase. f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
may 2, 2001 conclusion 7 &rqfoxvlrq network processors offer a significant opportunity to improve the architecture, design, and maintenance of todays networking devices. the opportunity, however, extends beyond the standard benefits of off-the-shelf merchant silicon. proces- sors that form the foundation of complete communications platforms, based on a simple programming model, promise to radically improve the way networking technology is brought to market. this adds up to better product features, faster time-to-market, and better reliability for network equipment vendors and their customers. f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
c-5, c-port, c-ware, and the c-port logo are all trademarks of c-port corporation. ? 1999, 2000, 2001 c-port corporation pm00wp100 f r e e s c a l e s e m i c o n d u c t o r , i freescale semiconductor, inc. f o r m o r e i n f o r m a t i o n o n t h i s p r o d u c t , g o t o : w w w . f r e e s c a l e . c o m n c . . .
|
