Tutorial IND1A: NeXperia - A Versatile Configurable Platform for Home and Mobile Computing

With the advent of the SoC, the concept of a semiconductor platform has undergone a sea change. More and more product functionality keeps continuously getting integrated into these SoCs, so that the multimedia platform begins to look more and more like an abridged version of the product. Different product segments have different requirements from the multimedia platform. Some, like digital TVs, require a level of integration that includes middleware, and frequently GUI customization tools. Others, like mobile or personal multimedia products, demand a somewhat lower level of integration where audio-video streaming use-cases can be put together using streaming components compliant to a streaming architecture. These requirements have led to the distinction of Nexperia multimedia platforms for home and mobile applications, aligned with the product roadmaps of the respective domains. In this tutorial, the audio-video streaming platforms for these two domains will be described in detail. The "SoC" platforms in question have the same external look and feel in terms of programmable interfaces, but there is considerable scope of diversity, both in terms of hardware and software features internal to the platform. Frequently tool support to users includes enablers for integrating robust multimedia use-cases at a high level. In terms of hardware, a multimedia platform typically includes a RISC control processor that runs the SoC-level middleware and user interfaces and a set of DSP cores, hardware accelerators and peripherals that provide for audio-video sourcing and playback. Access to external SDRAM through a fast bus is assumed. The Nexperia home multimedia platform provides for the programming of a set of streaming use-cases, where the functionality spanning the underlying hardware is seamlessly integrated to implement the requested playback or recording operation. From the user perspective, the visibility into the platform is limited to large-grained logical components thr- - ough an interface such as that specified in UHAPI. This level of user specification makes it possible to provide for robust virtual machines for audio-video playback that covers the vast majority of corner cases. Audio-video synchronization is also included implicitly into the use-cases. Internally, the platform uses streaming architectures like TSSA on the DSP processors and exports COM-like interfaces to the control processor. The Nexperia mobile multimedia platform follows an overall internal architecture similar to that for home, but primarily differs in its preoccupation with minimizing power consumption and its component-level user interfaces. OpenMax IL is the preferred user interface model for mobile multimedia platforms. This choice enables the user greater flexibility in devising audio-video use-cases, and to integrate their choice of differentiating software algorithms into the streaming chain