Multimedia Processor Design – From Black and White TV sets to Virtual Reality… and beyond

INTRODUCTION

I was watching an episode of ‘The Crown’ last week on Netflix. One of the conspicuous period ‘punctuation marks’ I noticed was the antiquated, small black and white TV they wheeled out at Buckingham Palace to watch the Coronation broadcast. How far we have come in a few decades.

A-family-sits-in-their-living-room-watching-a-childrens-television-programme.jpg

Today, technological advances in how we deliver visual content, and how we view it, has transported us light years away from that black and white TV set of the 50s. The acceleration of this technology in the last decade has been fuelled by social networking platforms and all day connectivity to name two significant examples. This in turn has created a demand for products with improved graphics, video and audio, in a smaller portable product that has a longer battery life and the ability to support multiple applications.

 

gpu image within a pc.jpg

 

At Sondrel we have worked on graphics and multimedia projects for many years – though not quite as far back as the 50s’! -  and work with Arm as an Approved Design Partner. The relentless advance in this sector continues. Our smart phones , smart TVs, gaming consoles and tablets have quickly been absorbed into the mainstream – ‘The Norm’. Application solutions continue to develop and flourish in emerging sectors – significantly automotive right now, medical, and with new user experiences supported by Virtual Reality and Augmented Reality. I was listening to an interview on the radio this morning with Shigeru Miyamoto, a legendary gaming developer, discussing technology being developed that will enable you to step into a TV programme or a game.

shutterstock_445344292 (1).jpg

So what has this meant for the engineers designing the chips that make all this possible? Chatting about this to some of our designers over the coffee machine, and getting beyond the immediate response of ‘it’s a nightmare’, they agreed that the development of SoC design for multimedia has progressed as a result of:

  • More computational blocks
  • Faster, more hierarchical memory capacity
  • GPUs, for example the ARM Mali or Imagination PowerVR cores  

The consensus amongst SoC designers is that the implementation of a multimedia SoC is amongst the most challenging designs to tackle. To process signal information that is 2 or 3 dimensional, requires a high amount of concurrent processing over multiple, often heterogeneous compute resources and complex on-chip interconnects that provide the necessary memory bandwidth to maximize the utilization of the available resources. This means high gate counts, high clock rates and complex signal routing. New products and solutions demand progress in the same basic key elements from customers – Power, Performance and Area. It needs to deliver more functionality, it needs to use less power whilst minimizing cost. The PPA targets are hard to achieve with graphics projects when you need such high computational performance, to be delivered using the lowest power and on the smallest possible footprint.

This table and illustration are an example of a graphics processor engine that a Sondrel team worked on, and clearly shows the specification evolving over a short period of time.

^B6BE0E2A6C6AD2158A4BB68E38F15250846B043B204754E86C^pimgpsh_fullsize_distr.jpg

Illustration of version 2

Year

2012

2014

2016

Process Technology

65nm

28nm

16nm

Power

20W

40W

60W

Instances

40M

80M

160M

Clock

400MHz

800MHz

1.2GHz

 

You can download a case study of this Sondrel project engagement, looking at version 2 of the design above. This processing engine was built to handle the latest encryption standards for 4K HD display broadcasts. It was a 28nm TSMC design. We worked with the customer on the project from RTL2GDSII, some consultation through to silicon, and we handled the DfT requirements.

The design techniques involved in these more complex SoCs, such as the example above, necessarily introduces significant design challenges.

ARM’s development of the Mali media processors reflects the progress of IP to deliver improved graphics and multimedia solutions. The specialist cores within the range reflect the key PPA priorities, for High Performance, High Area Efficiency and Ultra Low Power. Recognising the difficulty in SoC design for multimedia applications, even with better GPU solutions, they have developed big.LITTLE processing - a power management technology employed by ARM to enable CPU’s that work together with these GPU’s to deliver peak-performance capacity, higher sustained performance, and increased parallel processing performance, at significantly lower average power. To utlilise these technologies requires multiple heterogeneous processing units which must connect with dynamic memories through complex on-chip interconnects, with wide buses, and DDR or HPM interfaces, all of which can present significant challenges to SoC physical designers.


CONCLUSION

Up to this point, Moore’s law has ensured that the compute power to support progress in delivering high performance multimedia SoCs has been possible - with the development of the tools, IP  - and blood sweat and tears of the design engineers.

At the turn of the century a chip would typically be made up of a CPU, DSP, some memory, some connectivity and some additional differentiating IP blocks . Now you will encounter multiple CPUs, GPUs, video processors, MMUs, cache coherent NoCs and high speed memory interfaces on SoCs.

I look forward to seeing the products that will be coming to market on the back of the technologies that we are currently working on. Within the next 5 years, will stepping into another reality become the new 'norm'?...

Sondrel engineers have been engaged in many projects involving the implementation of graphics and multimedia solutions. We are an accredited ARM approved Design Partner. Contact us for more information on design services support for this application area.