It is extremely hard to design hardware that is capable of actually supporting an HD video call, as it requires a lot of processing power.

HD Video is hard to implement. Especially if you want to use it for real-time bi-directional communications, which are also known as video conferencing. The most frustrating thing about it, I think, is that it is extremely hard to design hardware that is capable of actually supporting an HD video call, as it requires a lot of processing power.This is why I've started a series of blog posts that deals with Designing Hardware (HW) for HD, which is targeted at developers who wish to learn more about how to build an HD video endpoint. In this series, I will try to identify the set of requirements that result from supporting HD video calling and how they affect hardware engineers who need to mix and match existing chips to fit the needs of the software engineers, who will have to develop the application (and codecs).

All that is very interesting, but it does bring about a much more interesting question:

What is my dream chip for HD video calling?

If I could ask chip manufacturers to "do the right thing", and come up with a suitable chip that is reasonably priced and supports video conferencing in a way that will allow an HD video terminal to cost $100 or less, what requirements would I specify for their new design?

A couple of chip vendors actually sat down with me and asked this question, while discussing their future chip roadmap. While I liked some of what I've seen, it still wasn't tailored to fit all of my needs. So for posterity, here is what I want in my dream chip:

Video

Video Codec * The basic processing requirement is encoding 2 streams of 720p@30fps and decoding 1 stream of 720p30fps; or encoding 1 stream of 720p@30fps and decoding 2 streams of 720p30fps.

* Better yet would be support for encoding 2 streams of 720p@60fps and decoding 1 stream of 720p60fps; or encoding 2 streams of 1080p@30fps and decoding 1 stream of 1080p30fps.

* The video codec should support H.264 baseline profile. If this is done through hardware acceleration, then I'd like to get access to the accelerators directly--not only have a ready-made H.264 codec as a closed box to use--it won't work for video conferencing.

* SVC support. It should be part of what the hardware accelerators should be capable of doing, and it is where video coding is going to next.

Video Processing * I need 4 scalers to work in parallel on different video streams. They should allow me to upscale and downscale the streams at reasonable quality (a Bi-cubic filter or Polyphase filter would be nice). They should be able to do it for 1080p@30fps or 720p@60fps content.

* Be able to layout the 4 video streams to a single display, or to two separate displays that show different content. It would also be nice if the video planes used can support alpha blending.

Audio * The chip should be capable of supporting audio processing in parallel to all the video processing it needs to do. I am not talking only about the audio codec, but also all of the 3-letter algorithm functions that are necessary for high quality voice applications: VAD, AGC, AEC, etc.

* Might be nice to support Stereo inputs, outputs and processing.

Peripherals

* I need at least 4 video ports for my inputs and outputs. They should be powerful enough to fit up to 1080p60fps each.

* The inputs and outputs should be able to make RGB/YUV and color space conversions "on the fly."

* HD Camera suitable for video conferencing as part of the reference design. This can greatly reduce the effort and risk of a developer.

* Built-in USB stack that can be used to connect headsets and whatnot. * WiFi isn't mandatory, but a nice-to-have requirement.

* 1 Gigabit Ethernet. While 100 Megabit is the norm these days, the migration towards 1 Gigabit is underway, so it's better to have it on such a new chip from day 1.

Visualization * An OSD (On-Screen Display) should be supported on top of all those video streams. It should have at least 16bit of color range, though 24bit would be nicer.

* The OSD should support alpha blending as well. Preferably on the pixel level.

3D graphics should be supported--we're in the iPhone era after all. I guess this will also require OpenGL to be supported.

Host * Embedded Linux. It can be MontaVista, uClinux, Android--whatever. Just don't make it a proprietary Linux version, that has only 20% of the drivers available and the rest are up to the customer.

* Android OS gets extra credit, as it gives a solid solution for a communication device. At the very least have Qt there from day one with good code samples to show how to use it on the chip.

System * Low power. People will want to integrate this chip into consumer electronics.

* Good development and debugging tools. Some chips out there have such lousy tools, that they shadow the chip's capabilities.

Have I missed anything?

Is it too much to ask from a modern multimedia chip?It is extremely hard to design hardware that is capable of actually supporting an HD video call, as it requires a lot of processing power.