Entertainment, Graphics, Software Programs

OpenVX The New Khronos API for Computer Vision and AR

OpenVX is the Khronos Group’s newest standard and it looks to fill the gap between OpenCV and OpenCL in order to improve the flow of visual data to the compute parts of the GPU. The goal of OpenVX is to delivery real-time power efficient vision processing with a focus on mobile and embedded applications. There will also be a broad array of supported implementations from FPGAs to GPUs to dedicated hardware pipelines through a tightly specced API.

As you can tell from the slide above, OpenVX designed to improve the efficiency of computer vision from the camera to the compute, which can easily be handled by OpenCL 2.0, the new open heterogeneous compute standard that was finalized at the same time as OpenVX was announced.

OpenVX’s functions will be to work with the core data structures of images and image pyramids as well as processing graphs, kernels and parameters. In addition to that, it will do a whole host of image processing which range from mulitichannel color and BitDepth extraction and conversion to image resizing and warping. It will also do some 2D filtering and morphological operations in addition to the standard arithmetic, logical and statistical operations. It will also do core computer vision including pyramid computation and integral image computation. In addition to these major features, it will also be capable of histogram computation and equalization as well as canny edge detection and sparse optical flow. Harris and FAST corner detection are the final features of what OpenVX is epxected to do, keeping in mind that it is not finalized as a standard quite yet.

Many people will consider that OpenVX is a competitor to OpenCL and OpenCV, however the real truth is that OpenVX appears to be an API specifically designed for very focused applications. OpenVX is designed to narrow down the parameters and improve the efficiency of computer vision and deliver a more finalized experience. OpenVX is designed to be hardware abstracted unlike OpenCV and will have rigid specifications and qualifications that will help make computer vision more prominent. Below, we’ve got two slides that Khronos has supplied us with that compare and contrast OpenCV against OpenVX as well as OpenVX against OpenCL. Keep in mind that OpenCL is Khronos’ own standard and that they aren’t interested in seeing it disappear from the computer vision world.

In the end, a lot of this is being done to resolve a problem with computer vision, which is the fact that it isn’t quick enough and it is generally too costly in terms of power. You can’t have it always on because the computation that is necessary results in too many GPU cycles. Khronos believes that they can help solve the AR problem with their APIs in multiple ways that will ultimately result in better, faster, smarter AR.

The companies currently participating in the OpenVX group are Aptina, ARM, Axis, Broadcom, BDTi, CEVA, Cognivue, Intel, itseez, Movidius, Nvidia, Qualcomm, Renesas, Samsung, STMicro, Texasi Instruments, Xilinx, Videantis and Vivante. Itseez is the working group chair and Qualcomm and Ti are specification editors. Based on this, you can see that one of the biggest implementors of OpenCV, itseez is chairing the working group, which means that they believe that OpenVX is necessary to improve upon the current state of computer vision. We also have both Qualcomm and Texas Instruments, each company representing a large chunk of mobile and embedded. Qualcomm’s mobile share is a significant one, especially when you count how many of their own GPUs they have in there and how much computer vision they have already done. And TI is clearly their partner for the embedded FPGA space. What is interesting is that Alterra is not part of the working group nor is Apple, which would pretty much lock up the majority of both the embedded and mobile industries major players.

They are currently aiming to finalize the specification by mid 2014, and hopefully we will see it shipping in products sometime later that year or in early 2015. After all, anyone half interested in AR knows that we need better performance and efficiency with computer vision, especially if we want wearables.