Sony today unveiled the world’s very first image sensors that have artificial intelligence capabilities built-in. The sensors are capable of processing captured images and extracting intelligent metadata at high speeds.
The upcoming IMX500 and IMX501 intelligent vision sensors can be paired with cloud services, and having built-in AI “reduces data transmission latency, minimizes any privacy concerns, and reduces power consumption and communication costs,” Sony says.
The IMX500 and IMX501 are backside-illuminated 1/2.3-inch sensors that have resolutions of approximately 12.3 megapixels and 4K/60fps video capabilities, and they’re expected to be priced at ¥10,000 and ¥20,000 ($93 and $187), respectively.
The secret to the sensors is a new “stacked” design that combines a pixel chip with a logic chip, the latter of which has AI image analysis and processing capabilities. After light is captured by the pixel chip, it’s processed immediately by the AI on the logic chip.
The sensor is capable of outputting both image and metadata together or only the metadata if that’s the only thing you need.
“The sensor outputs metadata (semantic information belonging to image data) instead of image information, making for reduced data volume and minimizing any privacy concerns,” Sony says. “Moreover, the AI capability makes it possible to deliver diverse functionality for versatile applications, such as real-time object tracking with high-speed AI processing […]
“[A] single type of camera can be used with versatility across different locations, circumstances, times, or purposes. When installed at the entrance to the facility it can be used to count the number of visitors entering the facility; when installed on the shelf of a store it can be used to detect stock shortages; when on the ceiling it can be used for heat mapping store visitors (detecting locations where many people gather), and the like.”
But even though the sensors are currently geared toward industrial and retail applications, it’s easy to see how such a sensor could be interesting for still photography cameras of the future — it’s a sensor that can understand that you just photographed a dog.
No word on if or when Sony may introduce AI-capable image sensors to its popular consumer camera lines, but the IMX500 has already begun shipping to Sony’s industry customers and the IMX501 is set to begin shipping next month, so this technology will likely see real-world use very, very shortly.
Image credits: Header illustration includes AI illustration by Mike MacKenzie.