Onboard AI also means you can get information processed in real-time. A camera at the front of a store could count the number of people entering, and even alert security if someone isn’t wearing a face mask (a major health risk as we’re still dealing with the COVID-19 pandemic). Another camera could keep track of item stock on store shelves, while others could monitor the flow of foot traffic to determine shopping “hot spots.”

To be clear, these image sensors won’t replace the work of sophisticated cloud-based machine learning tools, which can go far beyond just recognizing objects. But they’re a step towards a future where cameras can function more like self-contained computers.

And they can also assist more complex computer vision systems. For example, in a cashier-less store like Amazon’s, where tons of video data is being captured and processed by a slew of cameras, Sony’s sensors could detect exactly where customers are located. If nobody is by the milk aisle, there’s no reason to flip those 3D cameras on, right?

Hanson tells us samples for the IMX500 have already been sent out to Sony’s partners, while the IMX501, which is packaged to fit onto devices like smart boards, will start shipping in June. We can expect to see devices using the new hardware by the end of the year and throughout 2021. 

The first three areas it’s focusing on are gaming, imaging & sensing, and gastronomy. As for that last one, a concept video Sony released last year proposed the idea of “robot gastronomomy” that relied on autonomous tech to cook food at home or even on a night out. Sony AI global head Hiroaki Kitano previously led a cooking robot collaboration with Carnegie Mellon University, and it will be interesting to see what it develops there.

The application of advanced AI has shown up in products like the A6600 camera, where it contributes to an impressive improvement in autofocus, while opportunities for machine learning and AI in gaming are obvious, when it’s applied to improving cloud gaming or just making enemy characters a bit smarter.