AI-based image processing without prior knowledge

Deep learning opens up new fields of application for industrial image processing, which previously could only be solved with great effort or not at all. But the fundamentally different approach to classical image processing also causes new challenges. Therefore, IDS presents an all-in-one embedded vision solution. Thanks to IDS NXT ocean, every user can implement AI-based image processing in just a few steps without the need for programming knowledge.

Which skills are needed to bring machine learning into your company? Breaking down the usual development of an AI application into individual steps actually reveals tasks and concepts that are completely different compared to the classical approach of rule-based image processing. The handling and preparation of image data as well as training neural networks require new tools and development frameworks, which must be installed and executed on a suitable PC infrastructure. Although necessary instructions and open source software are usually freely available from cloud providers or on platforms such as Github, they only provide the fundamental basic tools which require a high degree of experience. And not only the creation, but also the execution and evaluation of the training results on an appropriate hardware platform requires understanding and knowledge of hardware, software and their interfaces.

Figure 1 IDS NXT ocean lowers the entry barrier with easy-to-use tools.

With the all-in-one AI vision solution IDS NXT ocean, IDS supports the user right from the first steps with the new technology. The company lowers the entry barrier and provides easy-to-use tools for creating inference tasks in a few minutes without prior knowledge.

The concept is based on three essential components:

  • An easy to use training software for neural networks
  • and an intelligent camera platform 
  • including an AI accelerator that executes the neural networks on the hardware side.

All components have been developed by IDS and are designed to work seamlessly together. The cloud-based training software IDS NXT lighthouse leads step-by-step through data preparation to the training of artificial intelligence in the form of a neural network. As a web application, IDS NXT lighthouse is immediately ready for use. The user has sufficient storage space and training performance for all his projects in an easy-to-use workflow.

The manufacturer uses supervised learning at IDS NXT lighthouse to train neural networks. The Deep Learning algorithms learn with predefined pairs of inputs and outputs. The teacher – in this case the user – has to provide the correct function value for an input during learning by assigning the correct class to a picture example. The network is trained to be able to make associations independently by making predictions regarding image data in the form of percentages. The higher the value, the more accurate and reliable the prediction.

The seamless interaction of the software with the IDS NXT camera families rio & rome ensures a quick success. Fully trained neural networks can be uploaded and executed directly and without programming effort on these cameras. Thus, the user immediately has a completely working embedded vision system that sees, recognizes and derives results from captured image data. With its digital interfaces, even machines can be controlled directly.

Figure 2 Seamless interaction of software and hardware for quick success

Embedded Vision Hybrid System

IDS has developed its own AI core for the FPGA of the intelligent IDS NXT camera platform called “deep ocean core”, which executes pre-trained neural networks hardware-accelerated. This turns the full-fledged industrial cameras into high-performance inference cameras that make artificial intelligence useful in industrial environments. Image analysis is performed decentralized, avoiding bandwidth bottlenecks during transmission. Cameras based on the IDS NXT platform can thus keep pace with modern desktop CPUs in terms of accuracy and speed of results – with significantly less space and energy consumption at the same time.

The perfect adaptation of IDS’ own software and hardware allows the user to choose the target inference time before training. IDS NXT lighthouse then ensures optimal training settings while considering the AI core performance of the camera. Thus the user expects no surprises during the subsequent execution of the inference, eliminating the need for time-consuming readjustment and re-training. Once integrated, the IDS NXT system remains 100% compatible and consistent in its behavior for the user. Especially with industrially certified applications this is a significant advantage.

Figure 3 The functionality of IDS NXT inference cameras will be extendable with apps and CNNs as required.

Due to the powerful hardware, the Embedded Vision platform is much more than just an inference camera used to execute neural networks. The feature set of the CPU-FPGA combination will be extendable by the user according to his needs in the next development step using vision apps. Recurring vision tasks can then be set up and changed quickly. Even a completely flexible image processing sequence can then be realized. Captured images are first preprocessed, for example, before a quite simple and fast classification sorts good and bad parts. If errors occur, a much more complex neural network can be reloaded in milliseconds to determine the error class in much more detail and transfer the results to a database. Customized solutions can then be easily implemented using an app development kit. Users can then create their own individual vision apps in just a few steps and install and run them on IDS NXT cameras.


Where rule-based machine vision has not been attempted or has reached its limits, there is a high potential for deep learning algorithms to support employees and drive forward automation. AI solutions usually require specialist knowledge, development effort, and investment in computing and data storage hardware. With the rise of cloud-based computing and dedicated training services, however, deep learning becomes more and more accessible. Lowering this threshold is the main focus of the new all-in-one AI vision system IDS NXT ocean. It requires neither special knowledge in deep learning nor camera programming in order to create and execute individual neural networks. Users only need to provide sample images and knowledge on how to evaluate them (e.g. “good” / “bad”). This makes the start into AI-based image processing particularly quick, easy and user-friendly.

More information:


You may also like...