SBB Cargo

Wagon inspection with artificial intelligence

Quality inspectors for freight wagons can now rely on modern technology like image recognition and artificial intelligence to assure proper maintenance.

About the project

The job of a freight train inspector is to physically walk around every wagon of a train and check, if there are any broken or used components that might cause a risk or need to be maintained. This new software transfers ultra high resolution images of freight wagons directly from the field into the office and allows a remote inspection. The inspection process is further supported by artificial intelligence, that analyses all images and visually highlights any components that are not within the tolerance range.
An inspector on the railroads observing a passing trainAn inspector on the railroads checking a cargo wagon
© Roland Fischer, Zürich (Switzerland)
The general concept is designed for two screens:
One is used for meta information, such as wagon identifiers, dimensions or annotated components.
Screenshot of schematic wagon with meta information
Screenshot of train list with modal filter overlay
Screenshot of the annotation cockpit
Screenshot of review queues for AI training
The second screen displays images, tools for image analysis and components which have been annotated by the machine learning model (artificial intelligence). This screen is vertically oriented with three images above each other. This reflects the position of the three different cameras, that are vertically stacked in the real world. This allows the inspectors to process all three images simultaneously:
Screenshot of wagon photos stacked vertically
Screenshot of wagon photos stacked vertically with active component bounding box
Screenshot of wagon photos stacked vertically with active warning popup
The content of both screens is synced. This way we make sure, that the user always sees the contextually relevant information.

Our efforts

From the very beginning of this project, we had the possibility to work closely with the train inspectors. Together we created and discussed the general concept of the software as well as the detailed interaction patterns. We followed a user centered design approach including workshops with various key users, and thorough user testings. Starting with first sketches, we then transferred these ideas to a visually appealing user interface, based on the SBB web design guidelines, which we needed to adapt and extend for the specific use within desktop applications.

Challenges

  • Transform the AI data into a visually processable representation
  • Addressing the very specific user needs with perfectly tailored solutions
Die dieser Anwendung zugrunde liegende Software (inkl. Quell- und Maschinencodes) ist urheberrechtlich geschützt und gehört ausschliesslich und umfassend der Schweizerische Bundesbahnen SBB Cargo AG. Jegliche Art der Verwendung (einschliesslich deren Gebrauch, Veröffentlichung, Reproduktion, Veräusserung, Vermietung etc.) ist nur mit vorheriger schriftlicher Zustimmung der SBB Cargo AG gestattet. © Copyright by SBB Cargo AG, 2018 – alle Rechte vorbehalten.