2.2.3 Virtual reality in training activities: the case of the Automotive sector

1. INTRODUCTION

Digital technologies play an important role in the learning process, not only supporting the adaptation of techniques for various theories and elements of the respective areas but also the acquisition of knowledge about new techniques for specific industries such as the automotive, aerospace, and aeronautic industries, among others. The collaboration between educational institutions and companies must be aimed at improving innovation and creativity competencies, providing students with a professional future with better employability prospects.

Some of these talents can be pursued in the areas of computer-aided design, manufacturing and engineering (CAD/CAM/CAE), and digital manufacturing, as well as in different fields of specialization such as simulation of processes and physical phenomena. In addition, 3D drawing training is useful, especially when analyzing situations beyond design such as simulating flow and making/testing product prototypes, as it offers more precision and control. Furthermore, 3D CAD drawing skills are useful to former students employed in consultancy and engineering design areas. 3D printing/I4.0 helped to improve students’ lifelong learning skills, creating an innovative and forward-thinking mindset. IoT, cloud computing, autonomous robots/AI, simulation, VR/AR, additive manufacturing, BD, informatics security, and integration of horizontal and vertical systems are areas of interest to be improved.

To introduce students properly to many relevant I4.0 elements, teaching and training in the automotive sector should be oriented to specific topics such as simulation studies on real-time data and optimization, AI, BD, enhanced productivity education on additive manufacturing and advanced robotics, maintenance/logistics studies through augmented reality, and evaluation studies on the environment, society, and economics. Within these topics, AR and VR involve an alternative method for perception of the surrounding real world. This is achieved by using virtual elements imposed through various technological devices such as smartphones, tablets, or virtual reality devices. Moreover, VR is defined as the development of simulated experiences similar to real-life situations. Improved VR/AR is being used in the early stages of product development and the construction and validation of prototypes. VR/AR technologies are valuable tools not only in the design stages but also in the training of operators in assembly processes, allowing safe interaction with the product prior to commissioning.

This part analyses actual situation of currently utilized educational methods in automotive sector with focus on application of innovative methods using ICT and describes possible implementation of virtual reality into employee/learner training in automotive industry.

Employee training equips workers with know-how that is essential for their work. Information and communication technology (ICT) enable applying various innovative methods for employee training.

Applying ICT in education may improve the quality of education and the active participation of trainees on educational process. One of it represents the virtual reality technology that represents interactive virtual environment with the most possible resemblance to physical reality. The most suitable sectors for implementing virtual reality trainings are e.g. flying trainings of pilots, simulating surgical interventions in chirurgery, remote control of robots or production and assembling. 

Therefore, this unit tries to provide and describe implementation of virtual reality for employee/learner training in the automotive sector in case of assembling a car instrument board of a personal car. It describes the development process stage by stage including brief description of processing at individual stages for better demonstration. 

2. Steps for creating training in virtual reality

This section describes a possible process of creating virtual reality training for employee education in automotive industry that can be re-proposed by a VET trainers. The a description deals with one selected assembling activity that the worker has to learn during this training and that he will execute in his job later on. The chosen task represents only one example of possible applications of virtual reality in employee training in production sector. The process of creating training content in the environment of virtual reality consists of several steps. The whole educational path will be applied on a specific situation: the example is the dashboard installation in personal car.

The process consists of the following steps:

1.      Selection of an object which shall be virtualized, supplying documents and scenarios by customer.

2.      Photo shooting of objects not included in documents provided by the customer, creating Detailed Functional Technical Specification (DFTS)

3.      Editing and processing of photos for textures, materials and panorama photos.

4.       Modeling of objects.

5.       Creating textures and materials.

6.       Control and summary of objects.

7.      Animation

8.      Creating virtual scenery and programming of logic.

9.      Testing of simulation.

10.   Customer delivery

2.1. Preparation

First step contains the selection of the object to virtualize; the object has to be photographed for digitalizing, a digital 3D model is created and parsed in the virtual reality environment. For this step the cooperation of the required customer/organization is necessary. 

Thus, the production company provides available technical documentation for all objects to be virtualized, preferably in electronic format or in printed form. This step is also called creating scenarios for designing simulation in virtual reality. The customer company has to collaborate in designing scenarios mirroring the individual working processes along the assembly line to be reality close emulated by the virtual reality simulation. 

The outcomes of this first step is the summary of all documents for all objects and scenarios for all work processes necessary for creating the required training in virtual reality environment. 

In our concrete example of virtual dashboard installation in a personal car, it means the selection of the dashboard to be installed with all its assembly parts and the place of installation at the assembly line. The producing company provides all materials denoting all 3D components and objects that are available in electronic form including all technical drawing documents containing the technical drawing of the assembly hall, tools and equipment utilized for the installation. 

The realizing team is then able to compose and build the dashboard in digital form in virtual reality and create the training for workers.

2.2. Complementing

The main purpose of this step is to complement all missing data for virtualization of all objects required for the virtual reality training. In other words, during this step it is necessary to physically visit the producing company in person and to analyze if all necessary documents for all needed objects and components are available for digitalization.

At this step, the personnel of the realizing team identifies and documents all objects that shall be included in the virtual reality training and for which the necessary documents and descriptions are missing from the preparation step. This means to shoot photos of all missing objects, to measure them to document their dimensions and to create their digitalized reproductions. During this complementing step the individual processes building the training scenarios are digitalized as well. This is done with camera recording of the processes, if possible. If recording is not possible, the process is emulated following the process descriptions delivered in first step in collaboration with representatives of the customer company to identify any gaps, unclear or forgotten activities or components and to clarify ambiguous parts.

At the end of this step, all data necessary for simulating the training in virtual reality environment will be collected.

 To demonstrate this step for the selected activity of assembling dashboard and its installation into a personal car, it means the realizing team visits the producing company including their assembly line and observes the reproduction of the assembling activity and records it with a camera.

The realizing team examines whether all components and objects utilized during the observed activities were already in possession of the realizing team for virtualization and identifies objects that are missing or deviating from delivered documents. It shoots photos of all components needed for the assembling of the dashboard and records all its attributes including dimensions, descriptions of surfaces and other features. It concludes in-depth description of all objects and detailed depiction of the assembly process step by step. The underlying document also narrowed determination of the functionality included in the final simulation in virtual reality environment along with the detailed description of the creating process.

2.3. Finalizing preparation

After all complementing tasks were finalized, the next step compounds photo shooting of all component parts and other parts as well as documenting the working space for assembling to be virtualized and reproduced in virtual reality. The photos can be divided into three types: reference photos, texturing photos and panorama photos.

Reference photos serve for better demonstrativeness and base for the 3D designer to accomplish his/her task. These photos are then used also to confront the 3D model depicting the imagination of the 3D designers with the real, physical and actual state of the virtualized object.

Second type of taken photographs denotes photos for texturing and material covering. The very important requirement for these photos is the conditions of shooting them in right angle to the surface of given object parts that are texturized.

Third and last photo types represent panorama records that are used for virtualization of surroundings. In other words, these photos build the background of the whole scenery and virtual workplace. These photos serve for reality close perception of the whole virtual training.

To demonstrate the aforementioned activities on the assembling and installation of a instrument panel for a personal car, this step compounds next personal visit of the realizing team representatives to the producing company. During this visit, the team members take photos of all components and objects using perspective mode to create reference photos. After all reference photos were created, the realizing team shoots photos without perspective for texturing and modelling the surface of the material.

After all components including screws, bolts, gripping, anchoring, front case cover of the dashboard, doors, tachometer, r. p. m. counter and other objects were documented and photographed, the working tools utilized for the assembly is being photo shoot including industry screw driver and air screw drivers. The photos are taken of the working table and assembly line too, including all utilized stands, technical drawings and support tables, eventually cranes or other levering tools. Panorama photos of the whole working place and assembly hall are shoot. These are made from the middle of the working place of the future assembling team. These photos are later combined in the virtual reality environment to build 360° panorama to emulate the reality close perception of the trained worker.

2.4. Modeling

Finalizing all the documenting part, the real physical objects and components are reproduced in virtual reality. Each single component, object and tool is reproduced individually using the lowpoly (low polygon technology) technology. This processing mode denotes modelling in simple forms where all complex surface characteristics are emulated through texturing, e.g. the threads of a screw etc. The selection of this technological procedure for modelling allows faster processing of the objects and also higher processing speed of the final software application due to lower visualizing complexity.

Electronic models provided by the customer company are transformed into lowpoly format. Components documented only in paper form by technical documentation and drawings or reference photographs are being created from the scratch by 3D modelers. All in lowpoly format. the 3D object models build the base and the essential part of the scenery with highest importance.

To demonstrate the processing activities on chosen procedure for assembling of car dashboard, it means that all materials and documents received from the producing company in first process step and created by the realizing team in second process step are delivered to designer team responsible for 3D modeling. They process component by component and transform it into lowpoly format.

2.5. Texturing

Next step, the texturing, is usually done simultaneously in parallel with 3D modeling step. During this step, the textures and surface emulating models for the materials are being created and prepared utilizing special technological procedure to eliminate the visibility of connecting lines and to ensure smooth flow.

It stands for creating 2D visualizations of 3D models. Graphic designers combine textures with unwrapped 2D visualizations and create details like oil flecks etc. to emulate real physical working environment to the greatest possible extend.

To demonstrate this processing on selected training activity, at this step a major part of the 3D designers creates 3D models of all objects and the remaining team members are dedicated to model unwrapping of composed 3D models of all objects including screws, pads, matrixes and bolt nuts as well as individual parts of the car dashboard like tachometer, r. p. m. counter etc. Each unwrapped object is then provided to 2D graphical team to get “dressed” which cover all models with appropriate corresponding textures. At this state, the modeling part and the graphical processing of 3D models for these objects is finished.

2.6. Model Summary

At this step, the comparison of created wrapped 3D models with reference photos is carefully

assessed. Finishing verifying produced models with reference photos, the planned and described scenarios are reviewed to double control whether all wrapped 3D models of needed objects are created in the virtual reality environment.

 To demonstrate the processing of this step on selected activity for assembling car dashboard, it concludes the verification of all created models through their comparing with technical drawings and documentation provided by the producing company. Each single model is checked to assure its creating, its compliance with documentation. Similar reviewing is conducted for the workplace scenery, working equipment and surroundings.

2.7. Animation

Animating team consisting of animators and riggers are provided with all 3D wrapped object models for processing. A rigger creates digital skeletons, bones, bone joints and muscles of all 3D object models.

This step denotes the processing of created models by riggers and animators for the chosen activity of assembling a car dashboard. A rigger reads and analyses all training scenarios accomplished in previous steps  and identifies all animation necessary parts.When all objects are furnished with rigs, animators start to animate objects through creating visualizations of moving behavior, e.g. rotating gear wheels, rotating sharpener of the angular grinder, rotating drill, etc. All the rotations are reproduced in virtual environment emulating various intensity levels of moving objects.

To demonstrate the processing of this step on selected training activity for assembling car dashboard in automotive industry, all animated and wrapped 3D model objects are inserted into the software for creating virtual reality simulations to create new scenery. All objects are placed to their positions described in the scenarios, technical drawing specification and reference photographs. For the surrounding of the scenery 360° panorama photo is used. After finishing the graphical visualizations, the programming for creating interactive dynamic content starts.

At this step, all components are processed and assigned interaction logic object by object until the whole dashboard is not completed.

2.8. Programming business logic

At this eight step, the prepared modeled and animated objects are inserted into the software for creating simulations and trainings in virtual reality. Finishing the input of all objects into the environment, the scenery of the working place is created based on underlying scenarios. Objects are placed to their positions according to the description of workplace in given scenario. The created workplace with all its components compounds the base for programming of business logic of given training scenario. In other words, all buttons and other interactive fields are assigned programmed events how to react.

To demonstrate the processing of this step on selected training activity for assembling car dashboard in automotive industry, all animated and wrapped 3D model objects are inserted into the software for creating virtual reality simulations to create new scenery. All objects are placed to their positions described in the scenarios, technical drawing specification and reference photographs. For the surrounding of the scenery 360° panorama photo is used. At this step, all components are processed and assigned interaction logic object by object until the whole dashboard is not completed.

2.9. Testing

When all objects are modeled, animated and enriched with interaction logic, the testing can start. The testing compounds several testing aspects: testing the graphical interface containing all model objects of the simulation, testing all required and programmed functionality of all objects and the simulated workplace surroundings, etc.

The testing consists of several testing scenarios applied in software development industry in following order:

·        Integration tests – testing all individual features separately and common interaction of all software objects working together;

·        System testing – the application is tested as whole with focus on compliance with functional requirements and technical specification;

·        Regression testing – analyzing whether adding new objects, functionality and/or bug fixes has no influence on previously achieved and tested functions

·        Installation testing – the overall computational performance and load capability of the whole system is sufficient with regards to end hardware, operating system and installed software;

·        Acceptance testing – end to end testing following various cases described in the specified scenarios and test cases. 

2.10. Customer Delivery

A handover protocol is created containing all important information including the documentation of the whole software as well as manual for simulation utilization. The software is then commissioned into the educational environment of the customer organization, in other words it is installed on customer hardware at customer side under supplier’s supervision and tested. First employee trainings are conducted by the supplier to school the employees and future trainers how to use the virtual reality training.

The final form of the proposed solution composes the virtual environment of the assembly line for assembling a car dashboard that resembles the reality to the greatest possible extend in graphical design as well as in functional features. Employees, trained in virtual reality via appropriate hardware suitable for virtual reality application, can execute all tasks of the car dashboard assembly that emulates real works on physical system.

The training solution in virtual reality integrated gamification factors that support trainee interaction and motivation and prevent monotony and tedious static learning based on text and pictures as is valid for traditional educational methods. At the same time, the trainee is forced to involve more senses and can control the learning process through his/her own interactions that provide better demonstrativeness and contribute to greater internalization of taught content. Among the advantages of virtual reality training belong the reality close perception of the training and the immense similarity of conducted tasks with physically executed activities and the graphical visualization of all components, tools and objects alike physical reality.

3. Cost calculation

For the development of described training solution in virtual reality a large group of experts is necessary including:

·        Team of programmers with experience in programming virtual reality;

·        3D designers;

·        2D designers;

·        Animators;

·        Riggers;

·        Photographers

·         Project manager; etc.

The cost of described solution can be estimated to approximately 20.000 euros where 3.000 euros represents the hardware costs.  However, such orders are usually not only for simulating one activity, but multiple activities of similar processes e.g. assembling various car dashboards with diverse appearance, but with similar assembly scenario.

The development of described final solution takes approximately  3 months from the customer's delivery of all necessary documents on.

4. Conclusion

The hardware needed for implementing the described virtual reality training is estimated at the price of 3.000 euros where 2.200 euros corresponds to desktop computer with the appropriate parameters for such type of virtual reality simulation and 800 euros corresponds to the displaying device for virtual reality such as glasses etc.

Emergence of information and communication technology allowed new ways of processing and enabled innovation of established methods and techniques. In the field of education, ICT provide interactive and attractive education in virtual reality environment with high resemblance to physical reality. The learners are acquiring new knowledge through attempts and failures in dynamic and interactive environment that provide the empirical “learning by doing” experience.

This example proposed implementation of virtual reality into employee education in production sector of car dashboard assembly in automotive industry. It outlined and described the development process of such a training with estimated cost calculation to provide more insights for applying virtual technology in education to broad public.