In the following work, a novel multimodal sensor data fusion on far infrared imaging and lidar is elaborated and its effects on avderse weather conditions are studied. Furthermore, a well-known problem in the thermal imaging domain is addressed, the existing large domain gap to existing datasets for the pre-training of deep neural networks. With the help of artificial thermal images the training of underrepresented object classes can be improved. The final analysis takes into account light fog, dense fog and snow during day and night. This information is finally evaluated, characterized and summarized with an outlook showing that infrared and far infrared wavelengths have strong challenges with atmospheric humidity. Both lidar and thermal imaging show degraded performance in these conditions.