For an autonomous vessel to navigate safely, knowing its own position is only part of the challenge. It must also understand what is happening around it.

Where are the other vessels? How fast are they moving? Is that radar return a ship, a buoy or another object? Is a small craft approaching on a collision course? What happens if visibility deteriorates or one source of information becomes unreliable?

A human bridge crew answers these questions by combining information from many sources: visual observations, radar, AIS, electronic charts, navigation instruments and experience. An autonomous vessel needs to build a similarly comprehensive picture digitally.

This is where sensor fusion for autonomous vessels becomes essential.

Sensor fusion combines information from multiple onboard sensors and data sources to create a single, continuously updated understanding of the maritime environment. Artificial intelligence can then interpret that information, identify objects, assess their movement and help the vessel make safe navigational decisions.

In this article, we look more closely at how maritime sensor fusion works, why individual sensors are not sufficient on their own, and how AI turns multiple streams of data into the situational awareness required for autonomous navigation.

---

If you are new to maritime autonomy, our guide What Is an Autonomous Surface Vessel? provides an introduction to ASVs and their applications, while How Autonomous Surface Vessels Navigate Without a Crew explains the wider autonomous navigation process. This article takes a deeper look at one of the most important technologies underpinning both: sensor fusion.

---

What Is Sensor Fusion?

Sensor fusion is the process of combining information from multiple sensors and data sources to create a more accurate and reliable picture than could be obtained from any individual sensor alone.

For an autonomous vessel, those inputs can include:

• Marine radar
• Electro-optical cameras
• Thermal cameras
• LiDAR
• AIS (Automatic Identification System)
• Sonar
• GPS and GNSS
• Inertial Navigation Systems (INS)
• Electronic compasses
• Motion sensors
• Weather and environmental data

Each source provides a different piece of information.

Radar is highly effective at detecting objects and measuring their range and bearing. Cameras provide visual information that AI can use to identify and classify objects. AIS supplies information transmitted by participating vessels. GNSS provides geographic positioning, while inertial systems help determine the vessel’s movement, heading and orientation.

Sensor fusion brings these separate observations together into a coherent operational picture.

For autonomous navigation, this fused picture becomes the vessel’s digital understanding of the world around it.

Why Can’t an Autonomous Vessel Rely on a Single Sensor?

The maritime environment is particularly challenging because no sensing technology works perfectly in every situation.

A camera can provide extremely rich visual information but may become less effective in darkness, fog, spray or heavy rain.

Radar can operate in poor visibility and detect targets over significant distances, but it may not provide enough information to identify exactly what an object is.

AIS can provide useful information about another vessel’s identity, course and speed, but not every object on the water will be transmitting AIS.

GNSS provides highly accurate positioning but is only one element of a resilient navigation system.

Marine AI’s existing autonomous navigation approach therefore combines multiple sensor inputs rather than depending upon a single source. Its GuardianAI™ technology is designed to integrate data including radar, LiDAR, sonar, cameras and AIS to build a real-time operational picture. (⁠Marine AI)

The principle is straightforward:

where one sensor has a weakness, another may provide the missing information.

That makes the overall system more capable and more resilient.

The Sensors Behind Autonomous Vessel Situational Awareness

Marine Radar

Radar is one of the most important sources of situational information at sea.

It can detect surrounding vessels and objects and provide information about their:

• Range
• Bearing
• Relative movement
• Speed
• Direction of travel

Crucially, radar can continue operating at night and in conditions where visual sensors may become less effective.

For an autonomous system, however, detecting a radar return is only the beginning. The software must determine whether observations made over time relate to the same object and understand how that object’s movement affects the vessel’s own navigation.

Electro-Optical Cameras

High-resolution cameras give autonomous systems something closer to the visual information available to a human lookout.

Computer vision can analyse these images to detect and classify objects such as:

• Ships
• Small boats
• Navigation buoys
• Harbour infrastructure
• Floating objects
• Other potential hazards

This ability to classify what the system can see is particularly important.

A radar return might indicate that an object is present. A camera may provide additional evidence about what that object actually is.

Thermal Imaging

Thermal cameras add another layer of perception, particularly at night or when conventional visual imagery is limited.

By detecting differences in thermal radiation rather than relying entirely on visible light, thermal imaging can provide additional information about vessels, people and other objects.

When combined with radar and electro-optical imagery, this creates another independent source against which detections can be compared.

LiDAR

LiDAR uses laser pulses to measure the distance between the sensor and surrounding objects.

It can generate highly detailed information about the immediate environment and can be particularly useful where precise spatial awareness is required, including close manoeuvring and operations around infrastructure.

LiDAR data can complement radar and camera information by providing highly accurate measurements of an object’s position and shape.

AIS

The Automatic Identification System provides another valuable stream of information.

Depending upon the vessel and transmission, AIS information can include:

• Vessel identity
• Position
• Course
• Speed
• Heading
• Navigational status

Unlike radar or cameras, AIS is not simply observing an object. It is receiving information transmitted by the other vessel.

That makes it extremely useful, but autonomous navigation cannot depend upon AIS alone. Some vessels may not carry AIS, it may not be operating, and floating objects and other hazards do not transmit AIS signals.

Sensor fusion allows AIS to be considered alongside independently observed radar and visual information.

How Does AI Combine All This Information?

Simply installing more sensors does not automatically create better situational awareness.

The real challenge is turning thousands of individual observations into information the autonomous system can understand and act upon.

Imagine an autonomous vessel approaching a busy area.

Its radar detects a moving target.

AIS reports a vessel at approximately the same location.

A camera identifies an object with the visual characteristics of a commercial vessel.

Successive observations show that it is travelling at a particular speed and heading.

Rather than treating these as unrelated pieces of information, the autonomy software can associate them with the same physical vessel.

The result is a much richer understanding of that target.

The system can continuously update information such as its position, direction of travel and relative movement and use this to determine whether it presents a navigational risk.

This process happens across multiple targets simultaneously and continues as the vessel moves through its environment.

From Detection to Situational Awareness

There is an important distinction between detecting objects and achieving situational awareness.

Detection answers:

What is around me?

Situational awareness goes further:

What is happening around me, what is likely to happen next, and does it require action?

An autonomous vessel therefore needs to understand not only that another vessel exists but how that vessel relates to its own route.

For example, another vessel may be:

• Moving safely away
• Travelling on a parallel course
• Overtaking
• Approaching head-on
• Crossing the autonomous vessel’s path
• Stationary
• Manoeuvring unpredictably

AI can continually analyse the fused sensor picture to determine how these situations are developing.

This is where vessel situational awareness becomes the foundation for intelligent autonomous decision-making.

Predicting What Happens Next

Safe navigation is not simply about reacting to where another vessel is now.

An experienced mariner anticipates where it is likely to be several minutes into the future.

Autonomous navigation software needs to do the same.

Using the fused situational picture, the system can evaluate factors including:

• Current course
• Heading
• Speed
• Relative bearing
• Relative velocity
• Closest Point of Approach (CPA)
• Time to Closest Point of Approach (TCPA)

By continuously recalculating these variables as new sensor data arrives, the autonomy system can identify potential conflicts before they become immediate dangers.

The system can then use that information as part of its collision-risk assessment and route-planning process.

Sensor Fusion and Collision Avoidance

Sensor fusion is therefore closely connected with autonomous collision avoidance, but the two should not be confused.

Sensor fusion establishes what is happening.

Collision-avoidance software decides what to do about it.

Before an autonomous vessel can make an appropriate manoeuvre, it needs sufficiently reliable information about the vessels and hazards around it.

The fused situational picture provides this foundation.

If the system determines that another vessel presents a potential collision risk, autonomous navigation software can evaluate possible responses such as altering course or speed.

Those decisions must also take account of the International Regulations for Preventing Collisions at Sea (COLREGs), helping autonomous vessels behave in a manner that is safe and understandable to other water users.

We will explore this process in greater detail in our forthcoming article on how autonomous vessel collision avoidance works.

What Happens When Sensors Disagree?

One of the most interesting challenges in sensor fusion occurs when different sensors provide different information.

For example, a radar may detect an object at a particular location while a camera produces an uncertain classification because visibility is poor.

Rather than automatically trusting one sensor, a sophisticated autonomy system can evaluate the available evidence across multiple inputs.

Additional observations may strengthen or weaken the confidence associated with a detection.

This is particularly important at sea because sensing conditions continually change.

Sun glare may affect a camera.

Sea clutter can complicate radar returns.

Rain, fog and spray can affect sensor performance.

Traffic can become dense.

Information sources may temporarily become unavailable.

Using multiple complementary sensors helps prevent the vessel’s situational awareness from depending entirely on one source of information.

Building Resilience Through Redundancy

This redundancy is one of the major benefits of sensor fusion for autonomous vessels.

If one source becomes degraded, other sources can continue contributing to the operational picture.

For example:

Poor visibility: radar and thermal sensing can continue providing valuable information when conventional cameras are less effective.

Incomplete AIS information: radar and vision systems can detect vessels and objects independently.

Uncertain visual classification: radar, LiDAR or other sensors can provide additional positional and movement information.

This does not mean autonomous vessels can operate without limits in every condition. Vessel design, sensor configuration, weather and the operational environment all affect capability.

Instead, sensor fusion helps create a more resilient perception system by avoiding unnecessary dependence on any single technology.

Situational Awareness in Congested Waters

The benefits become particularly apparent in complex maritime environments.

A vessel approaching a busy port may simultaneously encounter commercial ships, pilot boats, tugs, leisure craft, navigation marks and harbour infrastructure.

The autonomous system must continuously determine:

• Which objects are moving
• Which are stationary
• Which objects are vessels
• How each target is moving
• Which targets may interact with the planned route
• Which situations require greater attention

Marine AI describes GuardianAI™ as using AI-driven sensor fusion to integrate existing radar, AIS, sonar and camera systems into a real-time operational picture, supporting safe navigation in congested waters. (⁠Marine AI)

This is significantly more sophisticated than simply following a series of predetermined waypoints.

The vessel is continually interpreting a dynamic environment.

Sensor Fusion Isn’t Only for Uncrewed Vessels

Although sensor fusion is fundamental to autonomous vessels, the same technology can also improve situational awareness aboard conventional crewed ships.

Rather than replacing the mariner, AI can act as an additional decision-support layer.

A fused operational picture can help bridge teams understand complex situations by bringing together information from several systems and highlighting developing navigational risks.

Marine AI’s GuardianAI™ Software Suite is also designed for crewed vessels, where sensor fusion and data integration can provide enhanced situational awareness and real-time COLREG-compliant route optimisation. (⁠Marine AI)

This demonstrates an important point about maritime autonomy.

The technologies being developed for autonomous vessels can also support safer and more informed decision-making aboard crewed vessels.

Sensor Fusion Across Different Maritime Missions

The exact sensor configuration required by an autonomous vessel depends on what it is being asked to do.

A long-endurance offshore vessel may have different sensing priorities from a vessel operating within a harbour.

A hydrographic survey vessel may combine navigation sensors with specialist survey equipment.

A defence platform may require persistent surveillance and sophisticated object detection.

An underwater autonomous vehicle may depend much more heavily on sonar and inertial navigation because GNSS signals are unavailable while submerged. Marine AI’s underwater-vessel technology, for example, combines multi-sensor fusion with autonomous navigation and data collection for subsea operations. (⁠Marine AI)

The principle remains the same: combine complementary sources of information to create a more complete understanding of the operating environment.

The Future of AI-Driven Sensor Fusion at Sea

As sensors and AI continue to improve, maritime situational awareness is likely to become increasingly sophisticated.

Future developments will enable autonomy systems to interpret larger amounts of information, distinguish increasingly complex behaviours and operate effectively across a wider range of maritime environments.

Greater integration between onboard sensors, navigation software and remote operation centres will also allow information to be shared more effectively between vessels and shore-based teams.

For fleets of autonomous vessels, this creates further possibilities.

Individual vessels could contribute observations to a wider operational picture, enabling multiple autonomous platforms to coordinate missions and share information about their environment.

The progression is therefore from individual sensors, to a fused vessel-level picture, and ultimately towards increasingly connected maritime intelligence.

How Marine AI Builds Intelligent Maritime Situational Awareness

Marine AI develops autonomous navigation and vessel intelligence technology through its GuardianAI™ Software Suite.

GuardianAI™ integrates information from multiple sensor sources to help vessels perceive and interpret the maritime environment in real time. Marine AI states that its technology can combine radar, LiDAR, sonar, cameras and AIS as part of AI-powered navigation and collision avoidance. (⁠Marine AI)

By turning multiple streams of sensor data into a coherent situational picture, the technology provides the foundation required for intelligent navigation, collision-risk assessment and autonomous decision-making.

For uncrewed vessels, that capability is fundamental to operating safely with minimal human intervention.

For crewed vessels, the same technology can augment bridge teams with enhanced situational awareness and decision support.

As autonomous maritime operations become more widespread, effective sensor fusion will remain one of the technologies that makes safe, scalable autonomy possible.

Frequently Asked Questions

What is sensor fusion in autonomous vessels?

Sensor fusion is the process of combining information from multiple sensors and data sources, such as radar, cameras, LiDAR, AIS and navigation systems, to create a more complete and reliable picture of the vessel’s surroundings.

Why do autonomous vessels need multiple sensors?

No individual sensor performs perfectly in every maritime condition. Cameras, radar, AIS, LiDAR and other technologies have different strengths and limitations. Combining their information improves situational awareness and provides greater resilience if one source becomes degraded or unavailable.

How does AI improve vessel situational awareness?

AI can analyse information from multiple sensors, associate observations with individual objects, classify targets, track their movement and assess how they may interact with the vessel’s route. This turns raw sensor information into an operational picture that can support navigation decisions.

What sensors are used by autonomous vessels?

Depending on the vessel and mission, sensors can include marine radar, electro-optical and thermal cameras, LiDAR, sonar, AIS, GNSS, inertial navigation systems and environmental sensors.

How does sensor fusion help prevent collisions at sea?

Sensor fusion provides collision-avoidance software with a continuously updated picture of surrounding vessels and hazards. The system can use information about position, speed, heading and relative movement to identify potential conflicts and support appropriate navigational decisions.

Can sensor fusion be used on crewed ships?

Yes. Sensor fusion can enhance the situational awareness of bridge teams by combining information from multiple navigation and sensing systems into a more comprehensive operational picture. It can therefore be used both as part of autonomous navigation and as decision-support technology on crewed vessels.