Neptune: AI serving lifeguards

Every year, thousands of drownings occur in supervised areas. Despite lifeguards’ vigilance, environmental factors such as crowd density, sunlight reflections, and distractions make it difficult to identify danger. Moreover, it’s nearly impossible for a human to stay fully alert for 8 hours straight.

Neptune was born from a simple observation: just before drowning, most victims exhibit typical behavior. They sink and resurface several times, flailing their arms in a disorganized manner — a pattern known as the “bobbing motion”.

By using AI to detect these early warning signs, Neptune can trigger alerts before it’s too late.

Detecting people at sea is a major technical challenge. Swimmers appear small, are partially submerged, and are affected by distance and light reflections.

To address this, the NHD module was initially built on YOLOv11, a fast object detection model. However, early tests revealed its limitations: trained on urban datasets, YOLOv11 struggled to identify distant or partially submerged silhouettes.

The Neptune team then built a custom dataset from beach footage under various conditions: distances, viewing angles, and lighting. Fine-tuning YOLOv11 on this dataset significantly improved detection performance in aquatic environments.

In March 2025, a new model emerged: D-FINE, recently released as open source. Designed for high-precision object detection, D-FINE quickly outperformed the fine-tuned YOLOv11, especially in visually complex scenes.

After several comparative tests, NHD officially migrated to D-FINE, now serving as Neptune’s primary human detection engine.

In a dynamic environment like the sea, clearly separating water from land is crucial. The NWSD module was designed to segment the sea surface in real time, even with shifting light and wave conditions.

This precise water-plane detection is fundamental to Neptune’s operation: it helps filter out false alerts caused by beach activity and defines danger zones for other modules.

To train this module, the team created a custom dataset of coastal images captured under various weather conditions and at different times of day.

The Neptune Drowning Detection (NDD) module is at the heart of the system’s alert process. Its goal: detect distress situations in real time and trigger an alert before it’s too late.

To achieve this, the team developed a custom algorithm capable of tracking each detected individual in the water using a unique identifier.

This tracking system determines precisely when a person disappears beneath the surface, and for how long. Each individual is assigned a danger score that evolves dynamically based on distance from shore, immersion time, and frequency of disappearance.

When this score crosses a critical threshold, an automatic alert is triggered: a sound signal warns lifeguards while the interface highlights the estimated location of the person in danger.

This mechanism allows abnormal behavior detection without constantly watching the screen, enabling fast response even in complex conditions like crowded beaches or rough seas.

To assist lifeguards, Neptune projects detections onto a 2D top-down map of the beach.

This map is generated in real time using a mathematical method called homography, derived from a fixed camera.

The system automatically detects four anchor points in the image (two on the beach, two on the sea side), requiring no manual calibration. This allows adaptation to tides, weather, and camera angles.

Each detected person is represented by a colored dot (green to red depending on risk), with a short trail showing their recent trajectory.