Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
5a5a80f
[fix] - update PyTorch version mismatch and adjust build commands for…
mgiberna Apr 17, 2026
5630cf9
[fix] - update SSH_AUTH_SOCK path and correct workspace paths in dock…
mgiberna Apr 20, 2026
a7f7308
[fix] - correct PyTorch version in Dockerfile to match CUDA 12.6
mgiberna Apr 22, 2026
5d3a4af
add eomt scene segmenter support
mgiberna Jun 4, 2026
be8f890
migrate mono-inertial to ros2
mgiberna Jun 4, 2026
12acb01
[fix] add IMU.Threshold parameter to configuration files and disable …
mgiberna Jun 4, 2026
6b177bb
add mono-inertial sensor setup
mgiberna Jun 4, 2026
4f26dab
fix param initialization bug
mgiberna Jun 8, 2026
29405fe
refactor image handling in SharedState to use a queue for better time…
mgiberna Jun 8, 2026
bce94df
refactor SharedState to use deque for gyro and accel data, increase m…
mgiberna Jun 9, 2026
58354f8
enable rgb input support
mgiberna Jun 10, 2026
e3ee25e
refactor to remove private slam instance to shared pSLAM
mgiberna Jun 10, 2026
ea5a798
add checks to skip frames with insufficient IMU measurements or futur…
mgiberna Jun 10, 2026
2b0087c
add auxiliary depth support to KeyFrame and System, update ImageGrabb…
mgiberna Jun 11, 2026
afd349e
fix time inconsistency
mgiberna Jun 11, 2026
99c458f
prevent to accumulate stale frames in image queue
mgiberna Jun 11, 2026
a6b2275
convert and store pointcloud instead of depth frame to keyframe
mgiberna Jun 11, 2026
07bb298
reduce depth processing computational overweight on ros callbacks
mgiberna Jun 12, 2026
85d40e3
minor updates
mgiberna Jun 12, 2026
f8c4316
add launch aurgument for optional use of aux depth
mgiberna Jun 12, 2026
8c045e4
Delete docker/Jazzy_new.Dockerfile
mgiberna Jun 12, 2026
f5f7cb9
Modify docker-compose.yml
mgiberna Jun 12, 2026
46a850c
add instance mask handling in ImageGrabber
mgiberna Jun 16, 2026
e87ec87
integrate yolo26 scene segmenter
mgiberna Jun 16, 2026
418e7f2
add mask parameter to Frame, System, Tracking classes for ORB feature…
mgiberna Jun 17, 2026
de35cc7
fix thread default argument
mgiberna Jun 17, 2026
c674141
fix inverse mask bug and add safeguard for low feature frames
mgiberna Jun 17, 2026
68bc6c1
Merge branch 'master' of github.com:snt-arg/dynamic_visual_sgraphs
mgiberna Jun 17, 2026
75f319e
refactor ImageGrabber class by removing unused segmentation worker an…
mgiberna Jun 17, 2026
b463f33
add mask-based keypoints rejection
mgiberna Jun 17, 2026
6e8ae66
enhance ImageGrabber synchronization by adding instance mask buffer c…
mgiberna Jun 17, 2026
413bdd4
add support to keyframe_depth_estimator
mgiberna Jun 22, 2026
cdcb01d
add sky_handling flag for keyframe depth estimator
mgiberna Jun 22, 2026
2ac0aae
minor updates
mgiberna Jun 22, 2026
99687e0
add integration with keyframe depth validator
mgiberna Jun 24, 2026
f9a775f
add debug param
mgiberna Jun 24, 2026
d61ceac
add offline depth support
mgiberna Jun 25, 2026
7b91b75
add launch arguments for keyframe depth validator tuning and rviz con…
mgiberna Jun 25, 2026
c3652b0
add depth validator tuning related parameters
mgiberna Jun 25, 2026
337a3e6
minor
mgiberna Jun 26, 2026
f6e9d23
add dynamic keypoints lifting pipeline to launch file
mgiberna Jun 26, 2026
71ca6dd
fix static tf publisher
mgiberna Jun 29, 2026
9c23a81
add dynamic keypoint interpolator launch and arguments
mgiberna Jun 30, 2026
a0da6bb
update gitignore
mgiberna Jul 1, 2026
f1a9375
add object motion estimator configuration and launch arguments
mgiberna Jul 1, 2026
a67c0db
add motion estimate 3d viz params
mgiberna Jul 6, 2026
61ff504
update launch arguments for object motion BA and keyframe depth compo…
mgiberna Jul 7, 2026
0093ad0
update launch parameters and add static map points publisher
mgiberna Jul 8, 2026
fde6e4b
give dedicated thread to imuGrabber
mgiberna Jul 10, 2026
3226da7
fix docker dependencies
mgiberna Jul 10, 2026
7376808
add config for uHumans2 dataset
mgiberna Jul 13, 2026
2e3b3bd
add MapResetEvent and MapReadyEvent messages; update Atlas and common…
mgiberna Jul 14, 2026
f0adbf8
add evaluation node for uHumans2 datasets
mgiberna Jul 14, 2026
a3ed001
rescale global poses and centroids for markers, doors, planes, rooms,…
mgiberna Jul 14, 2026
a22eb17
add MapRescaleEvent message and related functionality for scale corre…
mgiberna Jul 15, 2026
f1cb866
Add IMU-aided relocalization and related features
mgiberna Jul 18, 2026
02a75db
Add IMU bias estimation functionality and related message
mgiberna Jul 20, 2026
a7c893a
Implement UpdateTrackedPointsFromCurrentFrame method and update track…
mgiberna Jul 21, 2026
76ab2e3
add install script
mgiberna Jul 27, 2026
fff23d8
Update launch configuration and docker-compose for new directories an…
mgiberna Jul 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,4 @@ borrar/*

*/ExecMean.txt

!Dockerfile
docker/docker-compose.yml
!Dockerfile
15 changes: 14 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ find_package(Eigen3 3.1.0 REQUIRED)
find_package(backward_ros REQUIRED)
find_package(geometry_msgs REQUIRED)
find_package(segmenter_ros REQUIRED)
find_package(keyframe_depth_estimator REQUIRED)
find_package(keyframe_depth_validator REQUIRED)
find_package(image_transport REQUIRED)
find_package(message_filters REQUIRED)
find_package(rviz_visual_tools REQUIRED)
Expand All @@ -66,6 +68,10 @@ rosidl_generate_interfaces(${PROJECT_NAME}
"msg/VSGraphsRoomData.msg"
"msg/VSGraphsAllWallsData.msg"
"msg/VSGraphsAllDetectdetRooms.msg"
"msg/MapResetEvent.msg"
"msg/MapReadyEvent.msg"
"msg/MapRescaleEvent.msg"
"msg/ImuBiasEstimate.msg"
DEPENDENCIES std_msgs geometry_msgs sensor_msgs
)

Expand All @@ -83,6 +89,8 @@ include_directories(
${Pangolin_INCLUDE_DIRS}
${PCL_INCLUDE_DIRS}
${segmenter_ros_INCLUDE_DIRS}
${keyframe_depth_estimator_INCLUDE_DIRS}
${keyframe_depth_validator_INCLUDE_DIRS}
)
include_directories(${OPENGL_INCLUDE_DIRS})

Expand Down Expand Up @@ -210,6 +218,8 @@ ament_target_dependencies(${PROJECT_NAME}_lib
image_transport
rviz_visual_tools
situational_graphs_msgs
keyframe_depth_estimator
keyframe_depth_validator
)

# Common source files
Expand All @@ -228,6 +238,8 @@ set(COMMON_DEPENDENCIES
sensor_msgs
geometry_msgs
segmenter_ros
keyframe_depth_estimator
keyframe_depth_validator
message_filters
situational_graphs_msgs
)
Expand All @@ -254,6 +266,7 @@ endfunction()
# ROS2 nodes
add_ros2_node(ros_rgbd src/ros_rgbd.cc)
add_ros2_node(ros_rgbd_inertial src/ros_rgbd_inertial.cc)
add_ros2_node(ros_mono_inertial src/ros_mono_inertial.cc)

# Install other resources
install(
Expand All @@ -267,4 +280,4 @@ install(TARGETS ${PROJECT_NAME}_lib
)

# Ament package configuration
ament_package()
ament_package()
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Once you have installed the required dependencies and configured the parameters,

> 🛎️ Note: The current version of vS-Graphs supports **ROS2 Jazzy** and is primarily tested on Ubuntu 24.04.2 LTS.

> 🛎️ Note: mono-inertial vS-Graphs is currently under development, having a stable version at commit ea5a79819e5f82a9a420cc4cc6b6709197dbf991
## 🐋 Docker

For a fully reproducible, environment-independent setup, see the [Docker](/docker) section.
Expand Down Expand Up @@ -73,3 +74,13 @@ To evaluate vS-Graphs against other visual SLAM frameworks, read the [evaluation
## 🔑 License

This project is licensed under the GPL-3.0 license - see the [LICENSE](/LICENSE) for more details.


## Usage

```bash
ros2 launch vs_graphs mono-imu.launch.py \
launch_keyframe_depth_estimator:=false \
keyframe_depth_validator_use_offline_metric_depth:=true \
keyframe_depth_validator_offline_sync_tolerance_ms:=20.0
```
2 changes: 1 addition & 1 deletion config/Monocular-Inertial/EuRoC.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ IMU.T_b_c1: !!opencv-matrix
0.0, 0.0, 0.0, 1.0]

# IMU noise
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 1.7e-4 #1.6968e-04
IMU.NoiseAcc: 2.0000e-3 #2.0e-3
IMU.GyroWalk: 1.9393e-05
Expand Down Expand Up @@ -81,4 +82,3 @@ Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -1.7
Viewer.ViewpointZ: 0.8
Viewer.ViewpointF: -500.0

1 change: 1 addition & 0 deletions config/Monocular-Inertial/LuxVSD_RS_D435i_640.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ IMU.T_b_c1: !!opencv-matrix
IMU.InsertKFsWhenLost: 0

# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 1e-3 # 2.44e-4 #1e-3 # rad/s^0.5
IMU.NoiseAcc: 1e-2 # 1.47e-3 #1e-2 # m/s^1.5
IMU.GyroWalk: 1e-6 # rad/s^1.5
Expand Down
1 change: 1 addition & 0 deletions config/Monocular-Inertial/RealSense_D435i.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ IMU.T_b_c1: !!opencv-matrix
IMU.InsertKFsWhenLost: 0

# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 1e-3 # 2.44e-4 #1e-3 # rad/s^0.5
IMU.NoiseAcc: 1e-2 # 1.47e-3 #1e-2 # m/s^1.5
IMU.GyroWalk: 1e-6 # rad/s^1.5
Expand Down
1 change: 1 addition & 0 deletions config/Monocular-Inertial/RealSense_T265.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ IMU.T_b_c1: !!opencv-matrix
IMU.InsertKFsWhenLost: 0

# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 0.000005148030141 # rad/s^0.5
IMU.NoiseAcc: 0.000066952452471 # m/s^1.5
IMU.GyroWalk: 0.000000499999999 # rad/s^1.5
Expand Down
1 change: 1 addition & 0 deletions config/Monocular-Inertial/TUM-VI.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ IMU.T_b_c1: !!opencv-matrix
0.0, 0.0, 0.0, 1.0]

# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 0.00016 # rad/s^0.5
IMU.NoiseAcc: 0.0028 # m/s^1.5
IMU.GyroWalk: 0.000022 # rad/s^1.5
Expand Down
1 change: 1 addition & 0 deletions config/Monocular-Inertial/TUM-VI_far.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ IMU.T_b_c1: !!opencv-matrix


# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 0.00016 # 0.004 (VINS) # 0.00016 (TUM) # 0.00016 # rad/s^0.5
IMU.NoiseAcc: 0.0028 # 0.04 (VINS) # 0.0028 (TUM) # 0.0028 # m/s^1.5
IMU.GyroWalk: 0.000022 # 0.000022 (VINS and TUM) rad/s^1.5
Expand Down
3 changes: 2 additions & 1 deletion config/Monocular-Inertial/UniLu_RealSense_D435i_640.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,12 @@ IMU.T_b_c1: !!opencv-matrix
IMU.InsertKFsWhenLost: 0

# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 1e-3 # 2.44e-4 #1e-3 # rad/s^0.5
IMU.NoiseAcc: 1e-2 # 1.47e-3 #1e-2 # m/s^1.5
IMU.GyroWalk: 1e-6 # rad/s^1.5
IMU.AccWalk: 1e-4 # m/s^2.5
IMU.Frequency: 200.0
IMU.Frequency: 400.0

#--------------------------------------------------------------------------------------------
# ORB Parameters
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
%YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters. Adjust them! (RealSense D435 camera - Serial Number: 044322070965)
#--------------------------------------------------------------------------------------------
File.version: "1.0"

Camera.type: "PinHole"

# Rectified Camera calibration (OpenCV)
Camera1.fx: 605.856689453125
Camera1.fy: 606.26611328125
Camera1.cx: 325.021026611328
Camera1.cy: 244.760711669922

# Distortion parameters
Camera1.k1: 0.0
Camera1.k2: 0.0
Camera1.p1: 0.0
Camera1.p2: 0.0

# Camera resolution
Camera.width: 640
Camera.height: 480

# Camera frames per second
Camera.fps: 30

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Transformation from body-frame (imu) to left camera
IMU.T_b_c1: !!opencv-matrix
rows: 4
cols: 4
dt: f
data: [1,0,0,-0.005,
0,1,0,-0.005,
0,0,1,0.0117,
0.0, 0.0, 0.0, 1.0]

# Do not insert KFs when recently lost
IMU.InsertKFsWhenLost: 0

# G2 Step 4 rerun (patched backend): enable IMU-aided relocalization during
# RECENTLY_LOST, ported from BinoSLAM. Off by default upstream in this fork;
# turned on here only, so this file is a rerun-only copy of
# UniLu_RealSense_D435i_640.yaml, not a shared-config edit.
IMU.ImuAidedReloc: 1

# IMU noise (Use those from VINS-mono)
IMU.Threshold: 0.5 # Default: 0.5
IMU.NoiseGyro: 1e-3 # 2.44e-4 #1e-3 # rad/s^0.5
IMU.NoiseAcc: 1e-2 # 1.47e-3 #1e-2 # m/s^1.5
IMU.GyroWalk: 1e-6 # rad/s^1.5
IMU.AccWalk: 1e-4 # m/s^2.5
IMU.Frequency: 400.0

#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------
# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1250

# ORB Extractor: Scale factor between levels in the scale pyramid
ORBextractor.scaleFactor: 1.2

# ORB Extractor: Number of levels in the scale pyramid
ORBextractor.nLevels: 8

# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1.0
Viewer.GraphLineWidth: 0.9
Viewer.PointSize: 2.0
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3.0
Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -3.5
Viewer.ViewpointF: 500.0
91 changes: 91 additions & 0 deletions config/Monocular-Inertial/uHumans2_TESSE.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
%YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters. TESSE simulator (uHumans2 dataset) - left_cam
# Extracted directly from the recorded bag:
# /tesse/left_cam/camera_info -> intrinsics (720x480, fx=fy=415.692, cx=360, cy=240, zero distortion)
# /tf_static (base_link_gt -> left_cam) -> IMU.T_b_c1 (t=[0,0.05,0], q=[0.5,-0.5,0.5,-0.5])
#--------------------------------------------------------------------------------------------
File.version: "1.0"

Camera.type: "PinHole"

# Rectified Camera calibration (OpenCV)
Camera1.fx: 415.69219381653056
Camera1.fy: 415.69219381653056
Camera1.cx: 360.0
Camera1.cy: 240.0

# Distortion parameters (TESSE renders undistorted images)
Camera1.k1: 0.0
Camera1.k2: 0.0
Camera1.p1: 0.0
Camera1.p2: 0.0

# Camera resolution
Camera.width: 720
Camera.height: 480

# Camera frames per second (measured from consecutive /tesse/left_cam header stamps, median dt=0.1s)
Camera.fps: 10

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Transformation from body-frame (imu, "base_link_gt") to left camera ("left_cam")
# Derived from the bag's own /tf_static: t=[0, 0.05, 0], q=[x=0.5, y=-0.5, z=0.5, w=-0.5]
IMU.T_b_c1: !!opencv-matrix
rows: 4
cols: 4
dt: f
data: [0.0, 0.0, 1.0, 0.0,
-1.0, 0.0, 0.0, 0.05,
0.0, -1.0, 0.0, 0.0,
0.0, 0.0, 0.0, 1.0]

# Do not insert KFs when recently lost
IMU.InsertKFsWhenLost: 0

# IMU noise - TESSE's actual simulated noise model is not recorded in the bag;
# these are generic starting values (same family as VINS-mono defaults used in the
# other configs here). Tune IMU.NoiseGyro/NoiseAcc/GyroWalk/AccWalk against the
# /tesse/imu/clean/imu vs /tesse/imu/noisy/imu topics if initialization remains unstable.
IMU.Threshold: 0.5
IMU.NoiseGyro: 1e-3
IMU.NoiseAcc: 1e-2
IMU.GyroWalk: 1e-6
IMU.AccWalk: 1e-4
IMU.Frequency: 200.0

#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------
# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1250

# ORB Extractor: Scale factor between levels in the scale pyramid
ORBextractor.scaleFactor: 1.2

# ORB Extractor: Number of levels in the scale pyramid
ORBextractor.nLevels: 8

# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1.0
Viewer.GraphLineWidth: 0.9
Viewer.PointSize: 2.0
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3.0
Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -3.5
Viewer.ViewpointF: 500.0
Loading