Sensor Fusion Application for
Lane Generation based on Marker Detection using HiKVision Camera and Hesaei OT128 LiDAR

System Overview

Our pipeline consists of the following key steps:

Part 1: Unified ROS2 Camera-LiDAR Synchronization Publisher 

First, we set up a ROS2 node that handles both offline and online sensor data synchronization between Hik camera (images) and Hesai LiDAR (point clouds). The core idea is to ensure these two sensor streams are perfectly aligned in either operating mode - whether processing recorded data in offline mode or handling live streams in online mode.

1.1: Offline Mode: File-based data synchronization from stored datasets

1.2: Online Mode: Real-time streaming with live sensor feeds

1.1: Offline Mode: File-Based Camera–LiDAR Synchronization

The reason of using offline mode is that, in  autonomous vehicle and robotics, testing with live sensors is not always practical — sometimes the hardware is not  accessible, or we  need to revisit previous experiments. That is where offline mode becomes incredibly valuable. It allows to load previously recorded camera and LiDAR data from computer and publish it into ROS2, simulating a real-time sensor stream for development, testing, or debugging as if they were coming from real sensors.

How It Works

For this, we creat a custom node that acts like a player. It's key components are:

• Reads Data from System:

It loads the paired sensor data  from the folders of 

Images: Hik camera frames (.png files)

self.img_path = "/media/mvcam/input"

Point Clouds: Hesai LiDAR scans (.pcd files)

self.pcd_path = "/media/ot128lidar/input"

• Sorts and indexes both streams:

It ensures that images and point clouds are read in sorted order so that their pairing remains consistent across frames.

self.img_files = sorted([...])

self.pcd_files = sorted([...])

• Timer-based sequencing:

The node uses a ROS2 timer (e.g., 1 Hz) to emulate live-streaming behavior while replaying the data sequentially.

self.timer = self.create_timer(1.0, self.publish_data)

• Synchronized Publishing Mechanism

At the heart of the offline mode lies a simple but powerful synchronization strategy that ensures camera images and LiDAR point clouds are published in perfect pairs. For this, the node uses two key components to maintain alignment:

• A 1Hz Timer

  • The timer acts as a metronome, triggering the publish_data() callback once per second.

  • This controlled rate mimics real-world sensor data flow while giving downstream nodes enough time to process each frame.

  • Why 1Hz? It’s a safe default for testing, but we can easily adjust it (e.g., 0.5 for slower playback or 10.0 for stress-testing).

• An Index Counter

  • The self.index variable is the glue that binds each image to its corresponding point cloud.

  • With every timer tick, it increments by 1, moving to the next file pair in the sorted lists.

  • This guarantees that frame_001.jpg always pairs with frame_001.pcd, and so on—no mismatches, no guesswork.

self.timer = self.create_timer(1.0, self.publish_data)  # Pacekeeper

self.index = 0  # The universal "frame ID"

• The Data Conversion Pipeline: Bridging Files to ROS2 Messages:

Every synchronized frame pair undergoes a conversion process to transform raw files into ROS2-ready messages. 

• • 1. Image Processing: From Pixels to ROS2

When an image file (.png) is loaded, OpenCV reads the file into a numpy array, preserving the original BGR color channels. CV Bridge performs the critical conversion to a ROS2 Image message, specifying the bgr8 encoding to maintain color fidelity. This step is essential—ROS2 expects images in a specific binary format, and cv_bridge handles the translation flawlessly.

# Convert OpenCV image → ROS2 message

msg = self.bridge.cv2_to_imgmsg(img, encoding='bgr8')

• • 2. Point Cloud Processing: PCD to PointCloud2

For LiDAR data, the pipeline is more involved:

• Open3D loads the .pcd file, extracting XYZ coordinates into a NumPy array.

• A ROS2 Header is crafted with:

  •  A timestamp (using the node's clock) to synchronize with other sensors.

  • The frame_id "map" to align with TF for coordinate transformations.

• Finally, pc2.create_cloud_xyz32 packs the points into a PointCloud2 message—the standard format for LiDAR data in ROS2.

header = Header()

header.stamp = self.get_clock().now().to_msg()  # Sync with system time

header.frame_id = "map"  # For TF integration

msg = pc2.create_cloud_xyz32(header, points)  # Efficient XYZ format

Each iteration reads the next pair of image and point cloud files and publishes them together with matching timestamps:

While these timestamps are simulated (not original capture times), they maintain synchronicity in the ROS2 ecosystem, which is crucial for our downstream tasks like: Object detection and tracking, Sensor Fusion, SLAM.

• Publishes synchronized topics:

 For each synchronized image–point cloud pair, it publishes:

• An Image message on the /synchronized_image topic.

self.img_pub = self.create_publisher(Image, 'synchronized_image', 10)

• A PointCloud2 message on the /synchronized_pointcloud topic.

    self.pcd_pub = self.create_publisher(PointCloud2, 'synchronized_pointcloud', 10)

This way, any other ROS2 node listening to these topics — like for object detection, visualization, or sensor fusion — can process them just like real-time data.

1.2: Online Mode: Real-Time Camera–LiDAR Synchronization

----------

Part 2: ROS2 Main Node

The heart of the fusion system is a custom ROS2 node, which manages everything from sensor data ingestion and parameter handling to triggering the fusion pipeline. At the beginning, the node initializes and declares essential ROS2 parameters via declare_parameter, which are configurable via launch files or parameter YAMLs. These include paths to the YOLO model weights (model_path), a directory for saving output images (save_path), and the voxel grid resolution used for LiDAR point cloud downsampling (voxel_size). These parameters allow flexibility in deployment, depending on the resolution of LiDAR data or desired performance tradeoffs.

To acquire the sensor data, the node subscribes to two synchronized topics: /synchronized_image for camera frames and /synchronized_pointcloud for LiDAR data. These topics are assumed to be time-synchronized using a prior system, such as a dedicated sync node or timestamp alignment mechanism. Each callback (image_callback and pointcloud_callback) stores the latest received data and optionally triggers the fusion pipeline when both modalities are available.

A notable feature of the node is on-the-fly calibration file generation. Instead of hardcoding calibration matrices within the code, it dynamically creates a temporary calibration file containing essential matrices: the camera projection matrix (P2), the LiDAR-to-camera extrinsic matrix (Tr_velo_to_cam), and the rectification matrix (R0_rect). This temporary file is parsed later during the fusion process and ensures modularity and easier calibration updates.

When processing the LiDAR data, the node uses Open3D to convert the raw ROS PointCloud2 messages into a NumPy-compatible point cloud object. It then applies voxel downsampling to reduce computation overhead while preserving structural features. This step is crucial, especially when dealing with dense point clouds at high frequency. Once both LiDAR and image data are available, the node converts the image to RGB format, extracts 3D points from the downsampled cloud, and passes both to the fusion pipeline housed in the external fus.py module. After processing, it saves and optionally republishes the fused output.

Key Features and Implementation

• Node Initialization and Parameter Setup

The node begins by declaring essential parameters needed for execution. These parameters allow flexible configuration at launch time:

• model_path: Path to the YOLOv8 model weights for real-time object detection.

• save_path: Directory to store the output images with fusion overlays.

• voxel_size: The downsampling resolution for the LiDAR point cloud, typically set to 0.2 meters for balance between performance and detail.

The ROS2 parameter API is used as follows:

self.declare_parameter('model_path', "/path/to/weights.pt")

self.declare_parameter('save_path', "/output/directory/")

self.declare_parameter('voxel_size', 0.2)  # meters

• Sensor Data Handling

To perform fusion, the system must process synchronized streams of image and point cloud data. The node subscribes to two key topics:

• /synchronized_image for receiving frames from the Hikvision camera

• /synchronized_pointcloud for 3D LiDAR scans from the Hesai sensor

Each topic is handled by a dedicated callback, storing the latest received message in memory:

self.img_sub = self.create_subscription(SensorImage, '/synchronized_image', self.image_callback, 10)

self.pcd_sub = self.create_subscription(PointCloud2, '/synchronized_pointcloud', self.pointcloud_callback, 10)

This approach ensures that once both types of data are available, the node triggers the fusion pipeline to process and merge them.

• Calibration Management

To project LiDAR points onto the camera plane accurately, we need a calibration file defining both intrinsic and extrinsic parameters:

• Camera intrinsics (P2): Specifies the focal length and optical center.

• LiDAR-to-camera extrinsics (Tr_velo_to_cam): Defines the spatial transformation from the LiDAR coordinate frame to the camera.

• Rectification matrix (R0_rect): Ensures consistency across image and LiDAR alignment.\Instead of reading from a static file, the node generates a temporary calibration file with hardcoded values. This makes deployment easier and ensures compatibility across different systems.

def create_temp_calibration_file(self):

    calib_content = """P0: 1.063782336002e+03..."""

    temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt')

    temp_file.write(calib_content)

    return temp_file.name

• Point Cloud Processing

The point cloud callback function processes incoming LiDAR data using the Open3D library. It performs the following steps:

• Convert ROS PointCloud2 to NumPy array
  The data is extracted into a structured NumPy array using pc2.read_points.

• Construct Open3D point cloud
  The XYZ coordinates are used to create an open3d.geometry.PointCloud object.

• Apply voxel filtering
  To reduce computational load, voxel grid downsampling is applied, controlled by the voxel_size parameter.

def pointcloud_callback(self, msg):

    points = np.array(list(pc2.read_points(msg, skip_nans=True)))

    cloud = o3d.geometry.PointCloud()

    cloud.points = o3d.utility.Vector3dVector(points[:, :3])

    down_cloud = cloud.voxel_down_sample(self.voxel_size)

• Fusion Execution and Output Publishing / 5. Fusion Pipeline Execution

Once both image and point cloud data are available, the node executes the fusion pipeline via the process_data() method. Here's the workflow:

• Convert point cloud to NumPy format for compatibility with the projection module.

• Convert BGR image to RGB format for compatibility with YOLO and visualization.

• Call the fusion processor (typically implemented in a separate file like fus.py) to:

  • Run YOLO detection on the image

  • Project LiDAR points to the image plane using calibration

  • Overlay the detections and projected points

def process_data(self):

    points = np.asarray(self.latest_pointcloud.points)

    image_rgb = cv2.cvtColor(self.latest_image, cv2.COLOR_BGR2RGB)

    final_result = self.lidar2cam.pipeline(image_rgb, points)

The resulting image, containing bounding boxes and 3D overlays, is then saved to disk and optionally published on a ROS2 topic for visualization in tools like RViz or Foxglove Studio.

Part 3: Fusion Processor 

In this step, we delve into the heart of 3D-to-2D sensor fusion — projecting LiDAR points onto the camera image plane and enhancing our object detection with distance information. To do this, we introduce a dedicated class called LiDAR2Camera, which handles the calibration, projection, filtering, and fusion of data from the two sensors.

We begin by initializing the class with a calibration file. This file contains the intrinsic and extrinsic matrices required to align the LiDAR and camera coordinate systems. The read_calib_file() method parses this file and stores the projection matrix P, the LiDAR-to-camera transform Tr_velo_to_cam, and the rectification matrix R0_rect. These matrices are reshaped into usable NumPy arrays and stored for later use in projection.

The core functionality starts with project_velo_to_image(), where we convert raw 3D LiDAR points into homogeneous coordinates and apply a series of matrix transformations. These operations translate and rotate the LiDAR points into the camera's reference frame and then project them onto the 2D image using the projection matrix. This projection is only valid for points with positive depth (i.e., in front of the camera), so we filter out invalid points accordingly.

To focus only on the points visible within the camera frame, we use get_lidar_in_image_fov(). This function checks each projected point and ensures it's within the boundaries of the image (based on width and height) and beyond a minimum depth to reduce noise. This not only crops irrelevant points but also prepares the subset of points that we want to overlay or fuse with detections.

Next comes show_lidar_on_image(), a utility that overlays the projected 3D LiDAR points on the image using OpenCV. Each point is rendered as a small white dot, providing a quick visual cue about depth-rich areas and verifying the quality of the calibration.

Once we have both the detections from YOLO and the projected LiDAR points, we bring everything together in the lidar_camera_fusion() method. This is where fusion happens. For each bounding box detected by the object detector, we iterate through all projected LiDAR points and check if they fall within that bounding box using a helper method rectContains(). If so, we record their depth (z-coordinate) and optionally highlight them with magenta circles. If enough points are found in the region, we remove outliers using filter_outliers() and compute a representative depth using get_best_distance() with the "average" technique. This results in more robust distance estimates for each detected object.

Finally, the pipeline() method orchestrates the full flow. It first calls show_lidar_on_image() to visualize the LiDAR data, then either uses a pre-loaded YOLO model or loads one on the fly for object detection. Detected bounding boxes are passed into the fusion method, and a final output image is produced where LiDAR and camera insights are unified.

This class is pivotal because it transforms raw sensor data into an enriched, perception-aware visualization that can support higher-level applications like autonomous navigation, collision avoidance, or advanced analytics.

Key Features

• LiDAR2Camera: Unified Fusion Class

To handle the complex task of projecting 3D LiDAR points onto 2D camera images, we encapsulate all logic into a single class: LiDAR2Camera. This class performs end-to-end fusion from calibration loading to point filtering and depth fusion. It’s the backbone of our perception system.

fusion = LiDAR2Camera(calib_file="calib.txt")

By initializing the class, with the calibration file, the system is ready to align LiDAR and camera data based on the intrinsic and extrinsic matrices.

• Accurate 3D-to-2D Projection Using Camera Calibration

To align LiDAR points with the camera view, we extract and reshape three camera’s calibration matrices:

P: Projection matrix

Tr_velo_to_cam: LiDAR to camera transform

R0_rect: Rectification matrix

These are loaded and reshaped. Then, we project the LiDAR points:

P = calib["P2"].reshape(3, 4)

Tr_velo_to_cam = calib["Tr_velo_to_cam"].reshape(3, 4)

R0_rect = calib["R0_rect"].reshape(3, 3)

Once loaded, they are used in

pts_img, pts_rect = fusion.project_velo_to_image(pc_velo)

This projects 3D LiDAR points into 2D image coordinates, ensuring they are in the field of view and valid (i.e., in front of the camera). As a result of this step 3D LiDAR points are transforms into 2D image coordinates for pixel-accurate fusion.

• Filtering LiDAR Points Within the Camera’s Field of View

Not all points are useful — some lie outside the image or are behind the camera. We clean them up using:

pts_img_fov, pc_rect_fov = fusion.get_lidar_in_image_fov(pc_velo, img_shape)

This preserves only those points that are both visible in the image and valid (positive depth).

• Overlaying LiDAR Points on the Camera Image

To verify that projection and calibration are accurate, we draw each 3D point onto the 2D image:

img_with_lidar = fusion.show_lidar_on_image(img.copy(), pts_img_fov)

White circles are drawn at the projected locations, providing immediate visual feedback of depth alignment.

• YOLO + LiDAR Fusion for Distance Estimation /Fusion of YOLO Detections with LiDAR Depth

After detecting objects using YOLO, we fuse the detections with 3D depth info from LiDAR. Each bounding box is checked to find which LiDAR points fall inside it:

for box in bboxes:

    if fusion.rectContains(bbox, point, shrink_factor=0.8):

        depth_points.append(z)

The fusion process assigns a depth value to each object by analyzing the z-values of included LiDAR points.

By integrating depth data, the system gains spatial awareness—understanding both the object and distance of each detected object

• Outlier Removal for Reliable Distance Estimation

Noisy points can distort depth values. We use standard deviation-based filtering:

filtered_points = fusion.filter_outliers(depth_points)

This removes inconsistent or extreme distance measurements.

• Best-Fit Distance Selection Strategy

Then we choose a representative distance using multiple strategies:

depth = fusion.get_best_distance(filtered_points, method="average")

Options include:

closest – pick the smallest distance

average – compute the mean

random – randomly choose one

median (default) – most robust to outliers

This ensures reliable object distance estimation for downstream tasks like obstacle avoidance or navigation.

• Full Fusion Pipeline in One Call

Everything — from projection to fusion and visualization — is s wrapped into one pipeline call:

img_fused = fusion.pipeline(img, pc_velo, model)

This handles projection, filtering, detection, fusion, and visualization — giving you a single frame output with YOLO boxes enriched with LiDAR depth.

• Supporting Helper Functions

• rectContains: Precise Point-in-Bounding-Box Check

Shrinks bounding boxes slightly to avoid edge noise and only include confident LiDAR points inside object boxes. This makes bounding box association with LiDAR points more reliable.

• filter_outliers: Denise Depth Measurements

Removes extreme distance values based on standard deviation, leaving a clean set of depth points per object.

• get_best_distance: Multiple Strategies for Final Depth

 Offers four different depth selection strategies

To summarize, this steps encapsulates the full 3D-to-2D sensor fusion workflow—handling calibration, projection, filtering, and object-depth fusion. It overlays LiDAR points onto camera images and enhances YOLO detections with reliable depth data, enabling spatially aware perception essential for autonomous systems.

Part 4: Lanes Generation and Vehicle Position Estimation based on Marker Inputs

(Utility Functions for Markers based Lane Generation with Vehicle Position Awareness/)

With both image and LiDAR data being published in sync, we now focus on processing these streams to detect lane markers, compute the center-line, evaluate vehicle positioning (left or right), and finally, export this valuable spatial insight into structured CSV logs for downstream analysis.

Our goal of this part is to

• Detect the markers 

• Generate  Lane based on  detected markers

• Separates left and right lanes 

• Computes vehicle position (LHS, RHS)

• Verifies if the vehicle is within lane boundaries

• Logs all frame data with timestamps into a CSV file

We begin with YOLOv8, one of the most powerful real-time object detection models, to identify lane markers from video or image streams. In our setup, we have configured YOLO to detect class 0, which corresponds to custom-trained lane marker objects. We deliberately set a low confidence threshold of 0.001 to capture even faint or distant lane markers, ensuring better sensitivity in varying lighting or visibility conditions. The detections are tracked frame-to-frame using ByteTrack, a popular tracking algorithm supported directly by YOLOv8.

Once the detection are obtained, the next step is separating them into left and right lanes using the RANSAC regression algorithm. RANSAC is robust to outliers, which makes it ideal for filtering noise or incorrectly detected markers. We first attempt to fit a line to the majority of marker points considered to be on the left. If a sufficient number of inliers are found, we assume the remaining outliers to belong to the right lane and apply RANSAC again to fit a second line. This way, we can robustly model two separate lane boundaries from the raw detection data.

With the left and right lanes fitted, we calculate the average x-coordinates of the markers that belong to each lane. Using this information, we estimate the vehicle's position relative to the lane. We assume that the center of the image represents the vehicle's position and compare it with the average positions of the left and right lane markers. If the vehicle's x-center is closer to the left markers, we declare the vehicle as being on the "LHS" (Left-Hand Side); otherwise, it is on the "RHS" (Right-Hand Side). Furthermore, we define a margin around the lane center to determine whether the vehicle remains within safe driving bounds. If the vehicle’s center falls outside this boundary, a warning message is generated.

To ensure the detections are contextually valid and localized within the desired area of interest in the image, we use a helper function called rectContains. This function checks if a given point (typically a detection center) lies within a defined rectangular region. The function also supports a shrink_factor that reduces the bounding region by a percentage, allowing you to ignore markers near the image’s edges. This is particularly useful in scenarios where lane boundaries should be limited to a central strip of the camera view.

Another utility function, filter_outliers, helps clean up noisy distance measurements, particularly useful when evaluating the lateral distance between the vehicle and each detected lane. It works by computing the mean and standard deviation of the input list and removing values that lie more than one standard deviation away from the mean. This statistical filtering helps maintain a reliable understanding of distances even when some detections are erroneous or sporadic.

Finally, the function get_best_distance allows flexible decision-making about which lane distance value to use in computations. Depending on your application, you might prefer the closest detected marker ("closest"), an average distance ("average"), a random marker ("random"), or even a robust median ("median"). This configurable logic makes the lane estimation system adaptable to different environments or tuning strategies.

The system overlays this information directly onto each processed frame using OpenCV. It draws bounding boxes for all detected markers, connects left markers with a purple line, and right markers with a blue line. It also prints a visual status indicator at the top of the frame. If the vehicle is closer to the left, a purple box appears with the text “Vehicle is closer to LHS.” If it is closer to the right, a blue box is shown with the message “Vehicle is closer to RHS.” When the vehicle exceeds the defined safe boundary, a red box appears stating “Outside the boundary.” These overlays help operators quickly understand both the detections and the vehicle’s orientation in real time.

In addition to visual feedback, the system logs every frame’s data to a CSV file. This includes the frame number, current date and time, estimated vehicle position (LHS or RHS), boundary status (within bounds or outside), the average x-positions of left and right markers, and the vehicle’s center x-position. This log is extremely useful for offline analysis, performance evaluation, or training data collection for future models. The CSV file is automatically initialized if it doesn’t exist and is appended to during runtime.

Key Features

• YOLOv8 for Marker Detection and Tracking

At the heart of the system is the YOLOv8 model, which detects lane markers in real-time. We leverage its .track() method combined with ByteTrack for consistent tracking of markers across consecutive frames, even when markers temporarily disappear or get occluded. The low confidence threshold (0.001) ensures even faint markers are captured

predictions = model.track(img, conf=0.001, classes=[0], stream=True, persist=True, tracker="bytetrack.yaml")

This enables smooth and continuous lane marker identification.

• Frame-wise Lane Separation Using RANSAC

Detected markers are not just collected randomly. We use RANSAC regression, a powerful technique robust to outliers, to fit lines to the markers that represent the left and right lanes. The algorithm iteratively selects inliers that fit a model line and rejects points that don’t, helping us accurately separate lanes:

ransac_left = RANSACRegressor(residual_threshold=20).fit(X, y)

left_inliers = [markers[i] for i in range(len(markers)) if ransac_left.inlier_mask_[i]]

If enough markers fit the left lane line, the remaining points are used to fit the right lane line similarly. This separation is essential for understanding vehicle position relative to lanes.

• Vehicle Position and Lane Boundary Analysis

We calculate the average x-coordinate of markers on the left and right lanes and determine the vehicle’s lateral position based on the center of the image frame.

center_x = img_processed.shape[1] // 2

vehicle_position = "LHS" if abs(center_x - left_avg) < abs(center_x - right_avg) else "RHS"

A boundary margin is defined around the centerline. If the vehicle center moves outside this margin, the system flags it as "Outside the boundary," alerting for potential lane departure.

Then, we define a boundary and check whether the vehicle is within it:

•  Informative Visual Feedback

Using OpenCV, the system overlays:

• Bounding boxes around detected markers

• Purple and blue lines connecting left and right lane markers respectively

• Status messages with colored backgrounds indicating vehicle position or warnings, like:

  • "Vehicle is closer to LHS" (purple)

  • "Vehicle is closer to RHS" (blue)

  • "Outside the boundary" (red)

This visualization provides intuitive real-time feedback for operators or developers.

• CSV Logging for Offline Analysis

Every processed frame is logged to a CSV file (lane_data.csv) including:

• Frame number

• Date and time

• Vehicle lateral position (LHS or RHS)

• Boundary status (within or outside)

• Average lane marker x-coordinates

• Vehicle center x-position

This log is invaluable for performance tracking, debugging, or machine learning model retraining.

To complement the main detection pipeline, we use a few helper functions that improve data filtering and validity checks.

• rectContains: Smart Point-in-Rectangle Check

This function checks if a detected point lies inside a defined rectangular boundary. But it does more than a simple check — it includes an optional shrink_factor that slightly reduces the rectangle’s size. Why? To ignore points too close to the edges, which are often unreliable or noisy detections. This filtering step helps ensure that only confidently detected markers are considered, preventing borderline points from skewing results.

• filter_outliers: Removing Noise for Cleaner Data

Real sensor measurements are rarely perfect — outliers and noise can easily distort the analysis. This function tackles that by calculating the mean and standard deviation of distance values, then discards any points that fall outside one standard deviation from the mean. By trimming these outliers, we maintain a cleaner, more reliable set of data points that improve lane fitting and tracking accuracy.

• get_best_distance: Flexible Strategies for Selecting the Ideal Lane Distance

Choosing the “best” lane distance isn’t always straightforward — different scenarios demand different approaches. This function offers multiple strategies to select a representative distance from a list of candidates:

• • • Closest: Picks the smallest distance, useful for cautious estimations.

    • Average: Computes the mean for a balanced perspective.

    • Random: Selects a random value, helpful for testing or introducing variability.

    • Median (default): Finds the median to minimize the effect of extreme values.

 Upcoming: Step 3 - Real-time Navigation & Control Commands

Now that we understand lane boundaries and vehicle position in real-time, we’re ready to move on to Step 3, where we’ll:

• Feed the centerline into a path planner

• Generate velocity and steering commands

• Implement collision checking using LiDAR data

Repo & Code Snippets

We’ll soon publish the core code on GitHub. Here’s a sneak peek of the CSV output per frame

frame_id,timestamp,centerline_x,centerline_y,vehicle_pos_side

frame_001,1716120001.12,[2.3, 3.1, 3.9],[1.0, 1.5, 1.8],LHS

frame_002,1716120001.38,[2.4, 3.2, 4.1],[1.1, 1.6, 1.9],RHS

...

How to run?

Steps:

In Ubuntu, ROS2 Humble, 

•  Open Terminator

•  Right click in it and select three Vertical Split

• And type the following commands 

cd ros2_infra

colcon build

ros2 run infra_pkg lidarpoints_camframe_sync_publisher

ros2 run infra_pkg infraNode