Welcome to the Lichtblick Documentation!

Lichtblick is an open-source application designed to streamline the workflow of automotive engineers and robotics users, helping them achieve engineering excellence with ease.

What is Lichtblick?

Built as a fork of Foxglove Studio, Lichtblick simplifies data visualization and debugging for robotics and automotive applications.

Documentation

We are actively working on new features! User documentation for recent updates will be added soon - Stay Tuned!

Lichtblick empowers robotics teams to efficiently explore, collaborate on, and interpret their robots' data, leading to smarter iterations and faster development cycles.

Workflows

Lichtblick offers a comprehensive suite of developer tools tailored for each phase of the robotics development lifecycle:

Recording

• Capture multimodal data across various supported encoding formats, including MCAP, ROS 1, ROS 2, Protobuf, and more.

Ingestion

• Import recordings from local devices and Edge Sites into a centralized cloud repository for streamlined access.

Processing

• Organize imported data recordings by device, timestamp, and topic for structured analysis.
• Implement retention policies to effectively manage your team's data storage.
• Set up webhooks to seamlessly integrate with your existing data pipeline.

Visualization

• Stream imported data to Lichtblick and third-party tools like Jupyter Notebooks for comprehensive analysis.
• Configure panels to gain insights into your robots' sensory inputs, decision-making processes, and actions.
• Develop shared layouts to facilitate recurring visualization and debugging tasks.

Collaboration

• Enhance logs with metadata and events to simplify search and discovery.
• Leverage collective expertise to triage incidents and conduct root cause analyses.

Connecting to Data

To begin visualizing data, navigate to the Lichtblick dashboard and select a data source option.

Data Source

Lichtblick allows its users to open data via two options, either a local data source or a live data source.

Live Data

Connect to live data sources using Lichtblick's WebSocket, Rosbridge, and Velodyne Lidar integrations for real-time streaming. You can also load remote data files via URL for easy access and processing.

Supported formats

Limitations

When connecting to a live robotics stack, each connection will have different capabilities.

Cross-Origin Resource Sharing (CORS) setup

To load remote data files, they must be hosted on a server or cloud provider that supports Cross-Origin Resource Sharing (CORS) and range requests.

For seamless playback and analysis in Lichtblick, we recommend using cloud providers like Amazon S3, Google Cloud Storage (GCS), or Azure Storage. While you can host files on your own server, setting up CORS and range request support can be complex and time-consuming.

Handling Sensitive Data

If your data is sensitive, use a signed URL to securely grant temporary access. Ensure that the URL points directly to the resource, as redirects will not work with CORS.

• S3 – Sharing objects using presigned URLs
• GCS – Signed URLs
• Azure Storage – Grant limited access with SAS

Since signed URLs expire after a set time, consider configuring your server to generate them dynamically for authenticated users.

Configuring CORS

To enable remote data loading, you need to properly configure CORS. For example, if you're using Amazon S3, you can define CORS settings using Terraform or another infrastructure-as-code tool. Check out the following example of a Terraform config for an S3 bucket:

cors_rule {
  allowed_methods = ["GET", "HEAD", "OPTIONS"]
  allowed_origins = ["https://yourdomain.com"]
  allowed_headers = ["*"]
  expose_headers = ["ETag", "Content-Type", "Accept-Ranges", "Content-Length"]
}

And a Terraform config for a GCS bucket:

cors {
  origin = ["https://yourdomain.com"]
  method = ["GET", "HEAD", "OPTIONS"]
  response_header = ["ETag", "Content-Type", "Accept-Ranges", "Content-Length"]
}

Local Data

Lichtblick enables you to load local data files for visualization.

Supported formats

Getting started

To visualize local files, follow these steps:

• Click "Open local file(s)…" from the dashboard or left-hand menu to browse and select files.
• Open files directly or drag-and-drop them from your OS file manager for quick access.

Frameworks

This section of the documentation provides a guide on how to connect Lichtblick to various data sources and understand the supported data formats and schema encodings. Whether you are working with live sensor data or recorded files, Lichtblick offers flexible options to integrate and visualize your information.

The pages within this section cover the following key areas:

• Connecting to Live Data Sources: Learn how to establish real-time connections with platforms like ROS 1 and ROS 2, as well as specialized sensors like Velodyne.

• Loading Recorded Data: Discover how to load and work with data from various file formats, including ROS 1 .bag files, ROS 2 MCAP files, and PX4 ULog files. Both local and remote file loading options are explained.

• Utilizing Rosbridge: Understand how Rosbridge facilitates communication between ROS environments (ROS 1 and ROS 2) and external applications like Lichtblick via websockets.

• Supported Schema Encodings: Get details on the different message and schema encodings that Lichtblick supports for MCAP-based and websocket data sources, including JSON, Protobuf, FlatBuffers, ROS 1, ROS 2, and OMG IDL.

By exploring these topics, you will gain a comprehensive understanding of how to bring your data into Lichtblick for analysis and visualization.

ROS 1

Load local and remote ROS 1 (.bag) files, or connect directly to a live ROS 1 stack.

Live data

Install ROS 1, and make sure you're connected to the same network as the robot.

Then, in Lichtblick, select "Open connection", either on the initial welcome pop up or via the app bar menu.

Live connections

You can use Rosbridge or Ros foxglove bridge to establish a live connection between Lichtblick and ROS. This enables real-time data streaming, allowing you to interact with ROS topics, services, and parameters directly from Lichtblick.

Native

For direct access to your ROS master and nodes, connect using a native TCP (Transmission Control Protocol) connection.

Ensure you have a working ROS 1 setup and then run roscore in your terminal.

Select "ROS 1" in the "Open data source" dialog, and enter your ROS_MASTER_URI (ROS master's IP and port) and ROS_HOSTNAME:

If you encounter connectivity issues, verify that your ROS stack and Lichtblick have unrestricted network access, as ROS communicates over multiple ports.

If ROS and Lichtblick are running on different machines, refer to the ROS 1 Network Setup documentation to properly configure your environment.

Remote File

For this option just select the "Remote file" in the "Open connection" option and enter the URL to your remote .bag file.

Don't forget to set up CORS if you intend to host the files yourself and load them into Lichtblick.

Local Data

You can load local files for visualization by:

• The "Open local file(s)..." in the initial pop up or the menu on the top left;
• You can drag'n drop the files from your OS file manager;

ROS 2

Load local and remote MCAP files containing ROS 2 data, or connect directly to a live ROS 2 stack.

Live data

Install ROS 2, and make sure you're connected to the same network as the robot.

Then, in Lichtblick, select "Open connection", either on the initial welcome pop up or via the app bar menu.

Live connections

You can use Rosbridge or Ros foxglove bridge to establish a live connection between Lichtblick and ROS. This enables real-time data streaming, allowing you to interact with ROS topics, services, and parameters directly from Lichtblick.

Local Data

You can load local files for visualization by:

• The "Open local file(s)..." in the initial pop up or the menu on the top left;
• You can drag'n drop the files from your OS file manager;

MCAP

Load local and remote MCAP files.

Live Data

In Lichtblick select "Open connection" in the initial pop up or the menu on the top left.

Remote File

For this option just select the "Remote file" in the "Open connection" option and enter the URL to your remote .mcap file.

If you intend to open more than one .mcap file via URL you'll need to change the URL manually, you can check how to do it on this page of our documentation.

Local Data

You can load local files for visualization by:

• The "Open local file(s)..." in the initial pop up or the menu on the top left;
• You can drag'n drop the files from your OS file manager;

When dealing with multiple files, Lichtblick will merge them into a single playback timeline. It's important that they all come from the same data source to avoid potencial data loss or erros during playtime.

PX 4

Load local PX 4 ULog (ulg) files for visualization.

Local Data

You can load local files for visualization by:

• The "Open local file(s)..." in the initial pop up or the menu on the top left;
• You can drag'n drop the files from your OS file manager;

Velodyne

Connect to Velodyne Lidar to load live incoming data.

Select "Open connection" in the "Open data source" menu, and select the "Velodyne Lidar" option.

Enter the local UDP port on which you want to listen for Velodyne packets:

To test your connection, add a 3D panel to your layout, and open your panel settings to toggle on the velodyne topic. You should now see an interactive representation of your Lidar scan in a 3D scene.

Rosbridge

The rosbridge package enables communication between ROS 1 or ROS 2 and external applications via a websocket connection. It allows non-ROS systems, including web applications, to interact with ROS topics, services, and parameters.

Overview

Rosbridge provides a websocket-based API for sending and receiving ROS messages over the network. It is designed for general-purpose communication, enabling integration with web applications, cloud services, and custom remote interfaces.

Key features:

• Supports ROS 1 and ROS 2

• Implements the rosbridge_protocol for structured websocket communication

• Enables non-ROS clients to subscribe, publish, and call services

• Allows remote monitoring and control of ROS systems from web applications

Connecting

A Rosbridge connection uses a standardized protocol to link Lichtblick with your ROS master over websockets. While it does require running an additional ROS node rosbridge_server, it is a good option if a network firewall separates ROS and Lichtblick, as it minimizes port exposure.

To open a Rosbridge connection, you need to have installed rosbridge-suite:

$ sudo apt install ros-noetic-rosbridge-suite

Next, start the websocket server, and review the command printout to determine the port it is listening on (e.g. ws://0.0.0.0:9090):

$ roslaunch rosbridge_server rosbridge_websocket.launch

Finally "Open connection" in the "Open data source" dialog, select "Rosbridge" and then enter the URL to your Rosbridge server:

To test if everything is working well, you can check the topics tab on the left sidebar.

Example connection

As you can see below this enables real-time communication between ROS and Lichtblick using rosridge websocket.

In the example, the ROSBridge server runs inside a Docker container, exposing a websocket interface that Lichtblick can connect to. Data is exchanged by publishing and subscribing to ROS topics via websocket messages, allowing seamless integration between ROS and external applications.

Schema encodings

Both MCAP-based and websockets sources support several message and schema encodings.

JSON

For JSON data, use schema encoding "jsonschema" and message encoding "json".

Connections via websocket require schemas to be JSON Schema definitions with "type": "object".

Each message must be UTF-8 encoded JSON representing an object. Any binary data should be encoded as a base64 string within the JSON object. The schema should specify this using "contentEncoding": "base64" (e.g., { "type": "string", "contentEncoding": "base64" }).

Protobuf

For Protobuf data, use schema encoding "protobuf" and message encoding "protobuf".

Lichtblick requires the schema data to be a binary FileDescriptorSet. For websocket connections, this binary data must also be base64-encoded since it is represented as a string.

Lichtblick also expects schemaName to be one of the message types defined in the FileDescriptorSet.

FlatBuffers

For FlatBuffers data, set the schema encoding to flatbuffer and the message encoding to flatbuffer.

Lichtblick requires the schema data to be a binary-encoded FlatBuffers schema (.bfbs) file, generated from the source FlatBuffers schema (.fbs) file. For websocket connections, this schema must be base64-encoded since it is represented as a string.

Use the FlatBuffers schema compiler to generate .bfbs files:

flatc --schema -b -o <PATH_TO_BFBS_OUTPUT_DIR> <PATH_TO_FBS_INPUT_DIR>

ROS 1 and ROS 2

For ROS 1 data, use the schema encoding ros1msg and the message encoding ros1.

For ROS 2 data, use the schema encoding ros2msg or ros2idl and the message encoding cdr.

Lichtblick requires the schema data to be a concatenation of the referenced .msg or .idl file along with its dependencies. For details on the concatenated format, refer to the MCAP specific documentation.

OMG IDL

For IDL schemas with CDR data, use the schema encoding omgidl and the message encoding cdr.

To encode OMG IDL schemas into MCAP, follow the conventions outlined in the MCAP Format Registry.

Multiple files

Starting with version 1.10.0 Lichtblick introduced support for opening multiple files simultaneously, an important feature that enhances the user experience and streamlines the visualization and analyzing process using Lichtblick.

In this section, we'll guide you through how to use this feature, discuss its behavior and share a few tips to get the most out of it.

A Lichtblick session with multiple files opened will look like this:

How to open multiple files

Just like with single files, Lichtblick allows users to open multiple files, whether they are stored locally or accessed remotely.

Opening multiple local files

To open local files, as described in our local data documentation, you can either select the files using "Open local file(s)…" option or simply drag-and-drop them into Lichtblick.

Alternatively, you can open multiple files before launching the application by using our CLI commands. To learn more about how this works, see this section of our documentation.

Opening multiple files from remote sources

To open multiple files from remote sources, you'll need to manually construct the URL that includes references to each file source. Below is an explanation of how Lichtblick handles multiple files via URL

A link that opens multiple files uses two specific query parameters:

• ds: Short for datasource, this parameter identifies the type of data source. In the case of remote files, it'll always be:

  ?ds=remote-file

• ds.url: This parameter contains the actual URL of the remote file to be opened. For example:

  &ds.url=http://localhost:8081/MCAP_1.mcap

To open multiple files you can repeat the ds.url parameter for each file, Lichtblick will parse them all and load the correspoding files. So, for instance, an URL that opens three files will look like:

  http://localhost:8080/?ds=remote-file&ds.url=http://localhost:8081/MCAP_1.mcap&ds.url=http://localhost:8081/MCAP_2.mcap&ds.url=http://localhost:8081/MCAP_3.mcap

Don't forget to set up CORS if you intend to host the files yourself.

Behavior

The current implementation aims to deliver the capability to load multiple .mcap files when they originate from the same source file.

When handling multiple files, Lichtblick organizes messages based on their timestamps (log time), maintaining chronological order across all files. If any gaps are detected in the timeline, they will be visually reflected in the UI, preserving the integrity of the data stream.

From the same source origin

When multiple files originate from the same data source, with identical structures (e.g., topics, schemas), Lichtblick will merge them into a single timeline, as if they were one continuous recording.

This is the most stable and recommended usage for multiple files in Lichtblick. It works especially well when files are split by time (e.g., consecutive recordings), allowing for seamless chronological playback and analysis.

From different sources

Lichtblick will attempt to load and render all data, even if the files have different topics, schemas, or structures and it will try to merge all data into a unified view.

While this is possible, it's less stable than working with files from the same source. Mismatches in structure or overlapping content can lead to visual inconsistencies or data interpretation issues.

Limitations

When dealing with multiple files, there are some important limitations to be aware of:

• Messages that share the exact same timestamp across files can lead to unexpected behavior. While Lichtblick will attempt to render all messages, panels that rely on a single value at each moment, such as the Raw Message panel, will only display the last-loaded message for that timestamp. On the other hand, panels that support cumulative data, like the Map, Plot, or 3D panels, will try to render everything available at that moment in time.

• It's also important to understand that Lichtblick merges all loaded files as if they came from a single source. Because of this, there is no way to distinguish which file a particular message originated from once the data is loaded. The interface treats the merged data as a unified timeline.

• Schema consistency is another critical factor. Lichtblick expects schema definitions to be unique across all files. If multiple files define schemas with the same name but different structures, only the first definition encountered will be used. This can cause panels to misinterpret the data, leading to incorrect rendering or visual glitches. To avoid this, we strongly recommend using consistent and non-conflicting schema definitions across all files.

Finally, Lichtblick includes built-in alerts to notify users of conflicting scenarios, such as duplicate schemas or ambiguous timestamps, that could impact the accuracy or integrity of the data.

Settings

General

Here you can find the general settings that allows you to configure core preferences, such as language, appearance, and default behaviors. These settings help customize your experience to better fit your needs.

Below is a list of all available options and their purposes:

Extensions

The Extensions menu allows users to manage and install additional features to Lichtblick. Under the LOCAL section, you’ll find extensions that are already installed. The DEFAULT section lists available extensions that can be installed to enhance functionality.

Recently we added a search bar to the extensions menu to facilitate the experience when managing extensions.

When an extension is selected, a menu opens with options to install or uninstall the extension. This menu also allows users to view the extension's README and Changelog.

Experimental features

The Experimental Features section includes early-stage functionalities that could be unstable and are not recommended for daily use. Options here allow users to test new capabilities, such as memory usage indicators, before they become part of the stable release. Use with caution, as these features may impact performance or cause unexpected behavior.

• When the Memory usage indicator checkbox is selected, an indicator appears on the top bar showing the percentage of memory in use. Hovering over the indicator displays the actual memory usage in MB.

About

This section provides information about the software version and legal details. Here, you can check the current version and access the license terms

Visualization

Lichtblick offers a comprehensive suite of visualization tools to help you analyze and interpret your robotics data effectively.

Getting Started

To begin visualizing your data, connect to a data source and open a panel.

Open Data Source:

• Click "Open data source" in the left-hand menu.
• Choose from available options: live data or local file.

Opening a Panel:

• Click "Add panel" in the dashboard or left-hand menu.
• Select the desired panel type (e.g., 3D, Raw Message, Image).

Desktop-only features

Connecting to data

• open a native ROS 1 connection
• connect to a Velodyne LIDAR hardware

Extensions

• Install via registry

Interface Overview

Lichtblick's interface is designed for intuitive navigation:

App Menu: Connect to a data source, toggle sidebars, or access resources.
Users Menu: Go to app settings, extensions catalog, experimental features, licenses, and version.
Add Panel: Add a new panel to your current layout.
Layout Menu: Save your workspace view as a layout and share it with teammates.
Left Sidebar: Edit panel settings, view data source topics, and troubleshoot connection issues.
Right Sidebar: Set layout-wide variables, view playback metrics.

Sidebars

Panel sidebar

Edit settings for any selected panel

Topics sidebar

View all topics available in the data source, along with their data types and message rates

Problems sidebar

See a list of playback errors to troubleshoot

Variables sidebar

Set layout-wide variables that can be used in different panels with the message path syntax

System Requirements

Lichtblick supports Windows, macOS, and Linux on both web and desktop platforms.

For the web application, use Chrome v111 or later.

For the desktop application, download the latest version for your operating system - check our latest release here: Lichtblick Releases

Playback

Lichtblick enables seamless navigation through both local and remote datasets using its playback controls.

Message Sequencing

Messages within Lichtblick are arranged and played in order of their log timestamps. The log timestamp typically represents the moment a message was captured but can be adjusted to reflect the most relevant time context for your analysis. Selecting an appropriate timestamp is crucial, as external factors such as network delays, buffering, or batch processing can introduce time discrepancies.

In robotics, messages often carry multiple timestamps beyond the log time. Lichtblick’s Plot and State Transitions panels allow users to organize data using alternative timestamps:

Message Handling and Optimization

Lichtblick is designed to efficiently manage large-scale robotics data, ensuring smooth navigation and playback.

Retrospective Message Fetching

When seeking a specific point in the data stream, it is unlikely that every subscribed topic has a message at the exact timestamp selected. To maintain data consistency across panels, Lichtblick implements a retrospective search for the most recent message on each topic. This ensures that when navigating to arbitrary time points, all active panels retain relevant and contextually accurate data.

Persistent Data for Latched Topics

By default, Lichtblick retains the latest received messages for all topics when handling ROS 1 .bag files, MCAP files, or direct Lichtblick data streams. When navigating through time, Lichtblick retrieves and displays the most recent messages from all topics, even if they were recorded minutes before the selected timestamp. This feature allows panels to visualize infrequently published data reliably, ensuring continuity even when reviewing sparse datasets.

Data Preloading for Enhanced Visualization

Certain Lichtblick panels, such as the Plot and Map panels, benefit from accessing data spanning the entire recording duration. Preloading enables these panels to analyze complete historical trends, detect anomalies, and observe long-term behavioral patterns in robotic systems.

Even panels that primarily display the latest data, such as the 3D panel, take advantage of preloaded data for precise rendering. For example, the 3D visualization panel preloads transformation messages to correctly position objects in a unified coordinate frame. In robotics, multiple reference frames (e.g., robotic arm joints, autonomous vehicle sensors) must be aligned for accurate visualization. Preloading ensures that Lichtblick has access to all necessary transform data, preventing inconsistencies in rendering dynamic robotic systems.

Shortcuts

Space - pause or play
shift + ⬅️ - seek backward 10ms
shift + ➡️ - seek forward 10 ms
⬅️ - seek backward 100ms
➡️ - seek forward 100ms
Alt + ⬅️ - seek backward 500ms
Alt + ➡️ - seek forward 500ms

Message Schemas

Lichtblick relies on structured message formats to ensure accurate data visualization and processing. By adhering to Lichtblick's schema standards, users can leverage the platform's robust visualization tools effectively.

Supported Schema Formats

Lichtblick supports a variety of message formats, enabling seamless integration with diverse data sources. The supported formats include:

• Protobuf
• JSON Schema
• ROS 1
• ROS 2
• TypeScript
• FlatBuffers

If your existing message formats differ from these, Lichtblick provides tools to convert them into compatible schemas using a message conversion extension.

Working with Protobuf and JSON Schema

To use Protobuf or JSON Schema with Lichtblick, follow these steps:

1. Protobuf: Include the necessary .proto files in your project. These files can be used to publish data via a WebSocket connection or log data into an MCAP file.
2. JSON Schema: Similarly, copy the required .json schema files into your project.

Note on Protobuf Time Formats: When using google.protobuf.Timestamp or google.protobuf.Duration, Lichtblick represents time values with sec and nsec fields (instead of seconds and nanos). This ensures consistency across time and duration formats in user scripts, message converters, and other platform components.

For JSON Schema integration, you can import schemas directly using the @foxglove/schemas npm package:

import { CompressedImage } from "@foxglove/schemas/jsonschema";

Lichtblick also offers WebSocket libraries for real-time data handling in Python, JavaScript, and C++, as well as MCAP writers for logging pre-recorded datasets. For a practical example, refer to our blog post on Recording Robotic Data with MCAP, which demonstrates how to use the MCAP C++ writer to log Protobuf data.

Schemaless JSON Support

Lichtblick supports schemaless JSON messages through MCAP. To send JSON data without a schema:

1. Set the channel's message encoding to json.
2. Assign the schema ID as 0 to indicate no associated schema.

For more details, consult the MCAP Specification on Channels.

ROS Integration

Lichtblick provides dedicated ROS message packages for both ROS 1 and ROS 2. To integrate:

1. Install the foxglove_msgs package:

2. Install the appropriate package for your ROS version:

sudo apt install ros-noetic-foxglove-msgs # For ROS 1

sudo apt install ros-galactic-foxglove-msgs # For ROS 2

3. Import the necessary schemas into your ROS project to begin publishing data:

from foxglove_msgs.msg import Vector2

...
msg = Vector2()
msg.x = 0.5
msg.y = 0.7

TypeScript Integration

Lichtblick schemas can be imported as TypeScript types, enabling type-checking and message validation. Here’s how to use them:

1. In TypeScript Projects: Import types directly from the @foxglove/schemas npm package:

import { Point2 } from "@foxglove/schemas";

const myImage: Point2 = { x: 1, y: 2 };

These types are compatible with JavaScript WebSocket or MCAP projects and can be used when writing custom data transformation scripts within Lichtblick's User Scripts panel.

Layouts

Lichtblick layouts enable users to design and save customized workspaces tailored to specific tasks or workflows. These layouts can be reused for recurring projects or shared with team members working on similar challenges.

Use Cases for Layouts

Layouts are highly versatile and can be adapted to various engineering and development scenarios. For instance:

• Perception Engineers: Create layouts for sensor calibration tasks.
• Planning Engineers: Design layouts to visualize and analyze routing algorithm outputs.
• Controls Engineers: Configure layouts to monitor and debug robot kinematics.

The Layouts menu provides all the tools needed to create, modify, and share layouts, ensuring a streamlined workflow.

Layouts Menu Overview

Creating a Layout

To create a new custom workspace:

1. Navigate to the Layouts menu.
2. Select Create new layout.

Customization Options:

• Add and arrange panels: Organize panels to suit your workflow.
• Adjust panel settings: Configure individual panel properties.
• Configure playback settings: Tailor playback behavior for your data.
• Set and manage variable values: Define and control variables within the layout.

Editing Layouts

When switching layouts after making changes to your current workspace, you will be prompted with the following options:

• Save changes: Save your modifications to the layout.
• Revert: Discard changes and restore the last saved version.

Web Version Considerations

For users accessing Lichtblick via the web version, it’s important to note that layouts are stored in IndexedDB, a client-side NoSQL database. Since browser data, including IndexedDB, can be cleared during cache or history deletion, it is recommended to export and save your layouts as JSON files to prevent data loss.

To ensure your layouts are preserved:

1. Export your layout as a JSON file using the Export... option in the layout’s context menu.
2. Store the exported file securely for future use or re-import it as needed.

This practice ensures that your custom layouts remain accessible even if browser data is cleared.

Importing and Exporting Layouts

Exporting a Layout

To export a layout as a JSON file:

1. Open the layout’s context menu.
2. Select Export....

Alternatively, you can access this option through the View submenu in the app menu (Export layout to file...).

Importing a Layout

To import a previously exported layout:

1. Navigate to the Layouts menu.
2. Select Import from file....

This option is also available in the View submenu in the app menu (Import layout from file...).

Additional Layout Actions

Each layout includes a Details menu, which provides options to:

• Rename: Change the layout’s name.
• Duplicate: Create a copy of the layout.
• Delete: Remove the layout permanently.

Batch Actions

To perform actions on multiple layouts simultaneously:

• Use Cmd (Mac) or Ctrl (Windows/Linux) to select multiple individual layouts.
• Use Shift to select a contiguous range of layouts.
• Right-click any selected layout and use the context menu to apply batch actions.

Open Lichtblick via CLI with a Layout Parameter

Desktop only

Once you have created and saved a layout, it can be referenced as a parameter when launching Litchblick via CLI.

lichtblick --defaultLayout="layout_example"

/Applications/Lichtblick.app/Contents/MacOS/Lichtblick --defaultLayout="layout_example"

C:\Users\<USER>\AppData\Local\Programs\lichtblick\Lichtblick.exe --defaultLayout="layout_example"

Automatically loading layouts from user directory

Desktop only

Lichtblick supports automatically importing layout files from a predefined user directory. This allows users to preload layouts on startup, making it easier to maintain a consistent workspace configuration across sessions or machines.

How It Works

When Lichtblick starts, it checks the user directory for layout files with the .json extension. These must be valid Lichtblick layout files. If a layout from the directory is not already present in your saved layouts, it will be imported automatically.

This mechanism is particularly useful for scenarios where:

• Teams want to distribute a standard set of layouts.
• Users frequently open Lichtblick on different systems or in new environments.
• Layouts need to be preloaded when launching via CLI or automation scripts.

Directory Path

*You can navigate to these paths and place your exported .json layout files directly into the folder.

Behavior and Characteristics

• Layout files are imported in alphabetical order.
• If no layout was previously selected, the first layout (by alphabetical order) will be automatically activated on startup.
• Layouts are identified by name:
  • If a layout already exists with the same name, it will not be re-imported.
  • If the layout is renamed, it will be treated as a new layout and imported again.
  • If you want to force an update to a layout, delete the existing one before restarting Lichtblick or rename the updated file.

This automatic import system ensures that your custom layouts can be effortlessly reused, even in fresh environments or shared workstations.

By leveraging Lichtblick's layout features, you can create efficient, reusable workspaces tailored to your specific needs, enhancing productivity and collaboration.

Panels

Overview

Panels in Lichtblick are flexible, modular elements that allow you to visualize and interact with data. You can customize and organize these panels within your layout. To add a panel, use the "Add Panel" menu to select a new panel, or drag and drop the panel directly into your layout.

Once added, you can easily move panels around by dragging their top bar.

Each panel's top bar has quick access to:

• Menu (represented by 3 dots) for common actions like panel splitting or changing the panel type
• Settings accessed through the cog icon to adjust the panel's configuration

Customizing Panels

To edit a panel, click on the cog icon on its top bar to open the settings in the left sidebar. Each panel will be highlighted with a colored border when selected.

The sidebar allows you to filter the topics from your data source, and you can drang and drop topic results into panels for quick visualization.

Topics can be mapped to specific panel types like:

• Raw messages and table panel for detailed message views
• Image panel for visual topics
• Plot and State Transiton panel for message path with graph-related data.

For selecting multiple message paths, hold shift for a range, or Ctrl (or Cmd on macOS) for multiple non-adjacent items.

Plot

The Plot Panel in Lichtblick is a user interface component designed to facilitate the visualization and control of plotting data. It allows to view, modify, and export graphical representations of datasets or modeling results.

Settings

General

Legend

Plot series legend.

Y Axis

X Axis

Series

To remove a series from the Panel you use the x, next to each data series, like it's shown below.

To enter the data you wanna plot use the message path syntax, Lichtblick will also show suggestions that exits on the topics available.

You can also hover the plot to see the details in a tooltip. A vertical bar will appear, as well as a yellow marker on the playback timeline. If clicked the playback will jump to that point on the timeline.

Download CSV data

With right click mouse button an option to download the plotted as .csv will appear.

Raw Messages

The Raw Messages panel in Lichtblick is a powerful debugging and inspection tool that enables users to visualize raw ROS or MCAP messages in a structured and interactive JSON format. It is particularly useful for understanding message structures, tracking state changes over time, and drilling into specific message fields for advanced diagnostics or visualizations.

As new messages are received for a specific path, the panel tree will show just the last message. It's also possible to expand and collapse keys, and that will persist across playback

Features

1. Diff mode

Compares messages showing additions (green), deletions (red), and changes (yellow) to their fields across 2 categories:

• previous message - Compare the immediately previous message on the same message path;
• custom - Compare different topic messages in the same timestamp;

2. Advanced Filtering

Users can define custom filters using logical expressions to isolate specific parts of the message. Supported features include:

• Array filtering, e.g., signals[0]
• Logical operators, e.g., fields[:]{offset<5}
• Global variables, such as $last, $index, or other runtime context helpers

Filters are written in a JavaScript-like syntax, parsed and validated in real time.

3. Hyperlinks to message schmemas documentation

A link to documentation about the selected schema is available at the top.

4. Contextual Actions (Right-click or Click)

Hovering over a field provides contextual actions, such as:

• Plot this variable

• Open state transition visualization

• Copy full path to field

• Set as filter base

This improves usability when exploring complex or unfamiliar messages.

5. Structured JSON Message View

Each message is displayed as a JSON object, preserving full structural fidelity. The panel supports:

• Primitive types (string, number, boolean)
• Arrays, with collapsible sections
• Objects, with recursive expansion
• Enumerations, shown with both label and numeric value (e.g., "state": "RUNNING (1)")

Settings

Best Practices

• Use filters to reduce noise when debugging high-frequency topics.
• Combine drag-and-drop with plots for rapid visualization of numerical data.
• Enable diff mode to track how actuator or decision variables evolve.
• Use contextual actions for quicker navigation between panels.

Playback Performance Panel

Overview

The Playback Performance panel provides a real-time visualization of playback performance metrics during the execution of a playback session. It displays four primary metrics over a sliding time window:

• Playback Speed relative to real-time (× realtime)
• Frame Rate (fps)
• Bag Frame Duration (ms bag frame)
• Bandwidth Usage (Mbps)

These metrics are useful for diagnosing playback behavior and ensuring performance meets system expectations.

Features

🔄 Real-time updates as playback state changes

📈 Sparkline graphs for each metric over the past 5 seconds

🧠 Smoothed display using averages to help detect trends

🎯 Compact and interactive UI, integrated with existing Suite UI components

Metrics Explained

Note that the MaxDisplayedValue only affects the vertical axis range of the sparkline, so values above 30 will still be calculated and shown, but they'll clip in the sparkline view. It does not limit the actual data values nor enforce any cap.

User Scripts Panel

User Scripts enable you to create synthetic topics in Lichtblick by processing and reshaping existing messages. These scripts are especially useful for extracting insights, generating visualization-friendly data, or filtering out irrelevant noise from playback or full-range messages.

This chapter walks you through the process of writing and debugging your own scripts in TypeScript, with examples and tips to help you get started.

What Are User Scripts?

User Scripts are custom functions written in TypeScript that run inside Lichtblick and allow you to:

• Filter messages (e.g., only show error logs).
• Aggregate values across multiple messages (e.g., average sensor readings).
• Transform messages into visualizations (e.g., markers or state transitions).
• Synthesize new topics from multiple inputs.

Scripts can process two types of data:

• Playback messages – Live, frame-by-frame data streamed into panels like Raw Messages or 3D.
• Range-loaded messages – Complete data over a playback range, used in panels like Plots or State Transitions.

How User Scripts Work

Each script requires three parts:

1. Input topics – Topics you want to consume.
2. Output topic – A synthetic topic where transformed messages are published.
3. Script function – The transformation logic.

Here’s a basic script that simply republishes every /diagnostics message under a new topic:

import { Input, Message } from "./types";

export const inputs = ["/diagnostics"];
export const output = "/custom_script/diagnostics_mirror";

export default function script(
  event: Input<"/diagnostics">
): Message<"diagnostic_msgs/DiagnosticArray"> {
  return event.message;
}

Creating Your First Transformation

Let’s say you want to highlight high-temperature readings from a sensor topic. Here's how you could write that:

import { Input } from "./types";

type Output = {
  timestamp: number;
  temperatureC: number;
  alert: boolean;
};

export const inputs = ["/sensors/temperature"];
export const output = "/custom_script/high_temp_alerts";

export default function script(
  event: Input<"/sensors/temperature">
): Output | undefined {
  const { temperatureC, timestamp } = event.message;

  if (temperatureC < 70) return; // Don't publish normal readings

  return {
    timestamp,
    temperatureC,
    alert: true,
  };
}

This script suppresses normal messages and only publishes when temperatures are 70°C or higher.

Using Multiple Input Topics

To combine data from different topics—say, GPS and IMU—you can use a union type:

import { Input, Message } from "./types";

export const inputs = ["/gps", "/imu"];
export const output = "/custom_script/combined_pose";

type UnionGPSandIMUOutput = {
  lat: number;
  lon: number;
  orientation: number[];
};

let latestGPS: Message<"foxglove.LocationFix"> | undefined;
let latestIMU: Message<"IMU"> | undefined;

export default function script(
  event: Input<"/gps"> | Input<"/imu">
): UnionGPSandIMUOutput | undefined {
  if (event.topic === "/gps") {
    latestGPS = event.message;
  } else {
    latestIMU = event.message;
  }

  if (!latestGPS || !latestIMU) return;

  return {
    lat: latestGPS.latitude,
    lon: latestGPS.longitude,
    orientation: [latestIMU.q.w, latestIMU.q.x, latestIMU.q.y, latestIMU.q.z],
  };
}

Type Safety and Message Types

User Scripts are written in TypeScript for type safety. This ensures that your script matches the structure of each message and catches schema mistakes early.

You can use known ROS message types by wrapping them in the Message<"type_name"> utility:

import { Input, Message } from "./types";

export default function script(
  event: Input<"/cmd_vel">
): Message<"geometry_msgs/Twist"> {
  return event.message;
}

If you're unsure of the message format, check the types.ts utility file in your script environment. This file is auto-generated and contains all known message types in your dataset.

Skipping Messages

You can simply return undefined when a message shouldn't be published. This is useful for filtering:

export default function script(
  event: Input<"/status">
): { error: boolean } | undefined {
  if (event.message.status !== "ERROR") return;

  return { error: true };
}

Logging for Debugging

Add log(...) calls in your script to inspect values during execution:

log(event.message.temperatureC, "Temperature received");

return {
  // your transformed output
};

Avoid logging functions or very frequent values in high-rate topics, as this can degrade performance.

Using Global Variables

Global variables can be used across invocations of your script, allowing you to create very dinamyc scripts. Those variables are placed as the second parameter of the script function and are read-only on user scripts.

import { Input, Message } from "./types";

export const inputs = ["/odom"];
export const output = "/custom_script/odom_z_global_variable";

type Output = {
  odom_accel_z: number;
  odom_accel_z_fixed: number;
};

type GlobalVariables = { multiplier: number };

export default function script(
  event: Input<"/odom">,
  globalVariables: GlobalVariables
): Output {
  const { accel } = event.message;
  const { multiplier } = globalVariables;
  return {
    odom_accel_z: accel.z,
    odom_accel_z_fixed: accel.z * multiplier,
  };
}

Using @foxglove/schemas Types

Lichtblick enables to use a library of known types that can be imported directly for structured output. While Lichtblick is still migrating its secondary dependencies, Foxglove schemas can be used as following:

import { Input } from "./types";
import { Color, Pose } from "@foxglove/schemas";

export default function script(event: Input<"/pose">): {
  pose: Pose;
  color: Color;
} {
  return {
    pose: event.message.pose,
    color: { r: 0, g: 1, b: 0, a: 1 },
  };
}

Useful Panels for Scripting

Variables

Variables in Lichtblick allow users to define global values that can be reused across multiple panels within a layout. This feature simplifies updates and ensures consistency throughout your workspace. Variables can store primitive data types such as strings, numbers, or booleans. They can also be structured as arrays (e.g., ["x", 2, false]) or maps (e.g., { "x": 2, "y": false }).

To manage variables, access the Variables tab in the sidebar, where you can view, add, and modify them.

Using Variables

Variables are referenced using the $ prefix. For example, a variable named my_global_var is accessed as $my_global_var.

In Message Paths

Panels that support message path syntax—such as Raw Messages, Indicator, Plot, and State Transitions—can leverage variables to dynamically filter or slice data. This enables flexible and interactive data visualization.

Example Workflow:

1. Create a variable named my_ID in the Variables tab and set its value to 101.
2. In a Raw Messages panel, use the path /my_objects.objects[:]{id==$my_ID} to inspect the object with id == 101.
3. In a Plot panel, add /my_objects.objects[:]{id==$my_ID}.velocity as the y-axis value to plot the velocity of the selected object.

In User Scripts

User scripts can reference variables but cannot modify them. When a script is executed, it receives all variables as an object, allowing for dynamic data processing.

Accessing and Modifying Variables in Extensions

Custom extension panels can interact with variables in two ways:

1. Accessing Variables:
   Extensions can access variables using the extension API RenderState. This allows panels to seamlessly integrate with user-defined values, enabling dynamic and context-aware visualizations.

2. Modifying Variables:
   Extensions can also create and modify variables programmatically using the extension API PanelExtensionContext. This capability is useful for updating variables based on user interactions or data processing within the extension.

For example, a custom panel could update a variable to reflect the current state of a simulation or user input, ensuring that the layout remains responsive and interactive.

Updating Variables

Variables can be updated in two ways:

1. Manually: Edit variable values directly in the Variables tab.
2. Dynamically: Use interactive elements in the 3D panel or Variable Slider panel to adjust variable values in real time.

Keyboard Shortcuts

• Press ] to toggle the visibility of the right sidebar.
• Use input + ↑ to increment numeric variable values.
• Use input + ↓ to decrement numeric variable values.

By utilizing variables, you can create dynamic and interactive layouts in Lichtblick, streamlining your workflow and enhancing data analysis capabilities.

Extensions

Lichtblick’s extensibility allows you to tailor the platform to your team’s unique workflows. By developing custom extensions, you can create specialized panels, convert custom message schemas into Lichtblick-supported formats, and alias topic names for seamless integration and visualization.

Once your extension is built and installed, you can manage it through the app settings, where all available and installed extensions are listed.

Custom Panels

While Lichtblick offers a robust set of built-in panels for robotics data visualization and debugging, custom panel extensions enable you to create domain-specific solutions tailored to your needs. These panels can:

• Subscribe to messages from various topics.
• Publish data.
• Display information in a way that aligns with your workflow.

Custom panels are ideal when your visualization or interaction requirements go beyond the capabilities of the built-in panels.

Example: Custom Panel

export function activate(extensionContext: ExtensionContext) {
  // Register a new panel
  extensionContext.registerPanel({
    name: "example-panel",
    initPanel: initExamplePanel,
  });
}

Custom Camera Models

Custom camera model extensions enable support for specialized lens distortion or projection models beyond Lichtblick’s built-in camera model. By registering a custom camera model, you can ensure that camera images with unique distortion (e.g. fisheye or other wide-angle lenses) are interpreted correctly in Lichtblick’s Image panel. This allows the Images panel to accurately render images using your custom projection logic, just as it does for the standard pinhole camera model.

Example: Custom Camera Model

import { CylinderCameraModel } from "./CylinderCameraModel";
import { ExtendedExtensionContext, CameraInfo } from "./lichtblick-suite.types";

export function activate(extensionContext: ExtensionContext): void {
  extensionContext.registerCameraModel({
    name: "CylinderCameraModel",
    modelBuilder: (cameraInfo: CameraInfo) => new CylinderCameraModel(cameraInfo),
  });
}

You can find out more details about Custom Camera Models and its usage on this page: Custom Camera Models

Message Converters

Message converter extensions allow you to transform messages from one schema to another, making them compatible with Lichtblick’s built-in visualization tools. For example, you can convert custom GPS messages into lichtblick.LocationFix messages for visualization in the Map panel.

Note: Message converters run on-demand when a panel subscribes to a topic.

Example: Message Converter

export function activate(extensionContext: ExtensionContext) {
  // Register a new message converter
  extensionContext.registerMessageConverter({
    fromSchemaName: "sensors.MyGps",
    toSchemaName: "lichtblick.LocationFix",
    converter: (inputMessage) => {
      // Logic to convert sensors.MyGps messages into lichtblick.LocationFix messages
    },
  });
}

Topic Aliases

Topic alias extensions enable you to rename topics in your data source to new names. Lichtblick panels can subscribe to both the original and aliased topics, providing flexibility in how you organize and visualize your data.

Example: Topic Aliases The registerTopicAliases function maps original topics to new names based on the current layout’s global variables. It automatically re-executes when the data source’s topics or global variables change.

import { ExtensionContext } from "@lichtblick/extension";

export function activate(extensionContext: ExtensionContext): void {
  // Register a topic alias function
  extensionContext.registerTopicAliases((args) => {
    const { globalVariables } = args;
    // Use the current value of the `camera` global variable
    const camera = globalVariables["camera"] ?? "FRONT";
    return [
      {
        sourceTopicName: `/CAM_${camera}/image_rect_compressed`,
        name: `/selected_camera_image`,
      },
      { sourceTopicName: "/imu", name: "/aliased_imu" },
      { sourceTopicName: "/odom", name: "/aliased_odom" },
    ];
  });
}

Writing an Extension

Extensions can be developed in JavaScript or TypeScript and packaged as .foxe files. These files can be shared privately within your organization or distributed publicly via the Lichtblick extension registry. The desktop app supports installing extensions directly from the registry. A single extension can include multiple panels, converters, or aliases.

Lichtblick provides starter templates and commands in the create-lichtblick-extension package to simplify the development process.

Requirements:

• Node.js 14+

To set up an extension project, follow the instruction from the GitHub repository.

By leveraging Lichtblick’s extensibility, you can create powerful, customized solutions that enhance your team’s productivity and data visualization capabilities.

Settings API

The Panel Settings API allows users to link settings to message converters based on panel types.

PanelSettings Interface

The PanelSettings<ExtensionSettings> interface defines the structure for managing custom settings associated with message converters and panels. It allows users to define settings that can be dynamically applied to specific topics or schemas, enabling flexible configuration of message processing behavior.

Generic Type Parameter

• ExtensionSettings: Represents the type of the custom settings object. This is user-defined and should match the structure of the settings you want to configure.

Properties

1. settings

settings: (config?: ExtensionSettings) => SettingsTreeNode;

• Purpose: Defines how the settings should be rendered in the settings UI.

• Parameters:

  • config: An optional object containing the current configuration values. Its type is inferred from the defaultConfig property.

• Returns: A SettingsTreeNode that describes the structure of the settings UI. This node will be merged with the settings tree for the associated topic (under the path ["topics", "__topic_name__"]).

• Example:

settings: (config) => ({
  fields: {
    threshold: {
      input: "number",
      value: config?.threshold,
      label: "Threshold Value",
    },
  },
}),

2. handler

handler: (action: SettingsTreeAction, config?: ExtensionSettings) => void;

• Purpose: Handles changes to the settings made by the user in the UI.

• Parameters:

  • action: A SettingsTreeAction object describing the user's action (e.g., updating a field).
  • config: A mutable object representing the current configuration. Modifying this object updates the state.

• Behavior:

  • This function is called after the default settings handler.
  • It allows you to validate or transform the settings before they are applied.

• Example:

handler: (action, config) => {
  if (action.action === "update" && action.payload.path[1] === "threshold") {
    // Ensure threshold is within valid range
    config.threshold = Math.max(0, Math.min(1, action.payload.value));
  }
},

3. defaultConfig

defaultConfig?: ExtensionSettings;

• Purpose: Provides default values for the settings. These values are used when no configuration is explicitly set.

• Type: Must match the ExtensionSettings type.

• Example:

defaultConfig: {
  threshold: 0.5,
  enableFeature: true,
},

Expected Behavior

When implementing this interface:

1. Settings UI: The settings function defines how the settings are displayed in the UI. It creates a settings tree node that is merged into the topic's settings.
2. Configuration Management: The handler function processes user interactions with the settings UI, allowing you to validate or transform the configuration.
3. Defaults: The defaultConfig provides initial values for the settings, ensuring the panel or converter has a valid configuration even if the user hasn't customized it.

Possible Outcomes

1. Dynamic Settings UI:

   • The settings defined in the settings function will appear in the UI under the associated topic.
   • Users can modify these settings, and changes will be handled by the handler function.

2. Custom Configuration:

   • The handler function allows you to enforce constraints or transform values before they are applied.
   • For example, you can ensure a threshold value stays within a valid range.

3. Default Behavior:

   • If no custom configuration is provided, the defaultConfig values are used.
   • This ensures the panel or converter works out of the box without requiring user input.

Example Implementation:

type Schema1Schema = {
  value: number;
}

type Schema2Schema = {
  value: number;
}

// Define the configuration type
type Config = { threshold: number };

// Helper function to cast PanelSettings to the correct type
const generatePanelSettings = <T>(obj: PanelSettings<T>) =>
  obj as PanelSettings<unknown>;

export function activate(extensionContext: ExtensionContext): void {

  // Register the message converter
  extensionContext.registerMessageConverter({
    fromSchemaName: "schema1",
    toSchemaName: "schema2",

    converter: (msg: Schema1Schema, event): Schema2Schema | undefined => {

      // Access the threshold setting for the current topic
      const config = event.topicConfig as Config | undefined;
      const threshold = config?.threshold;

      // Filter messages based on the threshold
      if (threshold && msg.value > threshold) {
        return { value: msg.value }; // Forward the message if it exceeds the threshold
      }

      return undefined; // Ignore the message if it doesn't meet the threshold
    },
    // Define the settings for the threshold
    panelSettings: {
      ThresholdPanel: generatePanelSettings({
        settings: (config) => ({
          fields: {
            threshold: {
              label: "Threshold Value",
              input: "number",
              value: config?.threshold,
              placeholder: "Enter a threshold value",
            },
          },
        }),

        handler: (action, config) => {
          if (config == undefined) {
            return;
          }

          // Update the threshold setting when the user changes it in the UI
          if (
            action.action === "update" &&
            action.payload.path[2] === "threshold"
          ) {
            config.threshold = action.payload.value as number;
          }
        },
        defaultConfig: {
          threshold: 0.5, // Default threshold value
        },
      }),
    },
  });
}

Use Case

This interface is typically used when registering a message converter:

extensionContext.registerMessageConverter({
  fromSchemaName: "schema1",
  toSchemaName: "schema2",

  converter: (msg: Schema1Schema, event): Schema2Schema | undefined => {

      // Access the threshold setting for the current topic
      const config = event.topicConfig as Config | undefined;
      const threshold = config?.threshold;

      // Filter messages based on the threshold
      if (msg.value > threshold) {
        return { value: msg.value }; // Forward the message if it exceeds the threshold
      }

      return undefined; // Ignore the message if it doesn't meet the threshold
},

Summary

The PanelSettings<ExtensionSettings> interface provides a structured way to:

1. Define custom settings for panels or message converters.
2. Render these settings in the UI.
3. Handle user interactions with the settings.
4. Provide default values for the settings.

By implementing this interface, you enable users to configure their panel or converter dynamically, making it more flexible and adaptable to different use cases.

Custom Camera Models

Custom camera model extensions enable support for specialized lens distortion or projection models beyond Lichtblick’s built-in camera model. By registering a custom camera model, you can ensure that camera images with unique distortion (e.g. fisheye or other wide-angle lenses) are interpreted correctly in Lichtblick’s Image panel. This allows the Images panel to accurately render images using your custom projection logic, just as it does for the standard pinhole camera model.

To register a custom camera model, use the extensionContext.registerCameraModel API in your extension’s activate function. This function ties a distortion model name (string) to a CameraModelBuilder – a function that takes a CameraInfo object (containing the camera’s calibration parameters like intrinsics and distortion coefficients) and returns an implementation of your camera model. The CameraInfo type represents the incoming calibration message (similar to ROS CameraInfo with fields such as K, D, etc.), and your CameraModelBuilder should use this data to construct a model with the necessary projection/unprojection logic. Once registered, Lichtblick will automatically use your model whenever it encounters a CameraInfo whose distortion_model matches the name you provided.

Note: Distortion model name matching is case-sensitive. Ensure that the distortion_model string in your incoming CameraInfo messages exactly matches the name you register. The Images panel will seamlessly switch to your custom model for any camera stream with that distortion model, without additional user configuration.

Example: To create a custom camera model, first implement a class that encapsulates your distortion model (for example, a CylinderCameraModel class implementing the necessary camera projection interface). Then, in your extension’s entry point (the activate function), register the new camera model using the extensionContext.registerCameraModel.

For example, the code below registers a camera model named "CylinderCameraModel" and supplies a builder function that instantiates a CylinderCameraModel with the provided calibration data:

import { CylinderCameraModel } from "./CylinderCameraModel";
import { ExtendedExtensionContext, CameraInfo } from "./lichtblick-suite.types";

export function activate(extensionContext: ExtensionContext): void {
  extensionContext.registerCameraModel({
    name: "CylinderCameraModel",
    modelBuilder: (cameraInfo: CameraInfo) => new CylinderCameraModel(cameraInfo),
  });
}

In the code above, the string "CylinderCameraModel" is the unique name of your distortion model. This name should exactly match the distortion_model field in any camera calibration messages (camera info) that you want to be handled by your custom model. When Lichtblick encounters a camera calibration with distortion_model: "CylinderCameraModel", it will call the provided builder function to create an instance of your CylinderCameraModel class for processing that camera’s data.

Testing a Custom Camera Model in the User Scripts Panel

After registering your custom camera model, you can quickly test it using the User Scripts panel. A user script can intercept an existing camera calibration message, alter its distortion_model to your custom model’s name, and publish the modified message on a new topic. This allows you to feed an image through your custom distortion model and verify its behavior in real time.

For example, suppose you have a camera info topic /CAM_FRONT/camera_info from a front camera. You can create a user script to output a new camera info on /camera_info_custom with the same calibration data but the distortion_model set to "CylinderCameraModel". An image panel subscribed to the new calibration topic (and the corresponding image topic) will then apply your custom distortion logic. Below is a sample user script that accomplishes this:

Note: Ensure that the /camera_info_custom (output) topic is not already in use to avoid conflicts.

import { Input } from "./types";
import { CameraCalibration } from "@foxglove/schemas";

export const DISTORTION_MODEL = "CylinderCameraModel";

export const inputs = ["/CAM_FRONT/camera_info"];
export const output = "/camera_info_custom";

export default function script(
  event: Input<"/CAM_FRONT/camera_info">,
): CameraCalibration {
  return {
    ...event.message,
    distortion_model: DISTORTION_MODEL,
  };
}

In this script, we listen to the original camera info message on /CAM_FRONT/camera_info, copy its contents, and override the distortion_model field to "CylinderCameraModel". The modified camera calibration is emitted on the /camera_info_custom topic. By opening the User Scripts panel in Lichtblick and running this script, you can then point an Image panel to use the /camera_info_custom calibration (along with the camera’s image topic). This setup will route the camera’s data through your custom distortion model, allowing you to verify that your CylinderCameraModel is functioning correctly in the visualization.

Message Path Syntax

In Lichtblick, message path syntax is utilized to precisely access specific data within your messages.

Topics and Fields

Consider a message published on the /my_models topic:

{
  "total": 4,
  "objects": [
    { "width": 10, "height": 20 },
    { "width": 15, "height": 30 },
    { "width": 20, "height": 40 },
    { "width": 25, "height": 50 }
  ]
}

To display all messages for this topic, simply use the topic name:

/my_models =>
{
  total: 4,
  objects: [
    { width: 10, height: 20 },
    { width: 15, height: 30 },
    { width: 20, height: 40 },
    { width: 25, height: 50 }
  ]
}

To access nested fields, append the field name using dot notation:

/my_models.total => 4

When typing in a message path input field, a list of matching autocomplete options will appear, including any topics or nested fields that contain the input text.

Indexing into an Array

To access specific elements within an array, use bracket notation with the desired index:

/my_models.objects[1].width => 15
/my_models.objects[-1].width => 25

Slices

Consider a message on the /my_options topic:

{
  "colors": [
    { "r": 10, "g": 20, "b": 100 },
    { "r": 15, "g": 30, "b": 50 },
    { "r": 20, "g": 40, "b": 20 },
    { "r": 25, "g": 50, "b": 70 },
    { "r": 30, "g": 60, "b": 90 }
  ],
  "numbers": [3, 5, 7, 9, 10]
}

Slices allow you to retrieve a subset of values:

/my_options.colors[1:3] => [{ r: 15, g: 30, b: 50 }, { r: 20, g: 40, b: 20 }]
/my_options.numbers[-2:] => [9, 10]

When using dot notation after an array of objects, you can retrieve a specific field across all elements:

/my_options.colors[1:3].r => [15, 20, 25]
/my_options.colors[:].g => [20, 30, 40, 50, 60]

Using Variables in Slices

To slice based on variables, prefix the variable name with $. For example, defining start as 3 and end as 5:

/my_options.colors[$start:$end] => [{ r: 25, g: 50, b: 70 }, { r: 30, g: 60, b: 90 }]
/my_options.colors[$start:$end].b => [70, 90]
/my_options.numbers[$start:$end] => [9, 10]

Filters

Consider messages on the /my_books topic:

Message 1:

{
  "stats": {
    "pages": 100,
    "author": "Beatrice Potter"
  },
  "readers": [
    { "id": 1, "name": "John", "currentlyReading": true },
    { "id": 2, "name": "Mary", "currentlyReading": false },
    { "id": 3, "name": "Scott", "currentlyReading": true }
  ]
}

Message 2:

{
  "stats": {
    "pages": 210,
    "author": "Tommy \"Two Gun\" Simon"
  },
  "readers": [
    { "id": 4, "name": "Anna", "currentlyReading": true },
    { "id": 5, "name": "Patrick", "currentlyReading": false },
    { "id": 6, "name": "Richard", "currentlyReading": false }
  ]
}

To filter messages based on field values (booleans, numbers, or strings), use comparison operators like ==, !=, <, <=, >, and >=.

Apply filters within curly braces {} to select only matching messages:

/my_books{stats.pages>200} =>
{
  stats: {
    pages: 210,
    author: "Tommy \"Two Gun\" Simon"
  }
}

Filtering on Nested Fields

To filter messages using nested fields:

/my_books{stats.pages>200} =>
{
  stats: {
    pages: 210,
    author: "Tommy \"Two Gun\" Simon"
  }
}

Using Variables to Filter

Filters can be dynamic using variables. If minPages is set to 150:

/my_books{stats.pages>$minPages} =>
{
  stats: {
    pages: 210,
    author: "Tommy \"Two Gun\" Simon"
  }
}

Using Multiple Filters

You can apply multiple conditions at once:

/my_books.readers[:]{id==1}{isCurrentlyReading==true}.name =>
  "John" // message 1
  // No value returned for message 2

/my_books.readers[:]{id==1}{isCurrentlyReading==false}.name =>
  // No value returned for message 1
  // No value returned for message 2

/my_books.readers[:]{id==5}{isCurrentlyReading==false}.name =>
  // No value returned for message 1
  "Patrick" // message 2

Other Considerations

• If a filter references a field that does not exist, the message is ignored.
• Filters can be used to refine queries dynamically, improving data handling efficiency.
• Brackets [ ] are used to reference specific elements in arrays while { } applies conditions on message selection.
• Quotation marks in strings are not escaped, but you can use either single or double quotes to represent most values:

/my_books{stats.author=='Tommy "Two Gun" Simon'}.readers[:].name =>
  // No value returned for message 1
  ["Dana", "Ethan", "Frank"] // message 2

• Variables are restricted to slicing and filtering within message paths and cannot be used elsewhere.

Open via CLI

Easily open local files using the command line by installing the Lichtblick desktop application. This allows you to quickly access .mcap files without manually navigating through the interface.

Local Files

Once the desktop application is installed, Lichtblick will automatically be set as the default handler for .mcap files. You can open files directly from the command line based on your operating system:

lichtblick /path/to/your/file.mcap

# Open multiple .mcap files simultaneously
lichtblick /path/to/your/file1.mcap /path/to/your/file2.mcap

# Open all .mcap files from a specific directory
lichtblick /path/to/your/files/*.mcap

# Open a single .mcap file using the default application handler
open /path/to/your/file.mcap

# Alternatively, execute the Lichtblick binary directly (using the absolute path)
/Applications/Lichtblick.app/Contents/MacOS/lichtblick /path/to/your/file.mcap

# To open multiple .mcap files, run the binary directly with multiple paths or use a wildcard
/Applications/Lichtblick.app/Contents/MacOS/lichtblick /path/to/your/file1.mcap /path/to/your/file2.mcap
/Applications/Lichtblick.app/Contents/MacOS/lichtblick /path/to/your/files/*.mcap

# Optional: add to PATH for easier usage
echo 'export PATH="/Applications/Lichtblick.app/Contents/MacOS:$PATH"' >> ~/.zshrc && source ~/.zshrc
lichtblick /path/to/your/file.mcap

# Open a single .mcap file using the default application handler
start /path/to/your/file.mcap

# Alternatively, execute the Lichtblick binary directly (using the absolute path)
C:\Users\<USER>\AppData\Local\Programs\lichtblick\Lichtblick.exe /path/to/your/file.mcap

# To open multiple .mcap files, run the binary directly with multiple paths or use a wildcard
C:\Users\<USER>\AppData\Local\Programs\lichtblick\Lichtblick.exe /path/to/your/file1.mcap /path/to/your/file2.mcap
C:\Users\<USER>\AppData\Local\Programs\lichtblick\Lichtblick.exe /path/to/your/files/*.mcap


# Optional: add to PATH for easier usage
setx PATH "%PATH%;C:\Users\<USER>\AppData\Local\Programs\lichtblick" # need to restart the terminal
lichtblick.exe /path/to/your/files/*.mcap

Flags and Parameters

Lichtblick supports several command-line parameters to streamline your workflow by preloading specific configurations, eliminating the need for manual adjustments.

• --defaultLayout: Loads a predefined layout upon opening Lichtblick. This does not upload or modify the layout—only selects an existing one.

  lichtblick --defaultLayout="my-custom-layout"
  

• --source: Opens one or multiple .mcap files, or an entire directory, directly upon launch.

  lichtblick --source="path/to/your/file.mcap"
  
  # For multiple mcaps
  lichtblick --source="path/to/your/file1.mcap,path/to/your/file2.mcap"
  
  # For a directory
  lichtblick --source="path/to/your/files/"
  

• --time: Opens Lichtblick player at a specific timestamp.

  # Specify the time as a UNIX timestamp (in seconds)
  lichtblick --time=1633046400  # Interpreted as 2021-10-01 12:00:00 AM UTC
  
  # Specify the time using a string format
  lichtblick --time="2024-12-02 11:45"
  lichtblick --time="2020-04-07 04:45:21 PM"
  lichtblick --time="2020-04-07 04:45:21 PM CET"  # Lichtblick will attempt to convert this to the timezone used in the MCAP file
  

  These parameters help optimize the user experience by enabling quicker access to files and configurations without navigating through the UI manually.

Important Note

Multiple files are available only to .mcap files at the moment.

Shortcuts

Use keyboard shortcuts to efficiently navigate the Lichtblick visualization interface.

Playback Controls

General

Panel Selection

Panel Settings

Panels

3D Panel

Image Panel

Plot Panel

Topic Graph

Annotating ROS Enum Fields

ROS messages do not natively support enumerations, but Lichtblick provides two ways to treat constant values as enums: using a separate enum message or applying inline annotations.

Using Separate Enum Messages

Traditionally, ROS messages define enums by declaring named constants within a message definition alongside a corresponding field of the same type. For example, the following PrimaryColor message defines constants for different colors:

# In color_msgs/PrimaryColor.msg

uint8 RED=1
uint8 YELLOW=2
uint8 BLUE=3

uint8 data

Another message can then reference PrimaryColor as a field:

# In color_msgs/Object.msg

color_msgs/PrimaryColor color

Lichtblick recognizes that PrimaryColor defines named constants (RED, YELLOW, BLUE) adjacent to the data field with a matching uint8 type. As a result, the platform displays these constant names alongside raw values in visualization panels such as Raw Messages and State Transitions.

Note:

Using separate enum messages requires accessing the value indirectly (e.g., .color.data). To simplify access, consider using inline annotations.

Using Inline Enum Annotations

Lichtblick allows you to annotate enum fields directly, eliminating the need for an intermediary message.

Steps:

1. Convert the enum field to its primitive type (e.g., uint8).
2. Add an accompanying field with a _lichtblick_enum suffix that references the enum schema. Define this annotation on the line before the field declaration:

# In color_msgs/Object.msg

color_msgs/PrimaryColor color_lichtblick_enum
uint8 color

Double underscores (__lichtblick_enum) are also supported, though only valid in ROS 1.

Updating the Enum Message

With inline annotations, you can remove the data field from the enum message, making it an empty structure:

# In color_msgs/PrimaryColor.msg

uint8 RED=1
uint8 YELLOW=2
uint8 BLUE=3

• In ROS 1, an empty message does not affect serialization.
• In ROS 2, an empty message introduces an extra padding byte in serialization.

Now, the color value is stored directly in .color, rather than .color.data. Lichtblick will recognize the annotation and display the PrimaryColor constants in the Raw Messages and State Transitions panels.

Migrate from Other Tools

Lichtblick streamlines robotics development by integrating commonly used developer tools into modular panels—providing a unified development environment.

Browser Support

Lichtblick is designed to work optimally on Chrome web browser. Below is the list of supported browsers and their minimum versions.

Officially Supported Browsers

Unsupported Browsers

• Any browser not listed in "Supported Browsers"

Best Practices

1. Always use the latest stable version of your browser

Troubleshooting

If you experience issues:

• Clear your browser cache and cookies

Last updated: April 2025

Experimental Features

⚠️ Experimental features are not guaranteed to be stable
These features are made available for testing, evaluation, and feedback purposes.

What Are Experimental Features?

Lichtblick occasionally introduces early-stage features marked as experimental. These are functional but not yet finalized, and they may:

• Be unstable or incomplete.
• Change significantly in future versions.
• Be removed if no longer relevant or viable.

They are disabled by default and can be manually activated by the user.

Enabling these features helps the team validate ideas and receive valuable user feedback before they become officially supported.

How to Enable Experimental Features

1. Open Lichtblick.
2. Go to Settings > Visualization settings > Experimental features.
3. Toggle on any experimental feature available.

Once enabled, the corresponding feature becomes available immediately in the user interface or workflow.

Staying Updated

The list of experimental features is dynamic—it may change across releases.
To stay informed about new features, updates, or deprecations, we recommend to check:

• Release notes
• Documentation
• In-app announcements

Feedback Welcome

Have suggestions or found a bug in an experimental feature?
Your feedback is essential to improve these early capabilities.

→ Please submit your thoughts via the Lichtblick issue tracker or contribute directly via pull request.

Synchronizing Lichtblick Instances

⚠️ This feature is experimental
Synchronization between Lichtblick instances is an early feature under active development. Instabilities or limitations may occur.

Overview

The Lichtblick Sync feature allows users to synchronize playback actions—such as play, pause, seek, and play until—across multiple running instances of Lichtblick on the same platform (desktop-to-desktop or web-to-web). This is especially useful for users working on multi-window/multi-screen setups and comparing MCAPs.

Enabling the Feature

To activate the synchronization:

1. Open Lichtblick.
2. Go to Settings > Visualization settings > Experimental features.
3. Toggle on Sync Lichtblick instances.

Once enabled, a small toggle button will appear in the bottom-right corner of the playback bar.

Using the Sync Feature

Once the feature is enabled and toggled on, the current instance will broadcast playback actions to other active instances running on the same platform.

Supported Broadcast Actions:

🔁 Each instance independently process its own data, but reacts to common control commands.

Requirements and Best Practices

To ensure the feature works properly:

• All instances must be:

  • Open on the same platform: either all desktop or all web.
  • Ideally running Lichtblick with the same file.
  • Aligned on playback range: The files should cover the same time range.
    Otherwise, syncing behavior may be inconsistent.

• Ensure the "Sync" toggle is enabled in all relevant instances.

Limitations

• Experimental Feature:
  This feature is still under evaluation and subject to change.

• Platform isolation:
  Synchronization only works between instances on the same platform:

  • Desktop-to-desktop ✅
  • Web-to-web ✅
  • Desktop-to-web ❌

• Performance-sensitive behavior:
  If one instance is under high load (e.g., parsing many topics), it might lag behind others.
  Even though actions are synchronized via broadcast, visual synchronization accuracy may vary depending on system performance and buffering behavior.

• Timing offsets may occur:
  Playback buffering is local to each instance. The sync mechanism sends time targets, but frame-perfect synchronization is not guaranteed.