What this post covers:

• Introducing the RealSense Perception Studio platform 
• Why perception must evolve through continuous software innovation 
• How developers can rapidly evaluate, deploy, and benefit from new perception capabilities

RealSense Perception Studio: A Faster Path to New Perception Capabilities

Perception is one of the defining technologies of the Physical AI era. As the Visual Cortex of Physical AI™, RealSense provides the perception technologies that enable intelligent machines to understand, navigate, and interact with the physical world.

This naturally raises an important question: How can developers access new perception capabilities earlier—without waiting for a new hardware generation or a full software productization cycle?

This is exactly why we created RealSense Perception Studio.

Robots are being asked to do more in increasingly complex environments. They must navigate crowded spaces, operate safely alongside people, manipulate objects with greater precision, and understand their own motion with increasing accuracy.

Each of these capabilities depends on perception.

Yet adding new perception capabilities has traditionally been slow and resource-intensive. Development teams often need to evaluate multiple libraries, integrate new processing pipelines, validate camera and sensor configurations, and repeat much of that work as perception technology evolves.

RealSense Perception Studio is designed to shorten that cycle.

RealSense Perception Studio is a continuous-release program that gives developers early access to next-generation perception capabilities built on RealSense depth cameras. Each release combines working technology with the resources needed to evaluate it, including tutorials, demonstration videos, technical documentation, and links to relevant libraries.

The goal is practical: shorten the time between a new capability becoming available and that capability running successfully on a robot. Perception Studio helps developers evaluate emerging technologies sooner, reduce integration friction, and create a faster path from experimentation to deployment.

Figure 1. RealSense Perception Studio gives developers early access to evolving capabilities for close-range depth, localization, and human awareness.

As robotics and AI continue to advance, perception cannot remain static. Developers need technologies that evolve alongside their applications, with new capabilities delivered through software, edge processing, and AI. Perception Studio was created to make that evolution practical.

Why Perception Needs a Different Development Model

Robotic perception is advancing rapidly.

Edge AI, visual localization, depth processing, and on-camera inference continue to expand what robots can perceive and how they interact with the physical world. But innovation often moves faster than the release cycles that deliver it.

Hardware platforms remain in robotic systems for years, while advances in algorithms, SDKs, processing architectures, and AI models can unlock entirely new capabilities on the same camera platform.

At the same time, transforming a promising algorithm into a production-ready capability requires extensive validation across camera platforms, host systems, operating environments, lighting conditions, and real-world use cases. That productization process is essential, but it also takes time.

RealSense Perception Studio introduces a different approach.

By giving developers early access to emerging perception capabilities, RealSense can gather feedback from real deployments while developers begin evaluating new technologies sooner. Real-world usage becomes part of the validation process, helping capabilities mature more quickly while reducing the time between innovation and deployment.

Instead of treating perception as fixed when hardware ships, Perception Studio enables capabilities to evolve continuously through software, host-side processing, and on-camera intelligence.

It also creates a more direct connection between developers and the RealSense product and engineering teams. Early-access users can test capabilities in real-world environments, identify limitations, provide feedback, and help shape future development.

More Than a Feature Preview

Perception Studio is more than a collection of experimental features.

Each capability provides a practical starting point for evaluation by connecting the technology with the cameras, libraries, workflows, and technical resources required to test it.

Every capability includes:

• A technical tutorial
• A demonstration video
• Links to relevant documentation and libraries
• Guidance for supported cameras and compute platforms
• A path for developer feedback and engagement

This structure allows tutorials to focus on implementation while the broader Perception Studio program explains why the capability matters, where it fits, and how it can contribute to a larger perception stack.

The initial release introduces three capabilities: Improved Close-Range Depth (Min-Z), Visual-Inertial Odometry (VIO), and People Detection—addressing three foundational needs for intelligent machines: near-field perception, motion understanding, and human awareness.

Improved Close-Range Depth: Seeing Inside the Near-Field Blind Zone

Many of the most important robotic interactions happen close to the camera.

A gripper approaches an object. A humanoid reaches toward a surface. A service robot moves through a narrow doorway. An inspection system examines a component only a few centimeters away.

These interactions often take place inside the native minimum sensing distance of a stereo depth camera. When an object enters that near-field region, the system may lose valid depth precisely where accurate spatial information is most valuable.

Improved Close-Range Depth, also referred to as Min-Z, extends the usable sensing range closer to the camera.

The initial Perception Studio capability extends depth sensing to under 3 cm on the RealSense D401 and D405, and approximately 12 cm across D43x and D45x cameras.

This can bring usable depth into the robot’s immediate working area, supporting applications such as:

• Robotic manipulation and grasping
• Bin picking
• Hand-eye coordination
• Close-up inspection
• Proximity sensing
• Navigation through tight spaces

The deployment model depends on the camera platform. On supported D400-series cameras, the capability runs on an NVIDIA Jetson host as part of the processing pipeline. On D500-series cameras, enhanced close-range depth runs natively on the RealSense Vision SoC V5.

For Jetson-based deployments, Perception Studio provides prebuilt .deb packages for JetPack 6 and JetPack 7 that integrate directly with the RealSense SDK. No SDK rebuild or custom toolchain is required, and existing applications can continue using their current pipeline while receiving the improved close-range depth stream through standard SDK interfaces.

For developers, the value is not only the shorter sensing distance. It is the ability to address close-range perception challenges without redesigning the system around a fixed near-field blind zone.

Visual-Inertial Odometry: A Faster Route to Robot Localization

Autonomous systems need to understand how they are moving.

Visual-Inertial Odometry, or VIO, estimates a camera’s position and motion by combining visual information with measurements from an inertial measurement unit. It is a foundational capability for humanoids, autonomous mobile robots, drones, and other systems that must navigate without relying entirely on external infrastructure.

Building a robust VIO pipeline can be complex. Developers must coordinate synchronized camera and IMU data, calibration, visual feature tracking, inertial fusion, and real-time processing.

Perception Studio provides a RealSense-native path to NVIDIA cuVSLAM, the GPU-accelerated visual SLAM library, using RealSense D455 and D436 RGB-D global-shutter cameras.

The initial configuration includes:

• Cameras: RealSense D455 and D436 with integrated IMUs
• SLAM engine: NVIDIA cuVSLAM
• Runtimes: C++ for production through NVIDIA Isaac ROS Visual SLAM; Python for prototyping and benchmarking
• Target platforms: Humanoids, AMRs, and drones

The same technology stack supports multiple operating modes:

• Stereo visual odometry
• Stereo visual-inertial odometry
• Hardware-synchronized multi-camera odometry

This gives developers a progression from initial pipeline validation to more robust configurations for demanding motion profiles. A single-camera stereo configuration provides a straightforward way to validate the pipeline; adding inertial data improves motion estimation, while multi-camera configurations can increase coverage and resilience during rapid movement, occlusion, or low-texture scenes.

For production deployments, developers can transition from Python-based prototyping to the C++ runtime through NVIDIA Isaac ROS Visual SLAM.

The broader value is reduced integration effort. Instead of assembling calibration, synchronization, and sensor fusion independently, developers can begin with a documented RealSense and NVIDIA workflow designed for real-time robotic applications.

People Detection: Moving Human Awareness onto the Camera

Robots operating around people need to understand when someone is present.

Traditionally, this requires the host system to receive image data, preprocess frames, run an AI model, and return detection results to the application. That consumes CPU or GPU resources that may already be supporting navigation, planning, manipulation, or other AI workloads.

Perception Studio People Detection moves inference onto supported RealSense D500-series cameras.

A compact model runs directly on the RealSense Vision SoC V5 and publishes detection results through the RealSense SDK or a dedicated ROS 2 DDS topic. The host receives structured detection information without having to perform the full inference workload.

This architecture can reduce:

• Host AI compute requirements
• Image-transfer bandwidth
• Preprocessing overhead
• Integration complexity
• End-to-end latency

The initial implementation focuses on people detection, supporting applications such as human-aware navigation, service robotics, collaborative automation, humanoid interaction, and safety-oriented behaviors.

People Detection also demonstrates a broader direction for the D500 platform: moving more perception capabilities to the edge, closer to the sensor, while allowing the host to remain focused on higher-level reasoning and control.

Building a Composable Perception Stack

The three initial capabilities are valuable individually, but their greater potential comes from composition.

Figure 3.Min-Z, VIO, and People Detection can be evaluated independently or combined to help robots navigate, interact, and operate safely.

A mobile manipulator could use VIO to estimate motion, Improved Close-Range Depth to maintain near-field awareness during manipulation, and People Detection to respond when someone enters the workspace.

A humanoid could combine VIO for motion and localization, close-range depth for hand-eye interaction, and People Detection to maintain awareness of nearby people.

An AMR could combine localization with improved near-field obstacle sensing and human-aware navigation.

Perception Studio supports this kind of modular development. Developers can evaluate individual capabilities, determine where they add value, and integrate them into a broader perception and autonomy stack.

A More Direct Path from Innovation to Deployment

Early-access technology is most valuable when developers can understand it, test it, and provide meaningful feedback.

The continuous-release model allows RealSense to make new functionality available more quickly, while giving developers the opportunity to evaluate capabilities before they become part of a broader production roadmap.

Not every early-access feature will be appropriate for every production system. Capabilities may evolve, improve, or require additional validation. That is the purpose of the program: to create a structured environment where new perception technologies can be tested against real-world robotic requirements.

Iterate Fast. Ship Faster.

Robotic perception is defined by the combination of cameras, edge compute, SDKs, AI models, and continuously evolving software.

RealSense Perception Studio is how those improvements reach developers sooner—and how developer feedback reaches us.

With Improved Close-Range Depth, Visual-Inertial Odometry, and People Detection, the first release addresses three immediate challenges in robotics: seeing nearby objects, understanding motion, and operating with greater awareness of people.

Each capability includes the technical resources needed to begin evaluation, with additional features planned as the platform evolves.

Sign up for RealSense Perception Studio to access the latest capabilities, tutorials, demonstrations, and technical resources.