将模拟想法转化为工作应用意味着集合资产，连接物理和渲染，并检查场景是否如所愿。 开发者正在将前沿AI模型与NVIDIA Omniverse 库相结合来帮助完成这项工作——构建用于探索情景的应用，调查故障并改进设计。
开发者通过自然语言指令来指导AI agents，审查结果并引导变化。 Omniversity库提供GPU加速物理，渲染和传感器模拟能力。
探索以下项目，以查看GPT-6 Astra等前沿AI模型在工作，并检查整个生态系统的NVIDIA团队和开发者的新例子。
从站点扫描创建可编辑的建筑模型
规划仓库自动化需要建筑物本身的模型，但Shelving和机械可以在站点扫描中掩盖其结构。 Omniverse 带来组合了真到相的工具和 agent 技能来帮助开发者将真实世界的捕捉转化为可编辑的起始环境，用于物理AI模拟。
NVIDIA模拟技术团队的大卫·韦尔(David Wehr)在这项工作的基础上利用Astra开发出一个工作流程，用于数字重建一座没有内容的建筑。 目标是一种可编辑，可模拟的可准备的建筑模型，开发者可以在此添加或重新定位谢尔文和机械，在进行物理改变前评价仓库布局和自动化计划。
Wehr引导Astra诠释全景相片，创建了几何结构并配合了点-云测量。 OpenUSD存储模型，Omniverse 的 ovrtx 库则提供检查结果。
探索NVIDIA Omniverse agent技能，用Omniverse库来帮助agents建设。
测试带有数字双子的气体-土耳其操作情景
在改变工业设备的运作方式之前，工程师需要一种探索后果的方法。 NVIDIA Omniverse开发者关系团队的成员Pradeep Kaushik指示Astra围绕模拟燃气涡轮和发电机构建一个什么-if分析应用。
应用程序允许用户调整负荷，环境温度和燃料组成等条件，然后比较功率，效率，燃料消耗和设备反应。 6个测试程序探索包括负载变化、被卡住的燃料阀门和排气传感器出错等各种情况。
该原型为工程师提供了一个互动的方法来调查操作变化和断层如何会影响设备行为，支持关于哪些情景需要进一步测试的决定。
Astra使用Cantera等工具来进行热化学，而SciPy则用于设备动力学。 应用地图解析器在3D涡轮机中生成转子动画和色码温度，压力和流速指标。 NVIDIA Omniverse 库更新了场景(ovstage)，将其与RTX(ovrtx)并流到浏览器(ovstream). 工程师可以旋转和zoom切出，在指标之间切换，并通过与图表和虚拟仪器并列的模拟反应来暂停或擦除。
以开源 SimReady 资产构建模拟场景
与SimReady Central Library，宣布 at GTC Berlin，开发者可以从设备制造商和资产提供商获得超过2000个模拟准备的USD资产，用于机器人模拟和AI工厂数码双胞胎工作流程。
NVIDIA的Omniverse文献团队成员Paul Kind通过对话将AI agent指向了图书馆并引导场景组装。 他的厨房场景包括从63个独特的图书馆资产中抽取的180个位置，以及18个定制几何图案。 请与Isaac Sim一起编写，并指导进一步调整。
对于机器人和工厂工作流程，开发者可以选择含有模拟属性的SimReady资产来配合他们所需要的任务，然后在一个组装环境中进行测试。
在SimReady中央图书馆找到资产，并学习如何作出贡献。
按规模测试仓库模拟
仓库模拟需要随着更多机器人手臂，传送器和盒子填满场景而保持平稳运行。
当它减速时，开发者需要识别是什么阻碍它。 NVIDIA的首席工程师Bhavik Sanganee利用Astra与NVIDIA Omniverse图书馆建立仓库模拟，增加其复杂性并调查减速。
当场面规模翻了一番时，性能首次低于目标。 运行物理计算，图像渲染和视频制备同时恢复了性能并实现了进一步的缩放。
以80个传送器，40个臂和800个箱子，模拟在两分钟的测试中以全HD分辨率维持了每秒30帧。 武器完成了640个成功的箱装。
场面的翻倍再次降低了性能。 测试将减速追溯到物理和机器人控制管上，给桑加内一个需要改进的特定领域。
与 NVIDIA Omniverse 库一起开始建设。
用于弹性电缆运行的测试机器人控制
在电子装配中，机器人可能需要将柔性电缆围绕组件放置后再插入端口。 电缆会随着机器人的移动而弯曲，连接器必须精确地排成一行以适应。
NVIDIA的机器人技术营销经理Oyindamola Omotuyi指导Astra使用NVIDIA Isaac Lab，牛顿物理和Omniverse渲染，用机器人抓取器处理电缆来构建机器人模拟。
Astra还开发了一个强化学习系统，将学到的有线线路运动与定位和定向反馈相结合，并对USB-C插入进行纠正调整。 模拟让奥莫图伊在转入物理-机器人实验之前，先测试电缆处理和连接器对接的方法。
控制器使用了由模拟直接提供的信息。 接下来的步骤包括测试摩擦控制控制并使用物理机器人可以从相机和传感器获得的信息。
开始在NVIDIA Isaac实验室学习机器人。
将机器人计算机辅助设计输入一个 SimReady 资产
在测试一个机器人能否接取并移动一个盒子之前，开发者需要有一个具有关节的模拟模型，这些关节可以正确移动，并可以握住对象。 计算机辅助设计(CAD)模型提供了机器人的外形，但是它仍然需要物理特性来完成这项任务。
开发者使用ABB Robotics Yumi机器人，用NVIDIA Omniverse内容库引导Astra将CAD文件转换为OpenUSD，将机器人外观与参考图像相匹配，并配置关节和相撞几何。 他们在NVIDIA Isaac Sim测试之前，根据SimReady基金会的要求检查了资产。
双臂和抓取器都完成了四个立方体取地周期，使用模拟的手指接触和摩擦。 这些测试显示了在模拟条件下的行为——估计的物理特性没有被校正出与真正的机器人相配。
跟随"五步"的"SimReady教程".
为仓库环境建造人造模拟器
在第一人称和第三人称视角中探索对人造机器人的游戏化物理控制。
在将仓库任务自动化之前，开发者需要一个交互式模拟环境来探索任务行为并评价工作是如何完成的。 NVIDIA的Omniverse产品管理器Frank DeLise利用Astra将一个SimReady仓库和人造机器人变成具有第一人称和第三人称视图的互动模拟器。
DeLise引导Astra连接NVIDIA Omniverse库，用于物理(ovphysx)，场景更新(ovstage)，渲染(ovrtx)和用户界面(ovui). 他还用Astra和SimReady(已准备就绪)在模拟中创建了物理场景。 Astra随后生成了动画和应用代码来将这些能力汇集在一起。
学习如何用前沿AI模型和NVIDIA Omniverse库来准备和验证SimReady机器人资产。
连接自动驱动测试工作流程
改变一个场景，传感器或驾驶模型会显著地影响自发车辆的模拟。 NVIDIA模拟技术团队的经理多尤布·金(Doyub Kim)要求Astra将"零"建为"阿尔帕马约"——一个基于旧金山市场街的可再使用的模拟环境。
Kim指示Astra绘制出工作流程，然后连接资产创建，流量，Omniverse RTX传感器模拟和Alpamayo分期行驶，检查每个集成。 由此产生的原型是用来比较模型和追踪场景或传感器变化如何影响下游驾驶行为的试验场。 一个单独的Cosmos3-Nano实验在被录制的模拟视频中不同天气和照明，使得金可以比较驱动模型在不同条件下对同一种情景的反应。
使用传感器差异来创建和改进数字双胞胎
比较记录的和模拟的相机和lidar输出，为开发者提供了关键的基于业绩指标的反馈，以完善场景。
要测试机器人和自主飞行器，开发人员需要知道模拟传感器与真实的传感器的相近程度。 Ashley Reid在NVIDIA从事RTX传感器验证工作，他指导Astra和Claude 第5号线agents将ovrtx相机和原始LiDAR输出与所记录的数据进行比较。 agents从头创造了两个数码双胞胎，并改进了两个现存的双胞胎。
在大约三天的时间里，Reid指导了一个迭代工作流程，其中agents测量了差异，创建或修改了OpenUSD场景，并检查了结果。 变化涉及缺失的物体、几何和材料，接受取决于相机和LiDAR的度量衡。 这为开发者提供了使用测量出的差异来指导场景的创建和改进的方法。
首先用 ovrtx 最小 Python 示例来渲染 OpenUSD 场景，然后定义一个传感器度量法来比较所记录的数据。
机器人奥运会：模拟测试机器人技能
教机器人新动作意味着检查这些行动是否在物理限制下有效。 领导NVIDIA Omniverse工程和产品的泰金使用体育视频和自然语言指令来指导Astra建设Robo Olympics，这是一个测试Unitree G1人形进行运动的实验项目。
在Kim的指导下，Astra建造了控制器，并通过物理试验加以改进。 牛顿物理引擎模拟了行为，开源 NVIDIA Warp框架加速了计算，而ovrtx提供了场景和虚拟相机图像。 在一个实验中，机器人在100个模拟试验中的64个中清除了单一的障碍。 这些试验为Kim改进机器人的时机和控制提供了反馈。
观察金与Astra和NVIDIA Omniverse图书馆一起探索建筑模拟。
使用计算机辅助设计和模拟进行机器人分解测试
在机器人能拆卸出产品之前，开发者需要知道其工具能否到达并去除部件。 Jens Jebens是OpenUSD at NVIDIA的高级产品经理，他指示Astra在PTC Onshape中建模一款汽车吊车并配置在NVIDIA Isaac Sim.
与Astra一起，Jebens探索了计算机辅助设计和对通过模拟得知的机器人的工具修改。 agent测量了可用的空间并设计出一个扳手，机器人可以用来达到悬浮螺栓。 Jebens报告在模拟中成功移除了悬浮组件。 这把设计和工具化决定联系起来，以拆分结果，为机器人政策培训提供了一个起点。
探索"Onshape"进口商指南。
将国际空间站引入浏览器
将3D模型转化为应用程序需要连接资产，活数据和接口。 NVIDIA的工程总监尼克·约翰斯(Nic Johns)促使阿斯特拉将NASA的资产组装成带有遥测功能的OpenUSD国际空间站模型。 约翰斯用单一的快取来构建应用程序，再用后续快取来将场景转移到地球的白天一面，这样行星就可见了。
工作流程使用Blender进行资产准备
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Turning a simulation idea into a working application means assembling assets, connecting physics and rendering, and checking that the scene behaves as intended. Developers are combining frontier AI models with NVIDIA Omniverse libraries to help carry out that work — building applications for exploring scenarios, investigating failures and improving designs.
Developers direct AI agents through natural-language instructions, review results and guide changes. Omniverse libraries provide GPU-accelerated physics, rendering and sensor simulation capabilities.
Explore the projects below to see frontier AI models such as GPT-6 Astra at work, and check back for new examples from NVIDIA teams and developers across the ecosystem.
Create an Editable Building Model From Site Scans 🔗
Planning warehouse automation requires a model of the building itself, but shelving and machinery can obscure its structure in site scans. Omniverse brings together real-to-sim tools and agent skills to help developers turn real-world captures into editable starting environments for physical AI simulation.
David Wehr, from NVIDIA’s simulation technology team, built on that work using Astra to develop a workflow for digitally reconstructing a building without its contents. The goal is an editable, simulation-ready building model where developers can add or reposition shelving and machinery to evaluate warehouse layouts and automation plans before making physical changes.
Wehr guided Astra to interpret panoramic photos, create building geometry and align it with point-cloud measurements. OpenUSD stores the models, while Omniverse’s ovrtx library renders results for inspection.
Explore NVIDIA Omniverse agent skills to help agents build with Omniverse libraries.
Test Gas-Turbine Operating Scenarios With a Digital Twin 🔗
Before changing how industrial equipment operates, engineers need a way to explore the consequences. Pradeep Kaushik, a member of the NVIDIA Omniverse developer relations team, directed Astra to build a what-if analysis application around a simulated gas turbine and generator.
The application lets users adjust conditions such as load, ambient temperature and fuel composition, then compare power, efficiency, fuel consumption and equipment response. Six test procedures explore scenarios including load changes, a stuck fuel valve and an exhaust-sensor error.
The prototype gives engineers an interactive way to investigate how operating changes and faults could affect equipment behavior, supporting decisions about which scenarios warrant further testing.
Astra used tools like Cantera for thermochemistry and SciPy for equipment dynamics. The application maps solver results to rotor animation and color-coded temperature, pressure and flow-velocity indicators in a 3D turbine. NVIDIA Omniverse libraries update the scene (ovstage), render it with RTX (ovrtx) and stream the view to the browser (ovstream). Engineers can rotate and zoom the cutaway, switch between indicators, and pause or scrub through a simulated response alongside charts and virtual instruments.
Build Simulation Scenes With Open Source SimReady Assets 🔗
With SimReady Central Library, announced at GTC Berlin, developers can access more than 2,000 simulation-ready USD assets from equipment manufacturers and asset providers for robotics simulations and AI factory digital twins workflows.
Paul Kind, a member of NVIDIA’s Omniverse documentation team, pointed an AI agent to the library and guided scene assembly through conversation. His kitchen scene included 180 placements drawn from 63 unique library assets, alongside 18 custom geometry pieces. Kind reviewed renders produced with Isaac Sim and guided further adjustments.
For robotics and factory workflows, developers can select SimReady assets with the simulation properties embedded to fit their required task then test in an assembled environment.
Find assets in SimReady Central Library, and learn how to contribute.
Stress-Test a Warehouse Simulation at Scale 🔗
A warehouse simulation needs to keep running smoothly as more robot arms, conveyors and boxes fill the scene.
When it slows down, developers need to identify what’s holding it back. Bhavik Sanganee, principal engineer at NVIDIA, used Astra with NVIDIA Omniverse libraries to build a warehouse simulation, increase its complexity and investigate slowdowns.
Performance first fell below the target when the scene doubled in size. Running physics calculations, image rendering and video preparation concurrently restored performance and enabled further scaling.
With 80 conveyors, 40 arms and 800 boxes, the simulation sustained 30 frames per second at full HD resolution during a two-minute test. The arms completed 640 successful box placements.
Doubling the scene again reduced performance. Testing traced the slowdown to the physics and robot-control pipeline, giving Sanganee a specific area to improve.
Start building with NVIDIA Omniverse libraries.
Test Robot Control for Flexible Cable Routing 🔗
In electronics assembly, a robot may need to position a flexible cable around components before plugging it into a port. The cable bends as the robot moves it, and the connector must line up precisely to fit.
Oyindamola Omotuyi, robotics technical marketing manager at NVIDIA, directed Astra to build a robot simulation using NVIDIA Isaac Lab, Newton physics and Omniverse rendering with a Robotiq gripper handling the cable.
Astra also developed a reinforcement learning system that combined learned cable-routing movements with position-and-orientation feedback and corrective adjustments for USB-C insertion. The simulation gave Omotuyi a way to test cable handling and connector alignment before moving to physical-robot experiments.
The controller used information supplied directly by the simulation. Next steps include testing a friction-held grasp and using information a physical robot could obtain from cameras and sensors.
Get started with robot learning in NVIDIA Isaac Lab.
Turn Robot Computer-Aided Design Into a SimReady Asset 🔗
Before testing whether a robot can pick up and move a box, developers need a simulation model with joints that move correctly and with grippers that can hold the object. A computer-aided design (CAD) model provides the robot’s shape, but it still needs physics properties to perform the task.
Using an ABB Robotics YuMi robot, developers directed Astra with NVIDIA Omniverse content libraries to convert CAD files into OpenUSD, match the robot’s appearance to reference images, and configure joints and collision geometry. They checked the asset against SimReady Foundation requirements before testing it in NVIDIA Isaac Sim.
Both arms and grippers completed four cube pick-and-place cycles, using simulated finger contact and friction. These tests demonstrated behavior under the simulated conditions — estimated physical properties were not calibrated against the real robot.
Follow the five-step SimReady tutorial.
Build a Humanoid Simulator for a Warehouse Environment 🔗
Explore a gamified, physics-based control of a humanoid robot in first- and third-person view.
Before automating warehouse tasks, developers need an interactive simulation environment to explore task behavior and evaluate how the work gets done. Frank DeLise, Omniverse product manager at NVIDIA, used Astra to turn a SimReady warehouse and humanoid robot into an interactive simulator with first- and third-person views.
DeLise directed Astra to connect NVIDIA Omniverse libraries for physics (ovphysx), scene updates (ovstage), rendering (ovrtx) and the user interface (ovui). He also used Astra with SimReady (simready-foundation) to create the physical scene in simulation. Astra then generated animation and application code to bring those capabilities together.
Learn how to prepare and validate SimReady robot assets with frontier AI models and NVIDIA Omniverse libraries.
Connect an Autonomous-Driving Testing Workflow 🔗
Changing a scene, sensor or driving model can significantly affect autonomous-vehicle simulations. Doyub Kim, a manager on the simulation technology team at NVIDIA, asked Astra to build Zero to Alpamayo — a reusable simulation environment based on San Francisco’s Market Street.
Kim directed Astra to map out the workflow, then connect asset creation, traffic, Omniverse RTX sensor simulation and Alpamayo driving in stages, checking each integration. The resulting prototype was a testing ground for comparing models and tracing how scene or sensor changes affect downstream driving behavior. A separate Cosmos3-Nano experiment varied weather and lighting in recorded simulation videos, allowing Kim to compare the driving model’s responses to the same scenario under different conditions.
Use Sensor Differences to Create and Improve Digital Twins 🔗
Comparing recorded and simulated camera and lidar outputs gives developers key performance indicator-based feedback for refining a scene.
To test robots and autonomous vehicles, developers need to know how closely simulated sensors match real ones. Ashley Reid, who works on RTX sensor validation at NVIDIA, directed Astra and Claude Fable 5 agents to compare ovrtx camera and raw LiDAR outputs with recorded data. The agents created two digital twins from scratch and improved two existing ones.
Over about three days, Reid guided an iterative workflow in which agents measured differences, created or modified OpenUSD scenes, and checked the results. Changes addressed missing objects, geometry and materials, with acceptance depending on camera and LiDAR metrics. This gives developers a way to use measured discrepancies to guide scene creation and improvement.
Start by rendering an OpenUSD scene with the ovrtx minimal Python example, then define a sensor measure to compare with recorded data.
Robo Olympics: Test Robot Skills With Simulation 🔗
Teaching robots new movements means checking whether those actions work under physical constraints. Tae Kim, who leads NVIDIA Omniverse engineering and product, used sports videos and natural-language instructions to guide Astra in building Robo Olympics, an experimental project that tests simulated Unitree G1 humanoids performing sports movements.
Under Kim’s direction, Astra built controllers and refined them through physics trials. Newton Physics Engine simulated behavior, the open source NVIDIA Warp framework accelerated calculations and ovrtx rendered scenes and virtual-camera images. In one experiment, the robot cleared a single hurdle in 64 of 100 simulation trials. The trials gave Kim feedback for improving the robot’s timing and control.
Watch Kim explore building simulations with Astra and NVIDIA Omniverse libraries.
Test Robotic Disassembly With Computer-Aided Design and Simulation 🔗
Before a robot can disassemble a product, developers need to know whether its tools can reach and remove the parts. Jens Jebens, a senior product manager for OpenUSD at NVIDIA, directed Astra to model a car suspension in PTC Onshape and configure it in NVIDIA Isaac Sim.
With Astra, Jebens explored computer-aided design and tooling revisions for robots informed by simulation. The agent measured the available space and designed a wrench the robot could use to reach the suspension’s bolts. Jebens reported successful removal of a suspension component in simulation. This connects design and tooling decisions to disassembly results, offering a starting point for robot policy training.
Explore the Onshape importer guide.
Bring the International Space Station Into the Browser 🔗
Turning 3D models into an application requires connecting assets, live data and an interface. Nic Johns, an engineering director at NVIDIA, prompted Astra to assemble NASA assets into an OpenUSD International Space Station model with telemetry. Johns built the application with a single prompt, then used a follow-up prompt to shift the scene to Earth’s daytime side so the planet was visible.
The workflow used Blender for asset preparation
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.