Projects | Liu Yue — Embodied AI Researcher

GDN-Memory: Constant-Footprint Embodied Memory for World Models

Wed, 01 Apr 2026 00:00:00 +0000

miracle-techlink / GitHub GDN-Memory 代码仓库（即将开源） GitHub GDN-Memory Demo · Bilibili 具身记忆系统演示视频 Bilibili

A three-layer hierarchical memory system for embodied agents built on Gated DeltaNet (GDN) fast-weight matrices, integrated into the Genie Envisioner Slow-Fast VLA architecture.

Architecture:

Layer 1 (Physical) — TTT layer inside each of the 28 DiT blocks of GE-Base (2B), inserted after cross-attention, before FFN. Shared W ∈ ℝ^{64×64} updated at 5 Hz Slow Path.
Layer 2–3 (Object / Behavioral) — External to Slow Path. CLIP/DINO detects objects from visual latents, writes W_obj and W_person via online GDN updates. Memory tokens injected into GE-Act (160M, 30 Hz) via cross-attention.

Key properties:

Constant 256 KB memory regardless of episode length
<1ms GDN update latency (SRAM fast-weight computation)
3D RoPE spatial encoding for object-position binding
Dual CUDA stream + Double Buffering scheduling

Target: ICRA / CoRL 2026 | Affiliations: Tongji CS × PKU DLIB × 智元机器人

Demo

Experiment Log

Staged validation: synthetic → real-scene → VLA integration → memory robustness → cold-start.

Exp A — GDN Object Memory: Synthetic Validation ✅ PASS

Validates the core GDN mechanism across 5 sub-experiments on synthetic data.

Hypothesis	Result
H1 Capacity	100% retrieval at N≤100; 99.2% at N=200 (4.5× safety margin)
H2/H3 Online update	99.8% recovery after 1 re-observation; 0.0 pp interference on unmoved
Hyperparameter robustness	All 25 (γ, η) combos → acc=1.000
Momentum ablation	η_s≥0.8 harmful, must be disabled
Key ablation	Random CLIP keys cost only 6.3 pp vs orthogonal

Exp B — GenieSim Real-Scene Validation 🔜 Planned

Transfers GDN to GenieSim home_g2 (13 scene variants) using real CLIP ViT-L/14 features on a 智元 G2 robot.

Hypothesis	Target
H1	hit@1m ≥ 80% for N≤50 objects
H2	After 20% objects moved: recovery hit@1m ≥ 75%, unmoved degradation ≤5 pp
H3	≥10/13 scenes individually pass H1 + H2

Exp C — GDN + GE-Act Integration 🔜 Planned

Injects GDN memory tokens into GE-Act’s Action DiT via attn3_hidden_states. Only a ~524K projection layer is trained (backbone frozen). Benchmarked on CALVIN, LIBERO, and ICRA26 World Model Track.

Hypothesis	Target
H_C1/C2	Reproduce GE-Act baseline (CALVIN Avg ≥ 4.0), ≤5 pp degradation on standard tasks
H_C3	nDTW: 0.388 → ≥0.538 (+15 pp) on ICRA26 competition data
H_C4	GDN achieves ≥90% of KV-Cache upper bound at 256 KB

Exp D — System 2 Surprise Routing 🔜 Planned

Tests a System 2 explicit event buffer (K=100 slots, 100 KB) using GDN update magnitude as a zero-cost surprise signal — protecting memory when 50% of objects are simultaneously displaced (System 1 alone degrades 20–30 pp).

Hypothesis	Target
H_D1	System 1+2 retains unmoved memory ≥ System 1 alone +15 pp
H_D2	Buffer exact recall hit@1m ≥ 95%
H_D3	Surprise decreases monotonically, ≤30% of initial after 5 reps (consolidation)

Exp E — System 3 Persistent Prior 🔜 Planned

Tests learnable prior tokens P ∈ ℝ^{32×256} (32 KB, frozen at deployment) to cut cold-start exploration from ~400 steps to ~180 steps.

Strategy	Expected gain
A: CLIP text embeddings of 32 common objects (zero training)	−40% steps-to-80%-hit
B: MAML meta-learning over multi-scene episodes	−57% steps-to-80%-hit
H_E2	Prior steady-state gap ≤3 pp at 500 steps (no long-term interference)

Endovascular Surgical Robot Platform (stEVE)

Sun, 01 Mar 2026 00:00:00 +0000

miracle-techlink / GitHub stEVE 平台代码仓库 GitHub Blog: VLA 如何落地医疗机器人 miracle-techlink.github.io · 综述文章 Link

A complete research platform for AI-assisted endovascular intervention, spanning preoperative 3D planning through intraoperative RL-guided navigation.

Pipeline:

CTA → STL — TotalSegmentator vessel segmentation, VMTK centerline extraction
SOFA FEM Simulation — Cosserat beam guidewire model, realistic vascular deformation
RL Navigation — SAC/PPO agents with geodesic distance reward + safety constraints
Biplane DSA — DLT triangulation + Beer-Lambert X-ray projection for 2D-3D registration
Intraoperative Correction — SE(3) IRC-RL with real DSA feedback

Ongoing research: Training a diffusion-based trajectory planner on large-scale SOFA-generated synthetic data, transferring to real vascular geometries — targeting to overcome clinical data scarcity.

Affiliations: Tongji University × PKU DLIB | 2026

System Architecture

stEVE simulation pipeline: vessel tree, instrument, SOFA engine, and fluoroscopy rendering.

Catheter navigation in patient-specific vascular anatomy — final frame of a clinical test episode.

MacroHard Maker — CCF 3D Printing Path Planner

Sun, 01 Feb 2026 00:00:00 +0000

MacroHard Maker is a full-stack toolpath planning system for multi-axis continuous carbon fiber (CCF) FFF printers, aligning fiber deposition with principal stress directions for maximum structural performance.

macrohard-maker-core Python 后端核心：FastAPI + FEA + 路径规划 + G-code 生成 GitHub macrohard-maker-desktop 连续碳纤维 3D 打印路径规划桌面软件 GitHub

Algorithm pipeline:

FEA (scikit-fem) — linear elastic stress analysis, principal stress direction extraction
SIMP topology optimization — print direction enumeration to minimize overhang area
Geodesic path planning — stress-aligned surface routing on arbitrary geometries
XYA motion planner — helical winding + infill layer G-code generation (16× speedup)
Three.js visualization — real-time layer-by-layer preview in browser

Stack: Python · FastAPI · scikit-fem · Next.js 15 · Three.js · TypeScript · Tailwind

Open-Source Robotics Collaboration with 子豪兄

Sat, 01 Nov 2025 00:00:00 +0000

A series of open-source robotics projects co-built with — Tongji University’s most prominent AI/robotics education creator — focused on making embodied AI hardware accessible to the community.

OpenDuckMini — Bipedal Duck Robot

Hardware internals: 3D-printed PLA/PETG frame, servo actuators at each joint, Raspberry Pi onboard compute, custom PCB wiring harness.

Dynamic walking and running gait — field-tested on Tongji University track
3D-printed structural components (PLA/PETG), servo-actuated joints, Raspberry Pi compute

OpenDuckMini 完整构建文档硬件组装 · 标定流程 · 步态调试指南 Feishu Docs 子豪兄 Bilibili 主页同济大学 AI/机器人教育创作者 Bilibili miracle-techlink / GitHub 开源机器人项目代码仓库 GitHub

Quadruped Platform

Four-legged robot platform for reinforcement learning locomotion experiments and sim-to-real transfer research, built alongside the bipedal work as part of the same hardware studio.

Community & Mission

All projects are fully open-source and documented — reproducible by anyone with access to a 3D printer and basic electronics. The goal is to democratize embodied AI hardware and accelerate the research-to-product cycle for students and researchers across China.

LeRobot & Embodied AI Hackathon Projects

Wed, 01 Oct 2025 00:00:00 +0000

LeRobot — Study & Deployment

Deep dive into Hugging Face LeRobot — the leading open-source embodied intelligence framework — covering architecture analysis, training pipeline reproduction, and real-robot deployment.

Study scope:

ACT (Action Chunking with Transformers) policy architecture
Diffusion Policy training and inference pipeline
TDMPC world-model-based control
Real-robot deployment on SO-100 / Koch v1.1 hardware

📖 Co-authored comprehensive Chinese-language documentation. Officially featured by Hugging Face.

LeRobot 中文社区文档架构解析 · 训练流程 · 真机部署 · HuggingFace 官方收录 Feishu Docs

Connection to research: LeRobot’s flow-matching policy backbone directly informed the GDN-Memory integration work — understanding the inference pipeline was prerequisite to designing sub-1ms memory update scheduling.

tri.me — Home-Care Robot Platform

🏆 3rd Place — 小红书巅峰赛 Hackathon

tri.me 家庭护理机器人 Demo Tri.me吹了吗家庭护理机器人 · 小红书巅峰赛三等奖小红书 persuer0/treame 松灵 Nero 七自由度机械臂 · 刷牙 / 吹头发 / 剃胡子 GitHub

三类护理场景：刷牙 × 吹头发 × 剃胡子

场景	技术要点
🦷 刷牙	牙刷轨迹规划 × 力控清洁
💨 吹头发	头部跟随 × 角度自适应
🪒 剃胡子	面部轮廓贴合 × 微米级精度

三层解耦系统架构：感知层 / 控制层 / 执行层

三层解耦架构：

感知层 — 人体骨骼/人脸关键点实时检测，摄像头硬件加速编解码
控制层 — 机械臂 SDK，支持关节空间、笛卡尔、直线、圆弧多种运动模式，内置 SE3 坐标变换
执行层 — SocketCAN 协议驱动松灵 Nero，实现真正"以人为中心"的护理闭环

Medical AI Consultation Platform

Fri, 01 Aug 2025 00:00:00 +0000

An end-to-end medical AI platform developed from Oct 2024 to Aug 2025, covering both model training and product deployment.

Part 1 — Multimodal Medical LLM:

Three-stage fine-tuning: Medical Alignment → Instruction Tuning & Routing → Domain Expert Routing
Image-text cross-attention alignment for CT/MRI images and medical text
MoE-style routing: directs tasks to specialized expert modules (radiology, clinical text, etc.)
Architecture reference: EMNLP 2024 routing framework

Part 2 — Cloud Platform:

Coze multi-agent workflow for medication assistance and health check-in
Visual imaging analysis + report OCR pipeline
Web frontend: symptom analysis, drug recommendation, health knowledge base
Deployed and validated with real hospital feedback

Reflection: Identified product-market fit issues early — physician trust barriers and unclear monetization in B2B healthcare remain the core challenge for medical AI products.

Stack: PyTorch · Transformers · Coze · Vue · Cloud deployment

Model Training Pipeline

Three-stage fine-tuning: Medical Alignment → Instruction Tuning & Routing → Domain Expert Routing (reference: EMNLP 2024).

Coze Multi-Agent Workflow

Coze multi-agent workflow for medication assistance, health check-in, and domain routing.

Platform Frontend

智慧医疗AI平台 — symptom analysis, drug recommendation, visual report OCR, deployed and validated with hospital feedback.

AGV Reception & Inspection Robot — Tongji Deployment

Wed, 01 May 2024 00:00:00 +0000

Participated in the R&D of an intelligent AGV robot platform deployed across Tongji University’s exhibition hall and teaching buildings. The system integrates two robot roles — reception (迎宾) and inspection (巡检) — on a shared AGV chassis with a modular AI brain architecture.

My contribution: Led the voice module localization for Middle East clients — adapting the NLP pipeline for Arabic speech recognition, TTS, and interaction flows to support overseas commercial deployment.

Deployment

Left: Reception robot in Tongji University Exhibition Hall. Right: Inspection robot in Tongji teaching building.

System Architecture

Dual-brain architecture: AI大脑 (data analysis / model inference / HRI) + AGV大脑 (autonomous navigation controller) on a unified AGV platform.

The robot uses a dual-brain architecture:

AI 大脑 — handles 数据分析, 模型推理, 人机交互 (vision, NLP, multimodal interaction)
AGV 大脑 — autonomous navigation controller (SLAM, path planning, motor control)

Hardware Specs

Hardware: T1 industrial LiDAR, SONY 4G i640 compute unit, 60W×2 brushless drive motors, 24V/16Ah battery pack.

Module	Spec
LiDAR	单线激光雷达，180° FOV，测距精度 0.1°
Compute	SONY 4G i640，Ubuntu 16.04
Drive	橡胶车轮，60W×2 无刷电机，最高速 0.5m/s
Battery	18650锂电，24V/16Ah，续航 2h

Perception Capabilities

人员监测（人脸/穿戴/姿态/行为识别）× 环境监测（温湿度/烟雾/火警）

人员监测：

人脸识别与比对
穿戴识别（衣物、着装规范）
姿态检测（人体骨骼关键点）
行为识别（吸烟检测、摔倒预警）

环境监测：

温湿度检测（区域/物体温度）
烟雾报警（有害气体 + 烟雾检测）
火警预警（融合温度、热图、烟雾多源判断）

Face Recognition & Edge AI

人脸识别 + 红外测温实时上传，支持二次开发与定制化功能扩展。

人脸识别 + 红外测温数据实时上传
考勤数据大屏可视化
支持二次开发，可为客户定制化功能模块（含中东客户阿拉伯语语音交互落地）

Robotic Arm Simulation Engine — Tongji × Siemens Industry 4.0 Lab

Thu, 01 Feb 2024 00:00:00 +0000

Three.js real-time robotic arm visualization — adjusting slider parameters updates the 3D model in real time.

A joint research project between Tongji University’s Sino-German Industry 4.0 Lab and Siemens, replacing legacy particle swarm optimization (PSO) with deep reinforcement learning for higher-precision robotic arm control.

Technical work:

DQN agent for joint trajectory optimization, replacing PSO baseline
Three.js 3D visualization — interactive slider-based parameter control, real-time model updates, importable industrial equipment scenes
SITP project certification completed

Key insight: After exposure to NVIDIA Isaac Sim, identified that the core need is not model import/visualization but custom algorithm interfaces and GPU-accelerated physics simulation — pivoted research direction accordingly.

Affiliations: Tongji University Sino-German Industry 4.0 Lab × Siemens | 2023.11 – 2024.02

Smart Car Design — Provincial Competition Silver Award (Team Lead)

Thu, 01 Jun 2023 00:00:00 +0000

Led a team in designing and manufacturing a cam-steered autonomous smart car for the provincial-level competition, taking the project from concept through mechanical design, simulation, and hardware implementation.

Role: Team Lead — responsible for overall mechanical architecture, steering system design, trajectory modeling, and circuit design.

Mechanical Design

Early-stage CAD concept model — initial chassis layout and wheel configuration.

Final SolidWorks detailed render — cam steering mechanism, gear transmission, drive wheels, and onboard camera mount.

The steering system uses a cam-based mechanism to convert rotational motion into precise lateral displacement, eliminating the need for servo actuators and improving mechanical reliability. The drivetrain uses a gear reduction stage paired with brushed DC motors.

Trajectory Simulation

MATLAB trajectory simulation — left: full path trace (x/y in mm); top-right: vehicle turning angle vs. displacement; bottom-right: cam curve profile.

Developed a MATLAB model to simulate the vehicle trajectory based on the cam profile. Key outputs:

Path trace — verifies the car follows the intended curved route
Turning angle curve — validates smooth, continuous steering input
Cam profile — designed for minimal acceleration jerk and stable contact force

Charging Circuit

Custom charging circuit schematic — onboard battery management for the competition vehicle.

Designed a custom charging circuit with battery management to ensure reliable power delivery during competition runs.

Award: Silver Award, Provincial Smart Car Competition | Team Lead Stack: SolidWorks · MATLAB · PCB Design · DC Motor Control

Projects | Liu Yue — Embodied AI Researcher

GDN-Memory: Constant-Footprint Embodied Memory for World Models

Demo

Experiment Log

Exp A — GDN Object Memory: Synthetic Validation ✅ PASS

Exp B — GenieSim Real-Scene Validation 🔜 Planned

Exp C — GDN + GE-Act Integration 🔜 Planned

Exp D — System 2 Surprise Routing 🔜 Planned

Exp E — System 3 Persistent Prior 🔜 Planned

Endovascular Surgical Robot Platform (stEVE)

System Architecture

Clinical Navigation Demo

MacroHard Maker — CCF 3D Printing Path Planner

Open-Source Robotics Collaboration with 子豪兄

OpenDuckMini — Bipedal Duck Robot

Quadruped Platform

Community & Mission

LeRobot & Embodied AI Hackathon Projects

LeRobot — Study & Deployment

tri.me — Home-Care Robot Platform

Medical AI Consultation Platform

Model Training Pipeline

Coze Multi-Agent Workflow

Platform Frontend

AGV Reception & Inspection Robot — Tongji Deployment

Deployment

System Architecture

Hardware Specs

Perception Capabilities

Face Recognition & Edge AI

Robotic Arm Simulation Engine — Tongji × Siemens Industry 4.0 Lab

Smart Car Design — Provincial Competition Silver Award (Team Lead)

Mechanical Design

Trajectory Simulation

Charging Circuit