Your Name
  • Bio
  • Papers
  • Startup
  • Blog
  • Experience
  • Projects
  • Awards
  • 中文 (简体)
ESC

Searching...

No results found

↑↓ Navigate ↵ Select
Powered by Hugo Blox
  • Recent & Upcoming Talks
    • Example Talk
  • Publications
    • GDN-Memory: Constant-Footprint Associative Memory for Embodied World Models
    • Synthetic-Data-Driven Diffusion Models for Endovascular Intervention Guidance
    • Stress-Driven G-Code Planning for Multi-Axis High-Speed Continuous Carbon Fiber 3D Printing
    • Efficient Inference Infrastructure for Deployable Vision-Language-Action Policies
  • Blog
    • 综述：VLA如何落地医疗机器人
    • LLM Finetuning Practice
    • LLM Finetuning Technical Report
    • VLA具身模型后训练综述
    • 拓扑优化与CFD — 高功率芯片微流体冷却系统设计
    • LLM底层架构 — 手撕 FlashAttention
    • LLM分布式训练 — DeepSpeed 实操
    • LLM分布式训练 — 数据并行策略
  • Awards & Competitions
  • Projects
    • GDN-Memory: Constant-Footprint Embodied Memory for World Models
    • Endovascular Surgical Robot Platform (stEVE)
    • MacroHard Maker — CCF 3D Printing Path Planner
    • Open-Source Robotics Collaboration with 子豪兄
    • LeRobot & Embodied AI Hackathon Projects
    • Medical AI Consultation Platform
    • AGV Reception & Inspection Robot — Tongji Deployment
    • Robotic Arm Simulation Engine — Tongji × Siemens Industry 4.0 Lab
    • Smart Car Design — Provincial Competition Silver Award (Team Lead)
  • Experience
    • Publications
      • GDN-Memory：具身世界模型的恒定占用联想记忆系统
      • 合成数据驱动的扩散模型用于血管腔内介入引导
      • 面向多轴高速连续碳纤维 3D 打印的应力驱动 G-code 规划
      • 面向可部署视觉-语言-动作策略的高效推理基础设施
    • Blog
      • 综述：VLA 如何落地医疗机器人
      • VLA 具身模型后训练综述
      • LLM 微调实战报告
      • LLM 微调实战：从 LoRA 到全参数
      • CFD 散热仿真：从建模到优化
      • FlashAttention 原理与工程实现解析
      • DeepSpeed ZeRO 训练大模型实战
      • PyTorch DDP 分布式训练实战
    • Projects
      • GDN-Memory: Constant-Footprint Embodied Memory for World Models
      • 血管介入手术机器人平台（stEVE）
      • MacroHard Maker — 连续碳纤维 3D 打印路径规划
      • OpenDuckMini — 开源双足机器人
      • LeRobot 与具身 AI 黑客松项目
      • 医疗 AI 问诊平台
      • AGV 迎宾巡检机器人 — 同济大学落地部署
      • 机械臂仿真引擎 — 同济×西门子工业 4.0 实验室
      • 智能小车设计 — 省赛银奖（队长）
    • 奖项与竞赛
    • 工作经历
    • _index
      • Liu Yue

LLM底层架构 — 手撕 FlashAttention

Apr 27, 2025 · 0 min read
Site
Last updated on Apr 27, 2025
LLM FlashAttention CUDA Systems
Liu Yue
Authors
Liu Yue (he/him)
Embodied AI Foundation Model Researcher / Startup Founder
← 拓扑优化与CFD — 高功率芯片微流体冷却系统设计 May 2, 2025
LLM分布式训练 — DeepSpeed 实操 Apr 23, 2025 →
Languages:
English
中文 (简体)

© 2026 Me. This work is licensed under CC BY NC ND 4.0

Made with Hugo Blox — Open Source. Build yours →