KuangjuX’s Notes

外观

GPU 编译与性能 · 论文 ​

论文总览 · 基础笔记

GPU 编程、张量编译器、Megakernel、性能优化与相关项目。

共 64 个清单条目；已读 29，待读 35。保留原有跨分类重复条目与阅读状态。

本地笔记 ​

Deep Learning Compiler ​

StatusPaperVenueWhy It MattersLinks
✅The Deep Learning Compiler: A Comprehensive SurveyTPDS'21梳理多层 IR、图优化与后端调度，建立深度学习编译器的整体知识框架Paper / Note / 归档标签
✅MLIR: Scaling Compiler Infrastructure for Domain Specific ComputationCGO'21通过可扩展 dialect 与逐层 lowering 复用编译基础设施，理解现代 AI 编译器架构Paper / Note
✅TIRAMISU: A Polyhedral Compiler for Expressing Fast and Portable CodeCGO'19分离算法、循环变换、数据布局和通信，理解多面体调度与跨硬件代码生成Paper / Note / 归档标签
✅ 📝Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasksOSDI'20用 rTask 统一算子内外并行性，在编译期生成静态时空调度以减少运行时开销Paper / Note / 归档标签
✅ 📝ROLLER: Fast and Efficient Tensor Compilation for Deep LearningOSDI'22用硬件感知 rTile 和性能模型构造候选 kernel，缩短传统大规模自动调优时间Paper / Note / 归档标签
✅ 📝BOLT: Bridging The Gap Between Auto-Tuners and Hardware-Native PerformanceMLSys'22利用 CUTLASS 等硬件原生模板进行搜索，连接自动调优的灵活性与库级性能Paper / Note / 归档标签
✅ 📝AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT ArchitecturesASPLOS'22联合探索线程映射、片上复用与算子融合，优化访存密集型训练和推理算子Paper / Note / 归档标签
✅ 📝AMOS: Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware AbstractionISCA'22将硬件 intrinsic 抽象成可分析的计算与访存语义，自动生成张量到加速器的映射Paper / Note / 归档标签
✅ 📝Welder: Scheduling Deep Learning Memory Access via Tile-graphOSDI'23用 tile-graph 与数据流量模型统一算子内外数据复用，降低端到端访存开销Paper / Note / 归档标签
✅Effectively Scheduling Computational Graphs of Deep Neural Networks toward Their Domain-Specific AcceleratorsOSDI'23将硬件资源纳入子图划分和跨层指令调度，减少片外数据搬运并提高加速器利用率Paper / 归档标签
✅ 📝Cocktailer: Analyzing and Optimizing Dynamic Control Flow in Deep LearningOSDI'23以 uTask 统一控制流和数据流，把动态分支与循环下沉到加速器以减少 CPU 同步Paper / Note / 归档标签
✅ 📝Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators FusionHPCA'23用分析模型选择计算块执行顺序，并结合硬件微内核优化计算密集型算子链融合Paper / Note / 归档标签
✅ 📝Graphene: An IR for Optimized Tensor Computations on GPUsASPLOS'23将数据和线程都表示为可分块张量，显式表达 GPU tensor 指令要求的复杂映射Paper / Note / 归档标签
✅Uncovering Nested Data Parallelism and Data Reuse in DNN Computation with FractalTensorSOSP'24通过嵌套张量与数组算子表达跨循环、跨算子的并行性和复用，突破平面计算图限制Paper
✅ThunderKittens: Simple, Fast, and Adorable AI KernelsICLR'25以 tile 数据结构、异步流水线模板和网格调度简化高性能 GPU kernel 编写Paper / 归档标签
✅Mirage: A Multi-Level Superoptimizer for Tensor ProgramsOSDI'25用多层 µGraph 搜索代数和调度变换，并验证等价性，探索超越既有算子库的融合Paper / 归档标签
✅PipeThreader: Software-Defined Pipelining for Efficient DNN ExecutionOSDI'25用 sTask-graph 和软件定义流水线协调 GPU 专用单元，自动发现计算与数据搬运的重叠方案Paper
✅TileLang: A Composable Tiled Programming Model for AI SystemsarXiv'25将 tile 数据流与布局、线程映射和流水线调度解耦，降低高性能 AI kernel 编程负担Paper / 归档标签
✅ 📝Tawa: Automatic Warp Specialization for Modern GPUs with Asynchronous ReferencesCGO'26以异步引用表达依赖并自动生成 warp specialization，减少手工管理并发和同步的负担Paper / 归档标签 / Note
✅KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI WorkloadsOSDI'25将性能测量实现为编译器 pass，观察 GPU kernel 内部的细粒度执行与重叠瓶颈Paper / 归档标签
⬜Optimal Software Pipelining and Warp Specialization for Tensor Core GPUsOSDI'26将软件流水线和 warp specialization 联合建模为约束优化，推导模型假设下的最优调度Paper
⬜GraCE: Unlocking CUDA Graphs with Compiler Support for ML WorkloadsOSDI'26自动变换程序并分析成本收益，扩大 CUDA Graph 覆盖率并减少参数拷贝和启动开销Paper
⬜AffineGraph: A Dataflow Graph Representation for Effective DNN Hardware-Aware Algorithms来源待补全标签：GPU 编译与性能优化PDF / Zotero / 归档标签
⬜Ansor: Generating High-Performance Tensor Programs for Deep Learning来源待补全标签：GPU 编译与性能优化 / 编译器架构与调度PDF / Zotero / 归档标签
⬜Graphiler: A Compiler for Graph Neural Networks来源待补全标签：GPU 编译与性能优化 / 编译器架构与调度PDF / Zotero / 归档标签
⬜The tensor algebra compilerProceedings of the ACM on Programming Languages · 2017标签：GPU 编译与性能优化 / 编译器架构与调度Paper / PDF / Zotero / 归档标签

LLM Inference / MegaKernel ​

StatusPaperVenueWhy It MattersLinks
✅ 📝Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1BBlog'25以 Llama-1B 实例解释 megakernel 如何消除 kernel 边界气泡，并管理细粒度同步和共享内存Paper / Note
✅ 📝MPK: A Compiler and Runtime for Mega-Kernelizing Tensor ProgramsOSDI'26将多 GPU 推理编译成 SM 级任务图，在持久化 kernel 内调度计算与通信以降低端到端延迟Paper / 归档标签 / Note
✅ 📝Event Tensor: A Unified Abstraction for Compiling Dynamic MegakernelMLSys'26以 Event Tensor 统一 tile 任务依赖、动态形状和数据相关控制流，支持动态 megakernel 编译Paper / Note / 归档标签
✅ 📝TileRT: Tile-Based Runtime for Ultra-Low-Latency LLM InferenceGitHub project'25用 tile 级运行时重排并重叠计算、I/O 与跨设备通信，探索大模型低延迟推理的工程路径Paper / Note
✅Compiling LLMs into a MegaKernel: A Path to Low-Latency InferenceBlog从编译器和运行时角度解释全模型 megakernel，连接任务依赖、跨算子调度与低延迟推理Paper
⬜Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM InferencearXiv'26用离线 DAG 搜索固定 megakernel 执行路径，减少资源受限 GPU 上动态调度和共享内存开销Paper
⬜ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective PrimitivearXiv · 2025标签：GPU 编译与性能优化Paper / PDF / Zotero / 归档标签
⬜mKernel: Fast Multi-GPU, Multi-Node Fused KernelsarXiv · 2026标签：GPU 编译与性能优化Paper / PDF / Zotero / 归档标签

LLM for Kernel Optimization ​

StatusPaperVenueWhy It MattersLinks
✅AVO: Agentic Variation Operators for Autonomous Evolutionary SearcharXiv'26用自主编码 agent 替代固定变异与交叉算子，让进化搜索自适应生成和验证程序候选Paper / Note
✅ 📝CAKE: Compiler-Agent Co-Design for Frontier Kernel EvolutionarXiv'26让 agent 编写显式硬件调度 IR，并结合验证、成本模型与局部诊断迭代优化 GPU kernelPaper / Note / 归档标签
⬜FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM SystemsMLSys'26以真实 LLM 工作负载、正确性验证和性能评测连接 AI kernel 生成与系统部署Paper
⬜Harness Engineering for LLM-Driven GPU Kernel GenerationarXiv'26围绕约束、正确性验证、性能分析和候选筛选设计 harness，提高 LLM 生成 GPU kernel 的可靠性Paper
⬜Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDATech report'26在正确性门控的 FlashInfer-Bench 流程中评估通用代码 agent 自动优化 MoE 与 DSA kernel 的能力Paper
⬜Open-CAKE-IR：跨硬件优化机制复用与编译器协同演进技术报告 · 2026标签：GPU 编译与性能优化PDF / Zotero / 归档标签

GPU Microarchitecture ​

StatusPaperVenueWhy It MattersLinks
⬜Understanding Latency Hiding on GPUsPhD thesis'16用简化微基准与性能模型解释 GPU 延迟隐藏，分析线程数、算术强度和吞吐的关系Paper

Math Foundations ​

StatusPaperVenueWhy It MattersLinks
⬜Categorical Foundations for CuTe LayoutsarXiv'26用范畴与布局代数形式化 CuTe 的组合、乘积和除法，并验证其与实现一致Paper / 中文全文 / 2025-09 PDF / Zotero

Compiler ​

StatusPaperVenueWhy It MattersLinks
✅HIDA: A Hierarchical Dataflow Compiler for High-Level SynthesisASPLOS'24用分层数据流表示与多级优化自动生成 FPGA 数据流架构，减少手工 HLS 调优Paper / Note / 归档标签

GPU 编译与性能优化 / Kernel Fusion ​

StatusPaperVenueWhy It MattersLinks
⬜CODA: Rewriting Transformer Blocks as GEMM-Epilogue ProgramsarXiv · 2026标签：GPU 编译与性能优化Paper / PDF / Zotero / 归档标签
⬜Automatic Horizontal Fusion for GPU Kernels2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO) · 2022标签：GPU 编译与性能优化 / Kernel FusionPaper / PDF / Zotero / 归档标签

GPU 编译与性能优化 / GPU 运行时与性能工具 ​

StatusPaperVenueWhy It MattersLinks
⬜TileSight: A First-Principles Tile-Centric Analytical GPU Performance Model from Cores to ClustersarXiv · 2026标签：GPU 编译与性能优化Paper / PDF / Zotero / 归档标签
⬜X-Stage: Modeling Post-Issue Backpressure in GPU Communication--Computation FusionarXiv · 2026标签：GPU 编译与性能优化Paper / PDF / Zotero / 归档标签

GPU 编译与性能优化 / GPU 算子 ​

StatusPaperVenueWhy It MattersLinks
⬜FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor CoresarXiv · 2023标签：GPU 编译与性能优化 / GPU 算子Paper / PDF / Zotero / 归档标签

GPU 编译与性能优化 / GPU 编程语言与 DSL ​

StatusPaperVenueWhy It MattersLinks
⬜Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model TrainingarXiv · 2023标签：GPU 编译与性能优化 / GPU 编程语言与 DSLPaper / PDF / Zotero / 归档标签
⬜Task-Based Tensor Computations on Modern GPUsarXiv · 2025标签：GPU 编译与性能优化 / GPU 编程语言与 DSLPaper / PDF / Zotero / 归档标签
⬜Tilus: A Virtual Machine for Arbitrary Low-Precision GPGPU Computation in LLM ServingarXiv · 2025标签：GPU 编译与性能优化 / GPU 编程语言与 DSLPaper / PDF / Zotero / 归档标签

GPU 编译与性能优化 / IR 与张量化 ​

StatusPaperVenueWhy It MattersLinks
⬜Dual-side Sparse Tensor Core2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) · 2021标签：GPU 编译与性能优化 / IR 与张量化Paper / PDF / Zotero / 归档标签
⬜LADDER: Enabling Efficient Low-Precision Deep Learning Computing through Hardware-aware Tensor Transformation来源待补全标签：GPU 编译与性能优化 / IR 与张量化PDF / Zotero / 归档标签
⬜MLIR-based code generation for GPU tensor coresProceedings of the 31st ACM SIGPLAN International Conference on Compiler Construction · 2022标签：GPU 编译与性能优化 / IR 与张量化Paper / PDF / Zotero / 归档标签
⬜PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation来源待补全 · 2023标签：GPU 编译与性能优化 / IR 与张量化PDF / Zotero / 归档标签
✅SparseTIR: Composable Abstractions for Sparse Compilation in Deep LearningProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 · 2023标签：GPU 编译与性能优化 / IR 与张量化Paper / PDF / PDF2 / PDF3 / Zotero / 归档标签

GPU 编译与性能优化 / 分布式编译 ​

StatusPaperVenueWhy It MattersLinks
⬜DISTAL: the distributed tensor algebra compilerProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation · 2022标签：GPU 编译与性能优化 / 分布式编译Paper / PDF / Zotero / 归档标签

GPU 编译与性能优化 / 计算图优化 ​

StatusPaperVenueWhy It MattersLinks
⬜DNNFusion: accelerating deep neural networks execution with advanced operator fusionProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation · 2021标签：GPU 编译与性能优化 / 计算图优化Paper / PDF / Zotero / 归档标签
⬜EINNET: Optimizing Tensor Programs with Derivation-Based Transformations来源待补全标签：GPU 编译与性能优化 / 计算图优化PDF / Zotero / 归档标签
⬜PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections来源待补全标签：GPU 编译与性能优化 / 计算图优化PDF / Zotero / 归档标签
⬜TASO: optimizing deep learning computation with automatic generation of graph substitutionsProceedings of the 27th ACM Symposium on Operating Systems Principles · 2019标签：GPU 编译与性能优化 / 计算图优化Paper / PDF / Zotero / 归档标签

GPU 编译与性能优化 / 软硬件协同设计 ​

StatusPaperVenueWhy It MattersLinks
⬜CoSA: Scheduling by Constrained Optimization for Spatial Accelerators2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) · 2021标签：GPU 编译与性能优化 / 软硬件协同设计Paper / PDF / Zotero / 归档标签
⬜HASCO: Towards Agile HArdware and Software CO-design for Tensor Computation2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) · 2021标签：GPU 编译与性能优化 / 软硬件协同设计Paper / PDF / Zotero / 归档标签
⬜SARA: Scaling a Reconfigurable Dataflow Accelerator2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) · 2021标签：GPU 编译与性能优化 / 软硬件协同设计Paper / PDF / Zotero / 归档标签

记录原理、连接知识、积累实践。

Copyright © KuangjuX