[COURSE] Add MIT6.5940: TinyML (#694)

* add tinyml

* nits

* nits
This commit is contained in:
Yinmin Zhong 2025-01-09 07:18:58 +08:00 committed by GitHub
parent 41aeee91f0
commit a06af57d9d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 55 additions and 1 deletions

View File

@ -0,0 +1,27 @@
# MIT6.5940: TinyML and Efficient Deep Learning Computing
## Descriptions
- Offered by: MIT
- Prerequisites: Computer architecture, Deep Learning
- Programming Languages: Python
- Difficulty: 🌟🌟🌟🌟
- Class Hour: 50h
This course, taught by MIT Professor [Song Han](https://hanlab.mit.edu/songhan), focuses on efficient machine learning techniques. Students are expected to have a pre-requisite of deep learning basics.
The course is divided into three main sections. The first section covers various key techniques for lightweight neural networks, such as pruning, quantization, distillation, and neural architecture search (NAS). Building on these foundations, the second section introduces efficient optimization techniques tailored to specific application scenarios. These include cutting-edge topics in deep learning, such as inference for large language models, long-context support, post-training acceleration, multimodal large language models, GANs, diffusion models, and so on. The third section focuses on efficient training techniques, such as large-scale distributed parallelism, automatic parallel optimization, gradient compression, and on-device training. Professor Song Hans lectures are clear and insightful, covering a wide range of topics, with a strong focus on trending areas. Those interested in gaining a foundational understanding of large language models may particularly benefit from the second and third sections.
The course materials and resources are available on the course website. Official lecture videos can be found on YouTube, and both raw and subtitled versions are available on Bilibili. There are five assignments in total: the first three focus on quantization, pruning, and NAS, while the last two involve compression and efficient deployment of large language models. The overall difficulty is relatively manageable, making the assignments an excellent way to solidify core knowledge.
## Course Resources
- Course Website: [2024fall](https://hanlab.mit.edu/courses/2024-fall-65940), [2023fall](https://hanlab.mit.edu/courses/2023-fall-65940)
- Recordings: [2024fall](https://www.youtube.com/playlist?list=PL80kAHvQbh-qGtNc54A6KW4i4bkTPjiRF), [2023fall](https://www.youtube.com/playlist?list=PL80kAHvQbh-pT4lCkDT53zT8DKmhE0idB)
- Textbooks: None
- Assignments: Five labs in total
## Personal Resources
All the resources and assignments used by @PKUFlyingPig in this course are maintained in [PKUFlyingPig/MIT6.5940_TinyML- GitHub](https://github.com/PKUFlyingPig/MIT6.5940_TinyML).

View File

@ -0,0 +1,26 @@
# MIT6.5940: TinyML and Efficient Deep Learning Computing
## 课程简介
- 所属大学MIT
- 先修要求:体系结构、深度学习基础、
- 编程语言Python
- 课程难度:🌟🌟🌟🌟
- 预计学时50小时
这门课由 MIT 的 [Song Han](https://hanlab.mit.edu/songhan) 教授讲授,侧重于高效的机器学习训练、推理技术。学生需要有一定的深度学习方面的知识基础。
课程主要分为三个部分首先讲授了让神经网络轻量化的各种关键技术例如剪枝、量化、蒸馏、网络架构搜索等等。有了这些基础之后课程第二部分会讲授面向特定领域场景的各种高效优化技术涉及了目前深度学习最前沿热门的各个方向例如大语言模型的推理、长上下文支持、后训练加速、多模态大语言模型、GAN、扩散模型等等。课程第三部分主要涉及各类高效训练技术例如大规模分布式并行、自动并行优化、梯度压缩、边缘训练等等。Song Han 教授的讲解深入浅出,覆盖的知识面很广,且都是当前热门的领域方向,如果是想对大语言模型有初步了解也可以重点关注第二和第三部分的内容。
课程内容和资源都可以在课程网站上找到视频在油管上有官方版本B站也有生肉和熟肉搬运可以自行查找。课程作业一共有5个前三个分别考察了量化、剪枝和 NAS后两个主要是对大语言模型的压缩和高效部署总体难度相对简单但能很好地巩固核心知识。
## 课程资源
- 课程网站:[2024fall](https://hanlab.mit.edu/courses/2024-fall-65940), [2023fall](https://hanlab.mit.edu/courses/2023-fall-65940)
- 课程视频:[2024fall](https://www.youtube.com/playlist?list=PL80kAHvQbh-qGtNc54A6KW4i4bkTPjiRF), [2023fall](https://www.youtube.com/playlist?list=PL80kAHvQbh-pT4lCkDT53zT8DKmhE0idB)
- 课程教材:无
- 课程作业共5个实验具体要求见课程网站
## 资源汇总
@PKUFlyingPig 在学习这门课中用到的所有资源和作业实现都汇总在 [PKUFlyingPig/MIT6.5940_TinyML - GitHub](https://github.com/PKUFlyingPig/MIT6.5940_TinyML) 中。

View File

@ -266,11 +266,12 @@ nav:
- 机器学习系统:
- "智能计算系统": "机器学习系统/AICS.md"
- "CMU 10-414/714: Deep Learning Systems": "机器学习系统/CMU10-414.md"
- "MIT6.5940: TinyML and Efficient Deep Learning Computing": "机器学习系统/EML.md"
- "Machine Learning Compilation": "机器学习系统/MLC.md"
- 深度学习:
- "UMich EECS 498-007 / 598-005: Deep Learning for Computer Vision": "深度学习/EECS498-007.md"
- "Coursera: Deep Learning": "深度学习/CS230.md"
- "国立台湾大学: 李宏毅机器学习": "深度学习/LHY.md"
- "UMich EECS 498-007 / 598-005: Deep Learning for Computer Vision": "深度学习/EECS498-007.md"
- "Stanford CS231n: CNN for Visual Recognition": "深度学习/CS231.md"
- "Stanford CS224n: Natural Language Processing": "深度学习/CS224n.md"
- "Stanford CS224w: Machine Learning with Graphs": "深度学习/CS224w.md"