1+ <div align =" center " >
2+
13# WayToAGI Gesture Control
24
3- ![ Python] ( https://img.shields.io/badge/python-3.10%2B-blue )
4- ![ Framework] ( https://img.shields.io/badge/backend-Flask%20%2B%20Socket.IO-black )
5- ![ Vision] ( https://img.shields.io/badge/vision-MediaPipe-green )
6- ![ License] ( https://img.shields.io/badge/license-MIT-lightgrey )
5+ Real-time hand-tracking mouse control system for desktop interaction.
6+
7+ [ ![ Python] ( https://img.shields.io/badge/python-3.10%2B-blue.svg )] ( https://www.python.org/ )
8+ [ ![ Flask] ( https://img.shields.io/badge/backend-Flask%20%2B%20Socket.IO-black.svg )] ( https://flask.palletsprojects.com/ )
9+ [ ![ MediaPipe] ( https://img.shields.io/badge/vision-MediaPipe-00A67E.svg )] ( https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker )
10+ [ ![ CI] ( https://img.shields.io/badge/CI-GitHub_Actions-2088FF.svg )] ( https://github.com/sudo-yf/hackathon2510/actions )
11+ [ ![ License] ( https://img.shields.io/badge/license-MIT-green.svg )] ( LICENSE )
12+
13+ </div >
14+
15+ ## Overview
16+
17+ WayToAGI Gesture Control 是一个面向实时人机交互的手势控制系统:通过摄像头捕获手部关键点,完成光标映射与点击手势识别,并通过 Web UI 提供低延迟可视化与参数调优能力。
18+
19+ ## Table of Contents
720
8- WayToAGI Hackathon 作品。
9- 一个面向实时交互的手势控制系统:摄像头识别手部关键点,后端完成动作映射,前端提供全屏可视化与低延迟控制面板。
21+ - [ Overview] ( #overview )
22+ - [ Key Features] ( #key-features )
23+ - [ System Architecture] ( #system-architecture )
24+ - [ Quick Start] ( #quick-start )
25+ - [ Docker Deployment] ( #docker-deployment )
26+ - [ Runtime Configuration] ( #runtime-configuration )
27+ - [ HTTP API] ( #http-api )
28+ - [ Project Structure] ( #project-structure )
29+ - [ Design References] ( #design-references )
1030
11- ## Highlights
31+ ## Key Features
1232
13- - Real-time 手势识别(21 landmarks)
14- - 鼠标映射算法(中心增稳 + 边缘加速)
15- - 动作触发(左键/右键点击)
16- - 设置持久化(SQLite)
17- - Docker 运行与 ` uv ` 环境管理
33+ - 21-point hand landmark tracking (MediaPipe Hands)
34+ - 非线性光标映射(中心稳态 + 边缘加速)
35+ - 手势点击触发(左键 / 右键)
36+ - Socket.IO 实时视频帧与状态推送
37+ - SQLite 参数持久化(平滑系数、灵敏度、边缘加速)
38+ - 可在无系统鼠标注入模式下运行(` DISABLE_MOUSE=1 ` )
1839
19- ## Architecture
40+ ## System Architecture
2041
2142``` mermaid
2243graph LR
23- Cam[Camera Stream] --> Vision[MediaPipe Hands]
24- Vision --> Engine[Gesture Engine]
25- Engine --> Mouse[Mouse Injection]
26- Engine --> WS[Socket.IO]
27- WS --> UI[Web UI]
44+ Cam[Camera Input] --> Vision[MediaPipe Hands]
45+ Vision --> Gesture[Gesture Engine]
46+ Gesture --> Cursor[Cursor Mapping]
47+ Gesture --> Click[Gesture Click Detection]
48+ Cursor --> Mouse[Mouse Backend]
49+ Gesture --> WS[Socket.IO Stream]
50+ WS --> UI[Web Dashboard]
2851 UI --> API[Flask API]
29- API --> Engine
30- API --> DB[(SQLite Settings)]
52+ API --> Store[(SQLite Settings)]
3153```
3254
33- ## Project Layout
55+ ## Quick Start
3456
35- ``` text
36- hackathon2510/
37- ├── app.py
38- ├── src/waytoagi_gesture/
39- │ ├── config.py
40- │ ├── service.py
41- │ ├── settings_store.py
42- │ ├── web.py
43- │ └── static/
44- ├── tests/
45- ├── pyproject.toml
46- ├── Makefile
47- ├── Dockerfile
48- └── RELEASE.md
57+ ### 1. Install Dependencies (uv)
58+
59+ ``` bash
60+ uv sync
4961```
5062
51- ## Quick Start
63+ ### 2. Run Service
5264
5365``` bash
54- uv sync
5566uv run python app.py
5667```
5768
58- Open [ http://localhost:5000 ] ( http://localhost:5000 )
69+ Open: [ http://localhost:5000 ] ( http://localhost:5000 )
5970
60- ## Local Quality Gates
71+ ### 3. Quality Gates
6172
6273``` bash
6374make lint
6475make test
6576make check
6677```
6778
68- ## Docker
79+ ## Docker Deployment
6980
7081``` bash
7182docker build -t waytoagi-gesture:latest .
@@ -79,29 +90,54 @@ docker run --rm -it \
7990 waytoagi-gesture:latest
8091```
8192
82- ## Configuration
93+ ## Runtime Configuration
8394
84- | Key | Default | Description |
95+ | Env | Default | Description |
8596| ---| ---| ---|
8697| ` HOST ` | ` 0.0.0.0 ` | 服务监听地址 |
8798| ` PORT ` | ` 5000 ` | 服务端口 |
8899| ` CAMERA_INDEX ` | ` 0 ` | 摄像头索引 |
89- | ` FRAME_WIDTH ` | ` 1280 ` | 视频宽度 |
90- | ` FRAME_HEIGHT ` | ` 720 ` | 视频高度 |
91- | ` FRAME_FPS ` | ` 30 ` | 视频帧率 |
92- | ` SETTINGS_DB_PATH ` | ` data/waytoagi.db ` | SQLite 配置库 |
93- | ` DISABLE_MOUSE ` | ` 0 ` | 禁用系统鼠标注入 |
100+ | ` FRAME_WIDTH ` | ` 1280 ` | 采集宽度 |
101+ | ` FRAME_HEIGHT ` | ` 720 ` | 采集高度 |
102+ | ` FRAME_FPS ` | ` 30 ` | 采样帧率 |
103+ | ` CORS_ORIGIN ` | ` * ` | 前端跨域来源 |
104+ | ` SETTINGS_DB_PATH ` | ` data/waytoagi.db ` | SQLite 配置存储路径 |
105+ | ` DISABLE_MOUSE ` | ` 0 ` | 禁用真实鼠标注入(1 为禁用) |
106+
107+ ## HTTP API
108+
109+ | Method | Path | Description |
110+ | ---| ---| ---|
111+ | ` GET ` | ` /healthz ` | 服务与摄像头状态 |
112+ | ` GET ` | ` /api/settings ` | 读取当前控制参数 |
113+ | ` POST ` | ` /api/settings ` | 更新控制参数 |
114+ | ` POST ` | ` /api/camera/start ` | 启动摄像头处理线程 |
115+ | ` POST ` | ` /api/camera/stop ` | 停止摄像头处理线程 |
116+ | ` POST ` | ` /api/mouse/toggle ` | 切换鼠标控制开关 |
117+ | ` POST ` | ` /api/settings/flip ` | 切换镜像翻转 |
94118
95- ## API
119+ ## Project Structure
120+
121+ ``` text
122+ hackathon2510/
123+ ├── app.py
124+ ├── src/waytoagi_gesture/
125+ │ ├── web.py
126+ │ ├── service.py
127+ │ ├── config.py
128+ │ ├── settings_store.py
129+ │ └── static/
130+ ├── tests/
131+ ├── Dockerfile
132+ ├── Makefile
133+ ├── pyproject.toml
134+ └── LICENSE
135+ ```
96136
97- - ` GET /healthz `
98- - ` GET /api/settings `
99- - ` POST /api/settings `
100- - ` POST /api/camera/start `
101- - ` POST /api/camera/stop `
102- - ` POST /api/mouse/toggle `
103- - ` POST /api/settings/flip `
137+ ## Design References
104138
105- ## Release Policy
139+ - [ MediaPipe] ( https://github.com/google-ai-edge/mediapipe )
140+ - [ OpenCV] ( https://github.com/opencv/opencv )
141+ - [ Flask] ( https://github.com/pallets/flask )
142+ - [ Flask-SocketIO] ( https://github.com/miguelgrinberg/Flask-SocketIO )
106143
107- See [ RELEASE.md] ( RELEASE.md )
0 commit comments