feat: initial EcoArk HarmonyOS wheel pipeline skeleton
- wheel_pipeline.py: CLI entry point for repack/rebuild flow - pipeline/detect.py: detect pure vs native extension wheels - pipeline/repack.py: repack pure-Python wheels with harmonyos_arm64 tag - pipeline/rebuild.py: native wheel rebuild entry (stub for OHOS NDK) - pipeline/cli.py: argument parsing and orchestration - Makefile: convenience targets (repack, rebuild, clean) - README.md: documentation with usage and limitations - examples/gen_test_wheels.py: test wheel generator - output/: output directory for generated wheels
This commit is contained in:
32
Makefile
Normal file
32
Makefile
Normal file
@ -0,0 +1,32 @@
|
||||
SHELL := /bin/bash
|
||||
|
||||
INPUT ?= examples/sample.whl
|
||||
OUTPUT ?= output
|
||||
|
||||
.PHONY: help repack rebuild native clean
|
||||
|
||||
help:
|
||||
@echo "EcoArk HarmonyOS Wheel Pipeline"
|
||||
@echo ""
|
||||
@echo "Usage:"
|
||||
@echo " make repack INPUT=path/to/pkg.whl — Repack a pure-Python wheel for HarmonyOS"
|
||||
@echo " make rebuild INPUT=path/to/pkg.whl — Trigger native rebuild entry (stub)"
|
||||
@echo " make clean — Remove output/"
|
||||
@echo ""
|
||||
@echo "Examples:"
|
||||
@echo " make repack INPUT=some_package.whl"
|
||||
@echo " make rebuild INPUT=some_native.whl"
|
||||
|
||||
repack:
|
||||
python3 wheel_pipeline.py "$(INPUT)" --output-dir "$(OUTPUT)" --force-repack
|
||||
|
||||
rebuild:
|
||||
python3 wheel_pipeline.py "$(INPUT)" --output-dir "$(OUTPUT)" --target-os harmonyos --arch arm64
|
||||
|
||||
native:
|
||||
python3 wheel_pipeline.py "$(INPUT)" --output-dir "$(OUTPUT)" --target-os harmonyos --arch arm64
|
||||
|
||||
clean:
|
||||
rm -rf "$(OUTPUT)"
|
||||
mkdir -p "$(OUTPUT)"
|
||||
touch "$(OUTPUT)/.gitkeep"
|
||||
83
README.md
Normal file
83
README.md
Normal file
@ -0,0 +1,83 @@
|
||||
# EcoArk HarmonyOS Wheel Pipeline
|
||||
|
||||
为 **EcoArk / HarmonyOS Python 环境** 准备的 wheel 重打包与重编译流水线。
|
||||
|
||||
## 用途
|
||||
|
||||
- 接收一个 `.whl` 文件作为输入
|
||||
- 自动检测是 **纯 Python wheel** 还是 **含 native 扩展的 wheel**
|
||||
- 纯 Python wheel → 直接重打包为 `harmonyos_arm64` 兼容标签,安装即用
|
||||
- Native wheel → 预留 HarmonyOS/arm64 交叉编译入口(当前为骨架,可扩展)
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
ecoarkpywhl/
|
||||
├── wheel_pipeline.py # CLI 入口(最简调用)
|
||||
├── pipeline/
|
||||
│ ├── cli.py # 参数解析 & 主流程
|
||||
│ ├── detect.py # 检测 wheel 类型(纯 Python / native)
|
||||
│ ├── repack.py # 纯 Python wheel 重打包
|
||||
│ └── rebuild.py # Native wheel 重编译入口(骨架)
|
||||
├── Makefile # 常用命令
|
||||
├── output/ # 输出目录(生成文件放这里)
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
|
||||
```bash
|
||||
# 纯 Python wheel → 重打包
|
||||
python3 wheel_pipeline.py some_pure_package.whl
|
||||
|
||||
# 指定输出目录
|
||||
python3 wheel_pipeline.py some_pure_package.whl --output-dir /tmp/wheels
|
||||
|
||||
# Native wheel → 触发重编译入口(当前只复制 + 写 stub)
|
||||
python3 wheel_pipeline.py some_native_package.whl
|
||||
```
|
||||
|
||||
或通过 Makefile:
|
||||
|
||||
```bash
|
||||
make repack INPUT=some_package.whl
|
||||
make rebuild INPUT=some_native_package.whl
|
||||
```
|
||||
|
||||
## 限制
|
||||
|
||||
| 场景 | 状态 |
|
||||
|------|------|
|
||||
| 纯 Python wheel → 重打包安装 | ✅ 支持 |
|
||||
| Pure Python → 重新打 tag | ✅ 支持 |
|
||||
| Native wheel → HarmonyOS 重编译 | 🚧 预留入口,需接入 OHOS NDK 工具链 |
|
||||
| 依赖复杂的 native 包(numpy/scipy 等) | 🚧 需要对应依赖也移植到 HarmonyOS |
|
||||
|
||||
### 纯 Python wheel
|
||||
|
||||
纯 Python wheel(文件名中包含 `py3-none-any` 等标签,且不含 `.so`/`.pyd` 文件)可以直接重打包为 `harmonyos_arm64` 标签,在 HarmonyOS Python 环境中安装运行。
|
||||
|
||||
```bash
|
||||
python3 wheel_pipeline.py mypackage.whl -o output/
|
||||
# → output/mypackage-1.0.0-py3-none-harmonyos_arm64.whl
|
||||
```
|
||||
|
||||
### Native wheel
|
||||
|
||||
含 C/C++/Rust 等 native 扩展的 wheel 需要:
|
||||
|
||||
1. HarmonyOS SDK / OHOS NDK
|
||||
2. 对应架构(arm64)的交叉编译工具链
|
||||
3. Python 头文件与目标 Python 版本匹配
|
||||
4. 所有 C 依赖也已移植到 HarmonyOS
|
||||
|
||||
当前版本仅输出 stub,待后续完善。
|
||||
|
||||
## 依赖
|
||||
|
||||
- Python ≥ 3.8(标准库,无需额外安装包)
|
||||
- 对 native 包:HarmonyOS 工具链(外部,不在本仓库内)
|
||||
|
||||
## 许可证
|
||||
|
||||
本项目许可证见仓库根目录。
|
||||
80
examples/gen_test_wheels.py
Normal file
80
examples/gen_test_wheels.py
Normal file
@ -0,0 +1,80 @@
|
||||
import zipfile
|
||||
import os
|
||||
import json
|
||||
|
||||
EXAMPLES_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
|
||||
def make_pure_wheel(name, version, out_dir):
|
||||
dist_info = f"{name}-{version}.dist-info"
|
||||
os.makedirs(f"/tmp/{dist_info}", exist_ok=True)
|
||||
|
||||
metadata = f"""Metadata-Version: 2.1
|
||||
Name: {name}
|
||||
Version: {version}
|
||||
Summary: Test pure-Python wheel
|
||||
"""
|
||||
|
||||
with open(f"/tmp/{dist_info}/METADATA", "w") as f:
|
||||
f.write(metadata)
|
||||
with open(f"/tmp/{dist_info}/WHEEL", "w") as f:
|
||||
f.write("Wheel-Version: 1.0\nGenerator: test\nRoot-Is-Purelib: true\nTag: py3-none-any\n")
|
||||
with open(f"/tmp/{dist_info}/RECORD", "w") as f:
|
||||
f.write("")
|
||||
|
||||
wheel_name = f"{name}-{version}-py3-none-any.whl"
|
||||
wheel_path = os.path.join(out_dir, wheel_name)
|
||||
|
||||
with zipfile.ZipFile(wheel_path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.write(f"/tmp/{dist_info}/METADATA", f"{dist_info}/METADATA")
|
||||
zf.write(f"/tmp/{dist_info}/WHEEL", f"{dist_info}/WHEEL")
|
||||
zf.write(f"/tmp/{dist_info}/RECORD", f"{dist_info}/RECORD")
|
||||
pkg_dir = f"{name}"
|
||||
zf.writestr(f"{pkg_dir}/__init__.py", "def hello(): return 'hello'\n")
|
||||
|
||||
for f in os.listdir(f"/tmp/{dist_info}"):
|
||||
os.remove(f"/tmp/{dist_info}/{f}")
|
||||
os.rmdir(f"/tmp/{dist_info}")
|
||||
|
||||
return wheel_path
|
||||
|
||||
|
||||
def make_native_wheel(name, version, out_dir):
|
||||
dist_info = f"{name}-{version}.dist-info"
|
||||
os.makedirs(f"/tmp/{dist_info}", exist_ok=True)
|
||||
|
||||
metadata = f"""Metadata-Version: 2.1
|
||||
Name: {name}
|
||||
Version: {version}
|
||||
Summary: Test native wheel
|
||||
"""
|
||||
|
||||
with open(f"/tmp/{dist_info}/METADATA", "w") as f:
|
||||
f.write(metadata)
|
||||
with open(f"/tmp/{dist_info}/WHEEL", "w") as f:
|
||||
f.write("Wheel-Version: 1.0\nGenerator: test\nRoot-Is-Purelib: false\nTag: cp38-cp38-manylinux_2_17_x86_64\n")
|
||||
with open(f"/tmp/{dist_info}/RECORD", "w") as f:
|
||||
f.write("")
|
||||
|
||||
wheel_name = f"{name}-{version}-cp38-cp38-manylinux_2_17_x86_64.whl"
|
||||
wheel_path = os.path.join(out_dir, wheel_name)
|
||||
|
||||
with zipfile.ZipFile(wheel_path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.write(f"/tmp/{dist_info}/METADATA", f"{dist_info}/METADATA")
|
||||
zf.write(f"/tmp/{dist_info}/WHEEL", f"{dist_info}/WHEEL")
|
||||
zf.write(f"/tmp/{dist_info}/RECORD", f"{dist_info}/RECORD")
|
||||
zf.writestr(f"{name}/__init__.py", "def hello(): return 'hello'\n")
|
||||
zf.writestr(f"{name}/_native.cpython-38-x86_64-linux-gnu.so", b"\x7fELF...fake")
|
||||
|
||||
for f in os.listdir(f"/tmp/{dist_info}"):
|
||||
os.remove(f"/tmp/{dist_info}/{f}")
|
||||
os.rmdir(f"/tmp/{dist_info}")
|
||||
|
||||
return wheel_path
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
out = EXAMPLES_DIR
|
||||
pure = make_pure_wheel("ecoark_demo", "0.1.0", out)
|
||||
native = make_native_wheel("ecoark_native_demo", "0.1.0", out)
|
||||
print(json.dumps({"pure": pure, "native": native}, indent=2))
|
||||
0
output/.gitkeep
Normal file
0
output/.gitkeep
Normal file
0
pipeline/__init__.py
Normal file
0
pipeline/__init__.py
Normal file
BIN
pipeline/__pycache__/__init__.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/__init__.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/cli.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/cli.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/detect.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/detect.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/rebuild.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/rebuild.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/repack.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/repack.cpython-312.pyc
Normal file
Binary file not shown.
63
pipeline/cli.py
Normal file
63
pipeline/cli.py
Normal file
@ -0,0 +1,63 @@
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
from . import detect, repack, rebuild
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="EcoArk HarmonyOS Wheel Pipeline — repack / rebuild Python wheels for HarmonyOS/arm64"
|
||||
)
|
||||
parser.add_argument("input", help="Path to a .whl file")
|
||||
parser.add_argument("--output-dir", "-o", default="output", help="Output directory (default: ./output)")
|
||||
parser.add_argument("--target-os", default="harmonyos", choices=["harmonyos", "linux", "ohos"])
|
||||
parser.add_argument("--arch", default="arm64", choices=["arm64", "aarch64", "x86_64"])
|
||||
parser.add_argument("--force-native", action="store_true", help="Treat as native wheel even if detection says pure")
|
||||
parser.add_argument("--force-repack", action="store_true", help="Force repack even for native wheels")
|
||||
args = parser.parse_args()
|
||||
|
||||
input_path = args.input
|
||||
if not os.path.isfile(input_path):
|
||||
print(f"Error: input file not found: {input_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
if not input_path.endswith(".whl"):
|
||||
print(f"Error: input must be a .whl file, got: {input_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
info = detect.detect_wheel_type(input_path)
|
||||
if "error" in info:
|
||||
print(f"Error: {info['error']}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
print(f"[pipeline] Input : {info['wheel']}")
|
||||
print(f"[pipeline] Pure Py : {info['pure_python']}")
|
||||
print(f"[pipeline] Detail : {info['detail']}")
|
||||
|
||||
output_dir = os.path.abspath(args.output_dir)
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
is_native = info["native"] or args.force_native
|
||||
|
||||
if not is_native:
|
||||
print("[pipeline] Pure Python wheel — repack with harmonyos tag")
|
||||
out = repack.repack_wheel(input_path, output_dir)
|
||||
print(f"[pipeline] Output : {out}")
|
||||
print("[pipeline] Done — wheel can be installed directly on HarmonyOS.")
|
||||
else:
|
||||
if args.force_repack:
|
||||
print("[pipeline] Pure wheel detected but force-repack requested.")
|
||||
out = repack.repack_wheel(input_path, output_dir)
|
||||
print(f"[pipeline] Repacked : {out}")
|
||||
else:
|
||||
print("[pipeline] Native wheel — needs cross-compilation.")
|
||||
out = rebuild.rebuild_native_wheel(
|
||||
input_path, output_dir,
|
||||
target_os=args.target_os, arch=args.arch,
|
||||
)
|
||||
print(f"[pipeline] Output : {out}")
|
||||
print("[pipeline] Done — native rebuild entry point triggered.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
43
pipeline/detect.py
Normal file
43
pipeline/detect.py
Normal file
@ -0,0 +1,43 @@
|
||||
import zipfile
|
||||
import os
|
||||
import re
|
||||
|
||||
PURE_TAGS = {"py2", "py3", "py2.py3", "none", "any"}
|
||||
|
||||
|
||||
def detect_wheel_type(wheel_path: str) -> dict:
|
||||
if not os.path.isfile(wheel_path):
|
||||
return {"error": f"File not found: {wheel_path}"}
|
||||
|
||||
if not wheel_path.endswith(".whl"):
|
||||
return {"error": "Not a .whl file"}
|
||||
|
||||
basename = os.path.basename(wheel_path)
|
||||
parts = basename.split("-")
|
||||
platform_tag = parts[-1].replace(".whl", "") if len(parts) >= 4 else "unknown"
|
||||
abi_tag = parts[-2] if len(parts) >= 4 else "unknown"
|
||||
|
||||
has_native = False
|
||||
native_detail = ""
|
||||
|
||||
if platform_tag not in PURE_TAGS or abi_tag != "none":
|
||||
has_native = True
|
||||
native_detail = f"platform={platform_tag}, abi={abi_tag}"
|
||||
|
||||
if not has_native:
|
||||
try:
|
||||
with zipfile.ZipFile(wheel_path) as zf:
|
||||
for name in zf.namelist():
|
||||
if name.endswith((".so", ".pyd", ".dll", ".dylib")):
|
||||
has_native = True
|
||||
native_detail = f"contains native extension: {name}"
|
||||
break
|
||||
except zipfile.BadZipFile:
|
||||
return {"error": "Corrupt or invalid zip file"}
|
||||
|
||||
return {
|
||||
"wheel": basename,
|
||||
"pure_python": not has_native,
|
||||
"native": has_native,
|
||||
"detail": native_detail or "tag=any/none, no native binaries found",
|
||||
}
|
||||
46
pipeline/rebuild.py
Normal file
46
pipeline/rebuild.py
Normal file
@ -0,0 +1,46 @@
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def rebuild_native_wheel(wheel_path: str, output_dir: str, target_os: str = "harmonyos", arch: str = "arm64") -> str:
|
||||
basename = os.path.basename(wheel_path)
|
||||
print(f"[rebuild] Native wheel rebuild triggered: {basename}")
|
||||
print(f"[rebuild] Target: {target_os}/{arch}")
|
||||
print(f"[rebuild] Output: {output_dir}")
|
||||
print()
|
||||
print(" Native extension rebuild for HarmonyOS requires:")
|
||||
print(" - HarmonyOS SDK / OHOS NDK")
|
||||
print(" - Cross-compilation toolchain for arm64")
|
||||
print(" - Python headers matching the target Python build")
|
||||
print(" - All C/Fortran source dependencies ported to HarmonyOS")
|
||||
print()
|
||||
print(" This entry point will be fleshed out in a future iteration.")
|
||||
print(" For now, the source wheel is copied as-is for manual inspection.")
|
||||
print()
|
||||
|
||||
output_path = os.path.join(output_dir, basename)
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
with open(wheel_path, "rb") as src, open(output_path, "wb") as dst:
|
||||
while True:
|
||||
buf = src.read(65536)
|
||||
if not buf:
|
||||
break
|
||||
dst.write(buf)
|
||||
|
||||
print(f"[rebuild] Copied original wheel to: {output_path}")
|
||||
|
||||
stub_path = output_path.replace(".whl", f"_{target_os}_{arch}_stub.txt")
|
||||
with open(stub_path, "w") as f:
|
||||
f.write(f"Native rebuild stub for {basename} ({target_os}/{arch})\n")
|
||||
f.write("Replace this file with the actual rebuilt harmonyos wheel.\n")
|
||||
|
||||
print(f"[rebuild] Stub marker: {stub_path}")
|
||||
return output_path
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python -m pipeline.rebuild <wheel.whl> [output_dir]")
|
||||
sys.exit(1)
|
||||
rebuild_native_wheel(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else "output")
|
||||
30
pipeline/repack.py
Normal file
30
pipeline/repack.py
Normal file
@ -0,0 +1,30 @@
|
||||
import zipfile
|
||||
import os
|
||||
import shutil
|
||||
|
||||
|
||||
def repack_wheel(wheel_path: str, output_dir: str) -> str:
|
||||
basename = os.path.basename(wheel_path)
|
||||
name_parts = basename.split("-")
|
||||
|
||||
if len(name_parts) >= 4:
|
||||
wheel_name = "-".join(name_parts[:-3])
|
||||
version = name_parts[-3]
|
||||
pure_tag = f"{name_parts[-2]}-{name_parts[-1]}"
|
||||
else:
|
||||
wheel_name = name_parts[0]
|
||||
version = "0.0.0"
|
||||
pure_tag = "none-any"
|
||||
|
||||
harmony_tag = f"{wheel_name}-{version}-py3-none-harmonyos_arm64.whl"
|
||||
output_path = os.path.join(output_dir, harmony_tag)
|
||||
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(wheel_path, "r") as zin:
|
||||
with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zout:
|
||||
for item in zin.infolist():
|
||||
data = zin.read(item.filename)
|
||||
zout.writestr(item, data)
|
||||
|
||||
return output_path
|
||||
9
wheel_pipeline.py
Executable file
9
wheel_pipeline.py
Executable file
@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import os
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from pipeline.cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user