feat: initial EcoArk HarmonyOS wheel pipeline skeleton
- wheel_pipeline.py: CLI entry point for repack/rebuild flow - pipeline/detect.py: detect pure vs native extension wheels - pipeline/repack.py: repack pure-Python wheels with harmonyos_arm64 tag - pipeline/rebuild.py: native wheel rebuild entry (stub for OHOS NDK) - pipeline/cli.py: argument parsing and orchestration - Makefile: convenience targets (repack, rebuild, clean) - README.md: documentation with usage and limitations - examples/gen_test_wheels.py: test wheel generator - output/: output directory for generated wheels
This commit is contained in:
32
Makefile
Normal file
32
Makefile
Normal file
@ -0,0 +1,32 @@
|
|||||||
|
SHELL := /bin/bash
|
||||||
|
|
||||||
|
INPUT ?= examples/sample.whl
|
||||||
|
OUTPUT ?= output
|
||||||
|
|
||||||
|
.PHONY: help repack rebuild native clean
|
||||||
|
|
||||||
|
help:
|
||||||
|
@echo "EcoArk HarmonyOS Wheel Pipeline"
|
||||||
|
@echo ""
|
||||||
|
@echo "Usage:"
|
||||||
|
@echo " make repack INPUT=path/to/pkg.whl — Repack a pure-Python wheel for HarmonyOS"
|
||||||
|
@echo " make rebuild INPUT=path/to/pkg.whl — Trigger native rebuild entry (stub)"
|
||||||
|
@echo " make clean — Remove output/"
|
||||||
|
@echo ""
|
||||||
|
@echo "Examples:"
|
||||||
|
@echo " make repack INPUT=some_package.whl"
|
||||||
|
@echo " make rebuild INPUT=some_native.whl"
|
||||||
|
|
||||||
|
repack:
|
||||||
|
python3 wheel_pipeline.py "$(INPUT)" --output-dir "$(OUTPUT)" --force-repack
|
||||||
|
|
||||||
|
rebuild:
|
||||||
|
python3 wheel_pipeline.py "$(INPUT)" --output-dir "$(OUTPUT)" --target-os harmonyos --arch arm64
|
||||||
|
|
||||||
|
native:
|
||||||
|
python3 wheel_pipeline.py "$(INPUT)" --output-dir "$(OUTPUT)" --target-os harmonyos --arch arm64
|
||||||
|
|
||||||
|
clean:
|
||||||
|
rm -rf "$(OUTPUT)"
|
||||||
|
mkdir -p "$(OUTPUT)"
|
||||||
|
touch "$(OUTPUT)/.gitkeep"
|
||||||
83
README.md
Normal file
83
README.md
Normal file
@ -0,0 +1,83 @@
|
|||||||
|
# EcoArk HarmonyOS Wheel Pipeline
|
||||||
|
|
||||||
|
为 **EcoArk / HarmonyOS Python 环境** 准备的 wheel 重打包与重编译流水线。
|
||||||
|
|
||||||
|
## 用途
|
||||||
|
|
||||||
|
- 接收一个 `.whl` 文件作为输入
|
||||||
|
- 自动检测是 **纯 Python wheel** 还是 **含 native 扩展的 wheel**
|
||||||
|
- 纯 Python wheel → 直接重打包为 `harmonyos_arm64` 兼容标签,安装即用
|
||||||
|
- Native wheel → 预留 HarmonyOS/arm64 交叉编译入口(当前为骨架,可扩展)
|
||||||
|
|
||||||
|
## 目录结构
|
||||||
|
|
||||||
|
```
|
||||||
|
ecoarkpywhl/
|
||||||
|
├── wheel_pipeline.py # CLI 入口(最简调用)
|
||||||
|
├── pipeline/
|
||||||
|
│ ├── cli.py # 参数解析 & 主流程
|
||||||
|
│ ├── detect.py # 检测 wheel 类型(纯 Python / native)
|
||||||
|
│ ├── repack.py # 纯 Python wheel 重打包
|
||||||
|
│ └── rebuild.py # Native wheel 重编译入口(骨架)
|
||||||
|
├── Makefile # 常用命令
|
||||||
|
├── output/ # 输出目录(生成文件放这里)
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## 快速开始
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 纯 Python wheel → 重打包
|
||||||
|
python3 wheel_pipeline.py some_pure_package.whl
|
||||||
|
|
||||||
|
# 指定输出目录
|
||||||
|
python3 wheel_pipeline.py some_pure_package.whl --output-dir /tmp/wheels
|
||||||
|
|
||||||
|
# Native wheel → 触发重编译入口(当前只复制 + 写 stub)
|
||||||
|
python3 wheel_pipeline.py some_native_package.whl
|
||||||
|
```
|
||||||
|
|
||||||
|
或通过 Makefile:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make repack INPUT=some_package.whl
|
||||||
|
make rebuild INPUT=some_native_package.whl
|
||||||
|
```
|
||||||
|
|
||||||
|
## 限制
|
||||||
|
|
||||||
|
| 场景 | 状态 |
|
||||||
|
|------|------|
|
||||||
|
| 纯 Python wheel → 重打包安装 | ✅ 支持 |
|
||||||
|
| Pure Python → 重新打 tag | ✅ 支持 |
|
||||||
|
| Native wheel → HarmonyOS 重编译 | 🚧 预留入口,需接入 OHOS NDK 工具链 |
|
||||||
|
| 依赖复杂的 native 包(numpy/scipy 等) | 🚧 需要对应依赖也移植到 HarmonyOS |
|
||||||
|
|
||||||
|
### 纯 Python wheel
|
||||||
|
|
||||||
|
纯 Python wheel(文件名中包含 `py3-none-any` 等标签,且不含 `.so`/`.pyd` 文件)可以直接重打包为 `harmonyos_arm64` 标签,在 HarmonyOS Python 环境中安装运行。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 wheel_pipeline.py mypackage.whl -o output/
|
||||||
|
# → output/mypackage-1.0.0-py3-none-harmonyos_arm64.whl
|
||||||
|
```
|
||||||
|
|
||||||
|
### Native wheel
|
||||||
|
|
||||||
|
含 C/C++/Rust 等 native 扩展的 wheel 需要:
|
||||||
|
|
||||||
|
1. HarmonyOS SDK / OHOS NDK
|
||||||
|
2. 对应架构(arm64)的交叉编译工具链
|
||||||
|
3. Python 头文件与目标 Python 版本匹配
|
||||||
|
4. 所有 C 依赖也已移植到 HarmonyOS
|
||||||
|
|
||||||
|
当前版本仅输出 stub,待后续完善。
|
||||||
|
|
||||||
|
## 依赖
|
||||||
|
|
||||||
|
- Python ≥ 3.8(标准库,无需额外安装包)
|
||||||
|
- 对 native 包:HarmonyOS 工具链(外部,不在本仓库内)
|
||||||
|
|
||||||
|
## 许可证
|
||||||
|
|
||||||
|
本项目许可证见仓库根目录。
|
||||||
80
examples/gen_test_wheels.py
Normal file
80
examples/gen_test_wheels.py
Normal file
@ -0,0 +1,80 @@
|
|||||||
|
import zipfile
|
||||||
|
import os
|
||||||
|
import json
|
||||||
|
|
||||||
|
EXAMPLES_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
|
||||||
|
|
||||||
|
def make_pure_wheel(name, version, out_dir):
|
||||||
|
dist_info = f"{name}-{version}.dist-info"
|
||||||
|
os.makedirs(f"/tmp/{dist_info}", exist_ok=True)
|
||||||
|
|
||||||
|
metadata = f"""Metadata-Version: 2.1
|
||||||
|
Name: {name}
|
||||||
|
Version: {version}
|
||||||
|
Summary: Test pure-Python wheel
|
||||||
|
"""
|
||||||
|
|
||||||
|
with open(f"/tmp/{dist_info}/METADATA", "w") as f:
|
||||||
|
f.write(metadata)
|
||||||
|
with open(f"/tmp/{dist_info}/WHEEL", "w") as f:
|
||||||
|
f.write("Wheel-Version: 1.0\nGenerator: test\nRoot-Is-Purelib: true\nTag: py3-none-any\n")
|
||||||
|
with open(f"/tmp/{dist_info}/RECORD", "w") as f:
|
||||||
|
f.write("")
|
||||||
|
|
||||||
|
wheel_name = f"{name}-{version}-py3-none-any.whl"
|
||||||
|
wheel_path = os.path.join(out_dir, wheel_name)
|
||||||
|
|
||||||
|
with zipfile.ZipFile(wheel_path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||||
|
zf.write(f"/tmp/{dist_info}/METADATA", f"{dist_info}/METADATA")
|
||||||
|
zf.write(f"/tmp/{dist_info}/WHEEL", f"{dist_info}/WHEEL")
|
||||||
|
zf.write(f"/tmp/{dist_info}/RECORD", f"{dist_info}/RECORD")
|
||||||
|
pkg_dir = f"{name}"
|
||||||
|
zf.writestr(f"{pkg_dir}/__init__.py", "def hello(): return 'hello'\n")
|
||||||
|
|
||||||
|
for f in os.listdir(f"/tmp/{dist_info}"):
|
||||||
|
os.remove(f"/tmp/{dist_info}/{f}")
|
||||||
|
os.rmdir(f"/tmp/{dist_info}")
|
||||||
|
|
||||||
|
return wheel_path
|
||||||
|
|
||||||
|
|
||||||
|
def make_native_wheel(name, version, out_dir):
|
||||||
|
dist_info = f"{name}-{version}.dist-info"
|
||||||
|
os.makedirs(f"/tmp/{dist_info}", exist_ok=True)
|
||||||
|
|
||||||
|
metadata = f"""Metadata-Version: 2.1
|
||||||
|
Name: {name}
|
||||||
|
Version: {version}
|
||||||
|
Summary: Test native wheel
|
||||||
|
"""
|
||||||
|
|
||||||
|
with open(f"/tmp/{dist_info}/METADATA", "w") as f:
|
||||||
|
f.write(metadata)
|
||||||
|
with open(f"/tmp/{dist_info}/WHEEL", "w") as f:
|
||||||
|
f.write("Wheel-Version: 1.0\nGenerator: test\nRoot-Is-Purelib: false\nTag: cp38-cp38-manylinux_2_17_x86_64\n")
|
||||||
|
with open(f"/tmp/{dist_info}/RECORD", "w") as f:
|
||||||
|
f.write("")
|
||||||
|
|
||||||
|
wheel_name = f"{name}-{version}-cp38-cp38-manylinux_2_17_x86_64.whl"
|
||||||
|
wheel_path = os.path.join(out_dir, wheel_name)
|
||||||
|
|
||||||
|
with zipfile.ZipFile(wheel_path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||||
|
zf.write(f"/tmp/{dist_info}/METADATA", f"{dist_info}/METADATA")
|
||||||
|
zf.write(f"/tmp/{dist_info}/WHEEL", f"{dist_info}/WHEEL")
|
||||||
|
zf.write(f"/tmp/{dist_info}/RECORD", f"{dist_info}/RECORD")
|
||||||
|
zf.writestr(f"{name}/__init__.py", "def hello(): return 'hello'\n")
|
||||||
|
zf.writestr(f"{name}/_native.cpython-38-x86_64-linux-gnu.so", b"\x7fELF...fake")
|
||||||
|
|
||||||
|
for f in os.listdir(f"/tmp/{dist_info}"):
|
||||||
|
os.remove(f"/tmp/{dist_info}/{f}")
|
||||||
|
os.rmdir(f"/tmp/{dist_info}")
|
||||||
|
|
||||||
|
return wheel_path
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
out = EXAMPLES_DIR
|
||||||
|
pure = make_pure_wheel("ecoark_demo", "0.1.0", out)
|
||||||
|
native = make_native_wheel("ecoark_native_demo", "0.1.0", out)
|
||||||
|
print(json.dumps({"pure": pure, "native": native}, indent=2))
|
||||||
0
output/.gitkeep
Normal file
0
output/.gitkeep
Normal file
0
pipeline/__init__.py
Normal file
0
pipeline/__init__.py
Normal file
BIN
pipeline/__pycache__/__init__.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/__init__.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/cli.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/cli.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/detect.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/detect.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/rebuild.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/rebuild.cpython-312.pyc
Normal file
Binary file not shown.
BIN
pipeline/__pycache__/repack.cpython-312.pyc
Normal file
BIN
pipeline/__pycache__/repack.cpython-312.pyc
Normal file
Binary file not shown.
63
pipeline/cli.py
Normal file
63
pipeline/cli.py
Normal file
@ -0,0 +1,63 @@
|
|||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from . import detect, repack, rebuild
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="EcoArk HarmonyOS Wheel Pipeline — repack / rebuild Python wheels for HarmonyOS/arm64"
|
||||||
|
)
|
||||||
|
parser.add_argument("input", help="Path to a .whl file")
|
||||||
|
parser.add_argument("--output-dir", "-o", default="output", help="Output directory (default: ./output)")
|
||||||
|
parser.add_argument("--target-os", default="harmonyos", choices=["harmonyos", "linux", "ohos"])
|
||||||
|
parser.add_argument("--arch", default="arm64", choices=["arm64", "aarch64", "x86_64"])
|
||||||
|
parser.add_argument("--force-native", action="store_true", help="Treat as native wheel even if detection says pure")
|
||||||
|
parser.add_argument("--force-repack", action="store_true", help="Force repack even for native wheels")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
input_path = args.input
|
||||||
|
if not os.path.isfile(input_path):
|
||||||
|
print(f"Error: input file not found: {input_path}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
if not input_path.endswith(".whl"):
|
||||||
|
print(f"Error: input must be a .whl file, got: {input_path}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
info = detect.detect_wheel_type(input_path)
|
||||||
|
if "error" in info:
|
||||||
|
print(f"Error: {info['error']}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
print(f"[pipeline] Input : {info['wheel']}")
|
||||||
|
print(f"[pipeline] Pure Py : {info['pure_python']}")
|
||||||
|
print(f"[pipeline] Detail : {info['detail']}")
|
||||||
|
|
||||||
|
output_dir = os.path.abspath(args.output_dir)
|
||||||
|
os.makedirs(output_dir, exist_ok=True)
|
||||||
|
|
||||||
|
is_native = info["native"] or args.force_native
|
||||||
|
|
||||||
|
if not is_native:
|
||||||
|
print("[pipeline] Pure Python wheel — repack with harmonyos tag")
|
||||||
|
out = repack.repack_wheel(input_path, output_dir)
|
||||||
|
print(f"[pipeline] Output : {out}")
|
||||||
|
print("[pipeline] Done — wheel can be installed directly on HarmonyOS.")
|
||||||
|
else:
|
||||||
|
if args.force_repack:
|
||||||
|
print("[pipeline] Pure wheel detected but force-repack requested.")
|
||||||
|
out = repack.repack_wheel(input_path, output_dir)
|
||||||
|
print(f"[pipeline] Repacked : {out}")
|
||||||
|
else:
|
||||||
|
print("[pipeline] Native wheel — needs cross-compilation.")
|
||||||
|
out = rebuild.rebuild_native_wheel(
|
||||||
|
input_path, output_dir,
|
||||||
|
target_os=args.target_os, arch=args.arch,
|
||||||
|
)
|
||||||
|
print(f"[pipeline] Output : {out}")
|
||||||
|
print("[pipeline] Done — native rebuild entry point triggered.")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
43
pipeline/detect.py
Normal file
43
pipeline/detect.py
Normal file
@ -0,0 +1,43 @@
|
|||||||
|
import zipfile
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
|
||||||
|
PURE_TAGS = {"py2", "py3", "py2.py3", "none", "any"}
|
||||||
|
|
||||||
|
|
||||||
|
def detect_wheel_type(wheel_path: str) -> dict:
|
||||||
|
if not os.path.isfile(wheel_path):
|
||||||
|
return {"error": f"File not found: {wheel_path}"}
|
||||||
|
|
||||||
|
if not wheel_path.endswith(".whl"):
|
||||||
|
return {"error": "Not a .whl file"}
|
||||||
|
|
||||||
|
basename = os.path.basename(wheel_path)
|
||||||
|
parts = basename.split("-")
|
||||||
|
platform_tag = parts[-1].replace(".whl", "") if len(parts) >= 4 else "unknown"
|
||||||
|
abi_tag = parts[-2] if len(parts) >= 4 else "unknown"
|
||||||
|
|
||||||
|
has_native = False
|
||||||
|
native_detail = ""
|
||||||
|
|
||||||
|
if platform_tag not in PURE_TAGS or abi_tag != "none":
|
||||||
|
has_native = True
|
||||||
|
native_detail = f"platform={platform_tag}, abi={abi_tag}"
|
||||||
|
|
||||||
|
if not has_native:
|
||||||
|
try:
|
||||||
|
with zipfile.ZipFile(wheel_path) as zf:
|
||||||
|
for name in zf.namelist():
|
||||||
|
if name.endswith((".so", ".pyd", ".dll", ".dylib")):
|
||||||
|
has_native = True
|
||||||
|
native_detail = f"contains native extension: {name}"
|
||||||
|
break
|
||||||
|
except zipfile.BadZipFile:
|
||||||
|
return {"error": "Corrupt or invalid zip file"}
|
||||||
|
|
||||||
|
return {
|
||||||
|
"wheel": basename,
|
||||||
|
"pure_python": not has_native,
|
||||||
|
"native": has_native,
|
||||||
|
"detail": native_detail or "tag=any/none, no native binaries found",
|
||||||
|
}
|
||||||
46
pipeline/rebuild.py
Normal file
46
pipeline/rebuild.py
Normal file
@ -0,0 +1,46 @@
|
|||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
def rebuild_native_wheel(wheel_path: str, output_dir: str, target_os: str = "harmonyos", arch: str = "arm64") -> str:
|
||||||
|
basename = os.path.basename(wheel_path)
|
||||||
|
print(f"[rebuild] Native wheel rebuild triggered: {basename}")
|
||||||
|
print(f"[rebuild] Target: {target_os}/{arch}")
|
||||||
|
print(f"[rebuild] Output: {output_dir}")
|
||||||
|
print()
|
||||||
|
print(" Native extension rebuild for HarmonyOS requires:")
|
||||||
|
print(" - HarmonyOS SDK / OHOS NDK")
|
||||||
|
print(" - Cross-compilation toolchain for arm64")
|
||||||
|
print(" - Python headers matching the target Python build")
|
||||||
|
print(" - All C/Fortran source dependencies ported to HarmonyOS")
|
||||||
|
print()
|
||||||
|
print(" This entry point will be fleshed out in a future iteration.")
|
||||||
|
print(" For now, the source wheel is copied as-is for manual inspection.")
|
||||||
|
print()
|
||||||
|
|
||||||
|
output_path = os.path.join(output_dir, basename)
|
||||||
|
os.makedirs(output_dir, exist_ok=True)
|
||||||
|
|
||||||
|
with open(wheel_path, "rb") as src, open(output_path, "wb") as dst:
|
||||||
|
while True:
|
||||||
|
buf = src.read(65536)
|
||||||
|
if not buf:
|
||||||
|
break
|
||||||
|
dst.write(buf)
|
||||||
|
|
||||||
|
print(f"[rebuild] Copied original wheel to: {output_path}")
|
||||||
|
|
||||||
|
stub_path = output_path.replace(".whl", f"_{target_os}_{arch}_stub.txt")
|
||||||
|
with open(stub_path, "w") as f:
|
||||||
|
f.write(f"Native rebuild stub for {basename} ({target_os}/{arch})\n")
|
||||||
|
f.write("Replace this file with the actual rebuilt harmonyos wheel.\n")
|
||||||
|
|
||||||
|
print(f"[rebuild] Stub marker: {stub_path}")
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
if len(sys.argv) < 2:
|
||||||
|
print("Usage: python -m pipeline.rebuild <wheel.whl> [output_dir]")
|
||||||
|
sys.exit(1)
|
||||||
|
rebuild_native_wheel(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else "output")
|
||||||
30
pipeline/repack.py
Normal file
30
pipeline/repack.py
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
import zipfile
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
|
||||||
|
def repack_wheel(wheel_path: str, output_dir: str) -> str:
|
||||||
|
basename = os.path.basename(wheel_path)
|
||||||
|
name_parts = basename.split("-")
|
||||||
|
|
||||||
|
if len(name_parts) >= 4:
|
||||||
|
wheel_name = "-".join(name_parts[:-3])
|
||||||
|
version = name_parts[-3]
|
||||||
|
pure_tag = f"{name_parts[-2]}-{name_parts[-1]}"
|
||||||
|
else:
|
||||||
|
wheel_name = name_parts[0]
|
||||||
|
version = "0.0.0"
|
||||||
|
pure_tag = "none-any"
|
||||||
|
|
||||||
|
harmony_tag = f"{wheel_name}-{version}-py3-none-harmonyos_arm64.whl"
|
||||||
|
output_path = os.path.join(output_dir, harmony_tag)
|
||||||
|
|
||||||
|
os.makedirs(output_dir, exist_ok=True)
|
||||||
|
|
||||||
|
with zipfile.ZipFile(wheel_path, "r") as zin:
|
||||||
|
with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zout:
|
||||||
|
for item in zin.infolist():
|
||||||
|
data = zin.read(item.filename)
|
||||||
|
zout.writestr(item, data)
|
||||||
|
|
||||||
|
return output_path
|
||||||
9
wheel_pipeline.py
Executable file
9
wheel_pipeline.py
Executable file
@ -0,0 +1,9 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
from pipeline.cli import main
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user