Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ jobs:
- name: Prepare bundled Python seed
shell: pwsh
run: |
$installer = Join-Path $PWD "python-3.11.0-amd64.exe"
$target = Join-Path $PWD "python311"
$installer = Join-Path $env:RUNNER_TEMP "python-3.11.0-amd64.exe"
$target = Join-Path $env:RUNNER_TEMP "latexsnipper-python-seed"
Invoke-WebRequest `
-Uri "https://www.python.org/ftp/python/3.11.0/python-3.11.0-amd64.exe" `
-OutFile $installer
Expand All @@ -93,6 +93,7 @@ jobs:
"PrependPath=0",
"Shortcuts=0"
) `
-WindowStyle Hidden `
-Wait `
-PassThru
if ($process.ExitCode -ne 0) {
Expand All @@ -102,11 +103,13 @@ jobs:
throw "Bundled Python seed was not created: $target"
}
Remove-Item -LiteralPath $installer -Force
& (Join-Path $target "python.exe") -m pip install --upgrade pip wheel
& (Join-Path $target "python.exe") -m pip install --upgrade pip wheel setuptools packaging
if ($LASTEXITCODE -ne 0) { throw "Bundled Python tooling installation failed." }
"LATEXSNIPPER_BUNDLED_PYTHON=$target" >> $env:GITHUB_ENV

- name: Build unsigned installer
shell: pwsh
run: powershell -ExecutionPolicy Bypass -File scripts\build_github_release_installer.ps1 -PythonPath python
run: ./scripts/build_github_release_installer.ps1 -BundledPythonPath $env:LATEXSNIPPER_BUNDLED_PYTHON

- name: Upload unsigned installer artifact
id: upload-unsigned
Expand Down
14 changes: 3 additions & 11 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,9 @@ office_plugin/release/InstallerAssets/
*.tar
*.gz

# Embedded runtime (do not version)
tools/deps/python311/
tools/deps/python311-*/

# Local runtime components (do not version)
/python311/
python-*.exe

# Logs
*.log

# Local app state
tools/deps/.deps_state.json

# Local OCR/PDF regression outputs
test_pdf/outputs/**
!test_pdf/outputs/
Expand All @@ -54,3 +43,6 @@ user_manual/*
!user_manual/*.typ
!user_manual/*.md
!user_manual/*.png

# Local editor settings
.vscode/
2 changes: 1 addition & 1 deletion .pylintrc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[MASTER]
jobs=0
ignore-patterns=^\.?#,.*__pycache__.*,.*\.venv.*,.*python311.*,.*build.*
ignore-patterns=^\.?#,.*__pycache__.*,.*\.venv.*,.*build.*

[MESSAGES CONTROL]
disable=all
Expand Down
5 changes: 0 additions & 5 deletions .vscode/settings.json

This file was deleted.

6 changes: 1 addition & 5 deletions Inno/latexsnipper.iss
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@
#define MyAppPublisher "MathCraft"
#define MyAppURL "https://github.com/SakuraMathcraft/LaTeXSnipper"
#define MyAppExeName "LaTeXSnipper.exe"
#if GetEnv("LATEXSNIPPER_REPO_ROOT") != ""
#define MyRepoRoot GetEnv("LATEXSNIPPER_REPO_ROOT")
#else
#define MyRepoRoot "E:\LaTexSnipper"
#endif
#define MyRepoRoot AddBackslash(SourcePath) + ".."
#define MyBuildDir MyRepoRoot + "\dist\LaTeXSnipper"
#define MyOutputDir MyRepoRoot + "\dist\installer"
#define MyLicenseFile MyRepoRoot + "\LICENSE"
Expand Down
22 changes: 8 additions & 14 deletions LaTeXSnipper.spec
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ sys.path.insert(0, str(SRC))
from runtime.product_version import PRODUCT_VERSION, write_windows_version_info

APP_NAME = os.environ.get("LATEXSNIPPER_BUILD_NAME", "LaTeXSnipper")
BUNDLED_DEPS_DIR_ENV = os.environ.get("LATEXSNIPPER_BUNDLED_DEPS_DIR", "").strip()
VERSION_INFO_PATH = ROOT / "build" / "version_info.txt"
write_windows_version_info(VERSION_INFO_PATH, PRODUCT_VERSION)

Expand Down Expand Up @@ -337,21 +336,16 @@ def _prune_qt_webengine_payload(dist_root: Path):
_remove_optional_qt_plugins(plugins_dir, removable_plugins)


def _resolve_bundled_deps_root() -> Path:
if BUNDLED_DEPS_DIR_ENV:
return Path(BUNDLED_DEPS_DIR_ENV).expanduser()
return ROOT
# The release workflow prepares a clean seed outside the checkout.
bundled_python = os.environ.get("LATEXSNIPPER_BUNDLED_PYTHON", "").strip()
if not bundled_python:
raise RuntimeError("LATEXSNIPPER_BUNDLED_PYTHON must identify the CI Python seed")
bundled_python = Path(bundled_python).resolve()
if not (bundled_python / "python.exe").is_file():
raise RuntimeError(f"Bundled Python seed is missing python.exe: {bundled_python}")
extra_datas += _collect_tree_as_datas(bundled_python, "deps/python311")


# Bundle dependency runtime.
BUNDLED_DEPS_ROOT = _resolve_bundled_deps_root()
BUNDLED_PY311 = BUNDLED_DEPS_ROOT / "python311"
if BUNDLED_PY311.exists():
extra_datas += _collect_tree_as_datas(BUNDLED_PY311, "deps/python311")
print(f"[SPEC] include bundled python311: {BUNDLED_PY311}")
else:
print(f"[SPEC] bundled python311 not found, skip: {BUNDLED_PY311}")

a = Analysis(
[str(SRC / "main.py")],
pathex=[str(SRC), str(ROOT)],
Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

[English](readme.md) · 简体中文

<img width="960" alt="LaTeXSnipper v3.0.0:识别历史、公式编辑器与实时预览" src="docs/latexsnipper-3.0.0.png" />
<img width="960" alt="LaTeXSnipper:识别历史、公式编辑器与实时预览" src="docs/latexsnipper.png" />

</div>

Expand Down
14 changes: 7 additions & 7 deletions README_MATHCRAFT_OCR.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,9 @@ mathcraft warmup --profile mixed --provider auto
Recognize an image:

```powershell
mathcraft ocr "C:\path\to\page.png" --profile mixed --provider auto --output result.md
mathcraft ocr "C:\path\to\page.png" --profile mixed --provider auto --output-dir "D:\MathCraft\outputs"
mathcraft ocr "C:\path\to\formula.png" --profile formula --provider auto --json
mathcraft ocr "./examples/page.png" --profile mixed --provider auto --output result.md
mathcraft ocr "./examples/page.png" --profile mixed --provider auto --output-dir "./outputs"
mathcraft ocr "./examples/formula.png" --profile formula --provider auto --json
```

Run JSONL worker mode:
Expand All @@ -122,14 +122,14 @@ Linux: ${XDG_DATA_HOME:-~/.local/share}/LaTeXSnipper/MathCraft/models
or from a custom root:

```powershell
$env:MATHCRAFT_HOME="D:\MathCraft\models"
$env:MATHCRAFT_HOME="./models"
mathcraft doctor --provider auto
```

Persist the custom root for future PowerShell sessions:

```powershell
setx MATHCRAFT_HOME "D:\MathCraft\models"
setx MATHCRAFT_HOME "./models"
```

Restore the default user cache root:
Expand Down Expand Up @@ -198,15 +198,15 @@ The actual provider is available on recognition results through the `provider` f
Run tests from the repository root:

```powershell
cd E:\LaTexSnipper
# Run from the repository root
python .\test\test_mathcraft_ocr.py
python .\test\test_mathcraft_document_engine.py
```

Build package artifacts:

```powershell
cd E:\LaTexSnipper
# Run from the repository root
python -m build --no-isolation --outdir .\release_assets\mathcraft-ocr-package\dist .
```

Expand Down
68 changes: 36 additions & 32 deletions benchmarks/mathcraft_ocr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Third-party systems may appear only in related work or public-result context, wi
Keep external datasets and large run artifacts outside the repository:

```text
E:\MathCraftBenchData
./benchmark-data
```

Manifest files live under:
Expand All @@ -45,32 +45,32 @@ images, and dataset-provided normalized LaTeX labels as ground truth.
Run MathCraft GPU:

```powershell
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_mathwriting_test.ps1
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_mathwriting_test.ps1 -DataRoot ./benchmark-data
```

The runner creates:

```text
E:\MathCraftBenchData\manifests\mathwriting_test.jsonl
E:\MathCraftBenchData\raw\mathwriting\test_images
E:\MathCraftBenchData\runs\mathwriting_test_gpu\mathwriting_test_gpu_full.jsonl
E:\MathCraftBenchData\runs\mathwriting_test_gpu\cdm_input\mathwriting_test_full_cdm.json
./benchmark-data\manifests\mathwriting_test.jsonl
./benchmark-data\raw\mathwriting\test_images
./benchmark-data\runs\mathwriting_test_gpu\mathwriting_test_gpu_full.jsonl
./benchmark-data\runs\mathwriting_test_gpu\cdm_input\mathwriting_test_full_cdm.json
```

It also writes exact, BLEU-4, normalized edit distance, and latency summaries
after the full JSONL is combined. Render success is computed separately:

```powershell
python benchmarks\mathcraft_ocr\reports\render_formula_success.py `
--results E:\MathCraftBenchData\runs\mathwriting_test_gpu\mathwriting_test_gpu_full.jsonl `
--manifest E:\MathCraftBenchData\manifests\mathwriting_test.jsonl `
--output-dir E:\MathCraftBenchData\runs\mathwriting_test_gpu\render_success
--results ./benchmark-data\runs\mathwriting_test_gpu\mathwriting_test_gpu_full.jsonl `
--manifest ./benchmark-data\manifests\mathwriting_test.jsonl `
--output-dir ./benchmark-data\runs\mathwriting_test_gpu\render_success
```

Run official CDM on MathWriting predictions:

```powershell
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_mathwriting_official_cdm.ps1
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_mathwriting_official_cdm.ps1 -DataRoot ./benchmark-data
```

Baseline comparisons must be restricted to the same split, normalized label, and
Expand All @@ -79,7 +79,7 @@ offline rasterization protocol.
## UniMER-Test

```powershell
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_unimer_test.ps1
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_unimer_test.ps1 -DataRoot ./benchmark-data
```

The script downloads UniMER-Test from Hugging Face when needed, creates the manifest,
Expand All @@ -90,18 +90,18 @@ Analyze text metrics:

```powershell
python benchmarks\mathcraft_ocr\reports\analyze_unimer_results.py `
--results E:\MathCraftBenchData\runs\unimer_test_gpu\unimer_test_gpu_full.jsonl `
--manifest E:\MathCraftBenchData\manifests\unimer_test_full.jsonl `
--output-dir E:\MathCraftBenchData\runs\unimer_test_gpu
--results ./benchmark-data\runs\unimer_test_gpu\unimer_test_gpu_full.jsonl `
--manifest ./benchmark-data\manifests\unimer_test_full.jsonl `
--output-dir ./benchmark-data\runs\unimer_test_gpu
```

Analyze render-consistency fallback samples:

```powershell
python benchmarks\mathcraft_ocr\reports\render_unimer_samples.py `
--results E:\MathCraftBenchData\runs\unimer_test_gpu\unimer_test_gpu_full.jsonl `
--manifest E:\MathCraftBenchData\manifests\unimer_test_full.jsonl `
--output-dir E:\MathCraftBenchData\runs\unimer_test_gpu\render_exact_underestimated `
--results ./benchmark-data\runs\unimer_test_gpu\unimer_test_gpu_full.jsonl `
--manifest ./benchmark-data\manifests\unimer_test_full.jsonl `
--output-dir ./benchmark-data\runs\unimer_test_gpu\render_exact_underestimated `
--per-subset 20 `
--min-similarity 0.95 `
--keep-images
Expand All @@ -111,9 +111,9 @@ Prepare official CDM batch JSON input:

```powershell
python benchmarks\mathcraft_ocr\reports\prepare_cdm_input.py `
--results E:\MathCraftBenchData\runs\unimer_test_gpu\unimer_test_gpu_full.jsonl `
--manifest E:\MathCraftBenchData\manifests\unimer_test_full.jsonl `
--output E:\MathCraftBenchData\runs\unimer_test_gpu\cdm_input\unimer_test_full_cdm.json `
--results ./benchmark-data\runs\unimer_test_gpu\unimer_test_gpu_full.jsonl `
--manifest ./benchmark-data\manifests\unimer_test_full.jsonl `
--output ./benchmark-data\runs\unimer_test_gpu\cdm_input\unimer_test_full_cdm.json `
--subset all
```

Expand All @@ -123,21 +123,21 @@ The generated JSON follows the official CDM batch fields `img_id`, `gt`, and
Run the official UniMERNet CDM runtime with resumable shards:

```powershell
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_official_cdm.ps1
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_official_cdm.ps1 -DataRoot ./benchmark-data
```

The CDM runner uses the official `evaluation.py` from the local UniMERNet clone
under `E:\MathCraftBenchData\sources\UniMERNet_official\cdm`. It intentionally
under `./benchmark-data\sources\UniMERNet_official\cdm`. It intentionally
uses the system Python configured for that official runtime; MathCraft OCR
inference remains on `tools\deps\python311\python.exe`. The default official
inference remains on the developer-selected Python interpreter. The default official
CDM pool count is 8 on the local Windows evaluation host.

For a quick validation run:

```powershell
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_official_cdm.ps1 `
-InputPath E:\MathCraftBenchData\runs\unimer_test_gpu\cdm_input\unimer_test_all_20_cdm.json `
-OutputDir E:\MathCraftBenchData\runs\cdm_official_unimer_20_runner `
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_official_cdm.ps1 -DataRoot ./benchmark-data `
-InputPath ./benchmark-data\runs\unimer_test_gpu\cdm_input\unimer_test_all_20_cdm.json `
-OutputDir ./benchmark-data\runs\cdm_official_unimer_20_runner `
-ShardSize 10 `
-Limit 20 `
-Pools 8
Expand All @@ -155,7 +155,7 @@ render skips are not confused with interrupted runs.
The full official CDM UniMER-Test run is complete:

```text
Run: E:\MathCraftBenchData\runs\cdm_official_unimer_full
Run: ./benchmark-data\runs\cdm_official_unimer_full
Mean CDM: 0.929
ExpRate: 0.648
Expected: 23757
Expand All @@ -170,7 +170,7 @@ benchmarks/mathcraft_ocr/results/unimer_test_gpu
```

Large JSONL files, downloaded data, and rendered PNG pairs remain under
`E:\MathCraftBenchData`.
`./benchmark-data`.

## OpenStax Page Images

Expand All @@ -181,7 +181,7 @@ for exact-match formula accuracy because no page-level ground truth is included.
Run the default systematic mixed-page benchmark:

```powershell
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_openstax_pages.ps1
powershell -ExecutionPolicy Bypass -File benchmarks\mathcraft_ocr\run_openstax_pages.ps1 -DataRoot ./benchmark-data
```

The runner creates a manifest, renders selected OpenStax PDF pages, runs mixed
Expand All @@ -205,10 +205,14 @@ Generate OpenStax block-overlay figure assets from the formal run:

```powershell
python benchmarks\mathcraft_ocr\visualization\make_openstax_block_gallery.py `
--results E:\MathCraftBenchData\runs\openstax_mixed_gpu_144dpi\openstax_mixed_gpu_144dpi_full.jsonl `
--output-dir E:\MathCraftBenchData\runs\openstax_mixed_gpu_144dpi\figures\block_gallery `
--results ./benchmark-data\runs\openstax_mixed_gpu_144dpi\openstax_mixed_gpu_144dpi_full.jsonl `
--output-dir ./benchmark-data\runs\openstax_mixed_gpu_144dpi\figures\block_gallery `
--limit 6
```

Generated OpenStax page images and overlays remain under `E:\MathCraftBenchData`
Generated OpenStax page images and overlays remain under `./benchmark-data`
because they are derived from licensed OpenStax PDF content.

## Portable benchmark environment

Run examples from the repository root. `./benchmark-data` is an example external-data directory; use `-DataRoot` to choose its location. The PowerShell runners resolve the checkout from the script location and use the active `python`; inference runners accept `-PythonPath`, and CDM runners accept `-Python`. Install CDM tools on PATH or pass `-PathPrepend` with your own tool directories. No drive letter or developer environment is required.
Loading