diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/README.md b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/README.md new file mode 100644 index 000000000..cecdb3a27 --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/README.md @@ -0,0 +1,45 @@ +# PaddlePaddle__Paddle-60417 + +This directory converts Paddle PR #60417 into a SWE-Paddle community task candidate. + +## Source + +| Field | Value | +| --- | --- | +| Repo | `PaddlePaddle/Paddle` | +| PR | [60417](https://github.com/PaddlePaddle/Paddle/pull/60417) | +| PR title | `[auto config] Resume from history csv file` | +| Base commit | `e4b39bb56a4e55213383e96daf262f4f72c1811d` | +| Merged at | `2023-12-29T07:40:45Z` | +| Task type | `feature_enhancement` | +| Resource | CPU | + +## Summary + +Allow distributed auto-tuner runs to reuse completed configurations from a history CSV and continue with only the unfinished work. + +## Why This Is A Good SWE-Paddle Candidate + +- Interrupted tuning runs are a practical distributed-training problem, and the wasted work is directly observable. +- The change coordinates CSV value restoration, configuration matching, recorder updates, and launch control rather than adding an isolated special case. +- The source PR contains no test changes, so the candidate adds focused behavior tests that execute the checkout's real `AutoTuner` and `launch()` control flow. +- The tests prove that a resumed task does not start a training controller while preserving the existing search path. +- Verification is deterministic on CPU and requires no Paddle wheel overlay, GPU, network, training data, or subprocess timing. + +## Files + +- `proposal.md`: candidate proposal for maintainer triage. +- `instruction.md`: self-contained problem statement for the coding agent. +- `solution/code.patch`: exact production patch from the merged commit. +- `tests/test.patch`: focused behavior tests for history resume and launch reuse. +- `tests/test.sh`: minimal target test command. +- `environment/README.md`: environment notes for reproduction. +- `README.md`: task overview and verification entrypoint. + +## Verification + +```bash +bash tests/test.sh +``` + +Expected behavior: applying `tests/test.patch` to `base_commit` should preserve the existing search case but fail the history-resume cases; applying both `tests/test.patch` and `solution/code.patch` should pass all target tests. diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/environment/README.md b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/environment/README.md new file mode 100644 index 000000000..bb4996c47 --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/environment/README.md @@ -0,0 +1,27 @@ +# Environment Notes + +This candidate is part of the SWE-Paddle community task set. + +## Expected Environment + +- Repository: `PaddlePaddle/Paddle` +- Base commit: `e4b39bb56a4e55213383e96daf262f4f72c1811d` +- Resource: CPU +- GPU required: no +- Build path: no Paddle source build or installed-wheel overlay is required; the tests execute the checkout Python modules with controlled dependencies. + +## Run Order + +1. Check out `PaddlePaddle/Paddle` at the base commit. +2. Apply `tests/test.patch`. +3. Run `bash tests/test.sh`; the target behavior should fail before the fix. +4. Apply `solution/code.patch`. +5. Run `bash tests/test.sh` again; the target behavior should pass after the gold patch. + +## Minimal Test Command + +```bash +bash tests/test.sh +``` + +The verifier is responsible for deriving stable F2P and P2P node IDs from repeated runs. diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/instruction.md b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/instruction.md new file mode 100644 index 000000000..01df4583a --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/instruction.md @@ -0,0 +1,23 @@ +# auto-tuner 中断后无法从历史记录继续运行 + +## 详细描述 + +使用分布式 auto-tuner 进行调优时,任务会因为机器重启、运行超时或人工停止而中断。重新启动 auto-tuner 后,即使已经保留了上一次运行生成的历史 CSV,程序仍会从第一个配置重新开始,导致已经完成的训练任务被重复执行。 + +应该让 auto-tuner 支持从指定的历史 CSV 继续调优:对于历史记录中已经完成的配置,直接复用原有结果;对于尚未完成的配置,继续正常运行。这样可以减少重复训练,缩短中断后恢复调优所需的时间。 + +当历史文件不存在、内容为空或没有匹配到当前配置时,auto-tuner 按照原有流程正常运行。 + +## 验收说明 + +* 可以读取指定的历史 CSV,并正确恢复其中已经完成的配置和指标。 +* 当前配置在历史记录中已经完成时,直接复用原有结果,不再重复启动训练。 +* 当前配置没有历史记录时,继续执行正常的调优任务。 +* 使用历史文件恢复时应保留原始文件,避免覆盖已有记录。 +* 未配置历史文件或文件不存在时,现有 auto-tuner 行为保持不变。 + +## 技术要求 + +* 熟悉 Python 文件读写及 CSV 数据处理。 +* 熟悉 PaddlePaddle 分布式 launch 流程。 +* 了解 auto-tuner 的配置搜索与历史记录机制。 diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/proposal.md b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/proposal.md new file mode 100644 index 000000000..7d5e51952 --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/proposal.md @@ -0,0 +1,56 @@ +# Task Proposal: PaddlePaddle__Paddle-60417 + +## 1. 来源信息 + +- Instance ID:`PaddlePaddle__Paddle-60417` +- PR 链接:https://github.com/PaddlePaddle/Paddle/pull/60417 +- PR 标题:`[auto config] Resume from history csv file` +- `base_commit`:`e4b39bb56a4e55213383e96daf262f4f72c1811d` +- merged 时间:`2023-12-29T07:40:45Z` +- 你的身份:熟悉该模块的 contributor +- 后续联系人:TBD + +## 2. 问题一句话 + +auto-tuner 重新启动后无法复用历史 CSV 中已经完成的配置,会重复运行全部调优任务。 + +## 3. 为什么适合作为 SWE-Paddle 样本 + +- **真实性**:长时间分布式调优可能被中断,恢复后重复运行已完成任务会直接增加训练成本。 +- **代表性**:任务同时涉及历史数据读取、配置匹配、结果记录和 launch 调度,是典型的断点恢复问题。 +- **边界清楚**:修改只服务于 auto-tuner 从历史 CSV 恢复,不改变训练脚本、搜索算法或分布式执行后端。 +- **非平凡性**:实现需要恢复 CSV 中的真实数据类型,正确识别同一配置,并在不启动控制器的情况下把历史结果接回现有记录与调度流程。 +- **环境友好性**:测试使用受控的 CPU doubles 隔离真实训练进程,但执行 checkout 中的 `AutoTuner` 和完整 `launch()` 控制流。 + +## 4. 任务类型和标签 + +- 任务类型:`feature_enhancement` +- 执行后端:`cpu` +- 设备范围:`cpu_only` +- 模块标签:`[distributed, auto-tuner, launch, resume]` + +## 5. 验证思路 + +- 目标测试命令:`bash tests/test.sh` +- 目标测试文件:`test/swe_test/test_auto_tuner_resume.py` +- 修复前预期:既有搜索与 history list 行为通过;历史 CSV 读取、配置匹配和 launch 跳过已完成任务三个目标场景失败。 +- 修复后预期:CSV 值恢复为可用类型并生成副本,配置按历史顺序匹配,命中的任务写入 recorder 且不启动训练控制器;既有搜索行为保持通过。 +- P2P 候选:`test_existing_search_and_history_flow_is_unchanged`。 +- F2P 候选:`test_resume_history_loads_values_and_preserves_a_copy`、`test_resume_lookup_returns_first_matching_configuration`、`test_launch_reuses_history_without_starting_training`。 + +## 6. 环境与资源 + +- 资源需求:CPU +- Paddle 来源:`PaddlePaddle/Paddle` source checkout at `base_commit` +- 是否能提供 Docker:暂无 +- patch 类型:Python-only +- 环境建议:测试通过 `importlib` 直接执行 checkout 中的目标模块,并使用 controlled doubles 隔离搜索器、记录器和训练控制器;无需安装或覆盖 Paddle wheel,也无需启动分布式进程。 +- 最小测试命令:`bash tests/test.sh` +- 是否有 oracle 日志:由 SWE-Paddle verifier 结果另行维护 + +## 7. 风险自查 + +- 泄露风险:任务说明只描述中断恢复的用户行为,没有给出 Gold patch 的方法名、分支位置或具体代码结构。 +- 环境风险:测试只依赖 Python 标准库和 pytest,直接读取 checkout 源码,不受已安装 Paddle wheel 版本影响。 +- flaky 风险:不启动训练进程、不访问 GPU、网络或外部数据集,历史文件全部在 pytest 临时目录中生成。 +- 拆分风险:CSV 恢复、配置匹配和跳过重复训练共同构成同一个断点恢复流程,来源 PR 没有混入其他问题。 diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/solution/code.patch b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/solution/code.patch new file mode 100644 index 000000000..c64204afb --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/solution/code.patch @@ -0,0 +1,188 @@ +diff --git a/python/paddle/distributed/auto_tuner/tuner.py b/python/paddle/distributed/auto_tuner/tuner.py +index b3b6cbf3cdc5289faa7076eff8101d5f20893ba8..6a6a0ba4e082ff246aec90864f81324a597a0e17 100644 +--- a/python/paddle/distributed/auto_tuner/tuner.py ++++ b/python/paddle/distributed/auto_tuner/tuner.py +@@ -12,6 +12,8 @@ + # See the License for the specific language governing permissions and + # limitations under the License. + ++import csv ++import os + + from .utils import default_candidates, gbs_default_candidates + +@@ -54,6 +56,8 @@ class AutoTuner: + raise NotImplementedError() + + self.history_cfgs = [] ++ self.resume_cfgs = [] ++ self.tuner_cfg = tuner_cfg + + def search_once(self): + """Return a new task config.""" +@@ -67,3 +71,76 @@ class AutoTuner: + def add_cfg(self, cfg): + """Add cfg into history cfgs""" + self.history_cfgs.append(cfg) ++ ++ def resume_form_history(self, history_csv_path="./history.csv"): ++ """Resume form history csv file""" ++ # The breakpoint resume function does not start when the resume csv file does not exist. ++ if not os.path.exists(history_csv_path): ++ return ++ resume_csv_path = os.path.join( ++ os.path.dirname(history_csv_path), ++ f'{os.path.basename(history_csv_path).split(".")[0]}_copy.csv', ++ ) ++ with open(history_csv_path, "r") as fread: ++ reader = csv.reader(fread) ++ data_list = list(reader) ++ with open(resume_csv_path, "w") as fwrite: ++ writer = csv.writer(fwrite) ++ for row in data_list: ++ writer.writerow(row) ++ # chang str type to real type ++ for row in data_list: ++ for i, value in enumerate(row): ++ try: ++ row[i] = int(value) ++ except ValueError: ++ try: ++ row[i] = float(value) ++ except ValueError: ++ pass ++ ++ data_dict = [] ++ keys = data_list[0] ++ values = data_list[1:] ++ for val in values: ++ val = [x if x != '' else None for x in val] ++ val = [True if x == 'True' else x for x in val] ++ val = [False if x == 'False' else x for x in val] ++ dictionary = dict(zip(keys, val)) ++ time_val = -1 ++ target_key = self.tuner_cfg["metric_cfg"]["name"] ++ if dictionary[target_key]: ++ time_val = dictionary[target_key] ++ dictionary["time"] = time_val ++ data_dict.append(dictionary) ++ self.resume_cfgs = data_dict ++ ++ def get_cfg_from_resume(self, cur_cfg): ++ """Get cfg from resume cfgs""" ++ keys_to_compare = [ ++ 'mp_degree', ++ 'sharding_degree', ++ 'pp_degree', ++ 'dp_degree', ++ 'sharding_stage', ++ 'micro_batch_size', ++ 'vpp_degree', ++ 'use_recompute', ++ 'recompute_granularity', ++ 'num_gpus', ++ 'nodes', ++ 'global_batch_size', ++ 'sharding_overlap', ++ 'acc_steps', ++ ] ++ for cfg in self.resume_cfgs: ++ ret_is_same = True ++ for key in keys_to_compare: ++ if not cfg.get(key) and not cur_cfg.get(key): ++ continue ++ else: ++ is_same = str(cfg.get(key)) == str(cur_cfg.get(key)) ++ ret_is_same = ret_is_same and is_same ++ if ret_is_same: ++ return cfg ++ return None +diff --git a/python/paddle/distributed/launch/main.py b/python/paddle/distributed/launch/main.py +index 0869ac7bbfcd956ba1396390098865ce49e3f2c9..40caf7f223677b548cc0185644f26819a449dd78 100644 +--- a/python/paddle/distributed/launch/main.py ++++ b/python/paddle/distributed/launch/main.py +@@ -587,6 +587,10 @@ def launch(): + logger.info( + f"Launch {len(auto_tuner.algo.all_tasks)} tasks by auto tuner: " + ) ++ resume_csv_file_path = tuner_cfg.get( ++ "resume_csv_file_path", history_file_path ++ ) ++ auto_tuner.resume_form_history(resume_csv_file_path) + cur_cfg = auto_tuner.search_once() + auto_tuner.add_cfg(cur_cfg) + assert cur_cfg is not None, "No config can run." +@@ -658,6 +662,73 @@ def launch(): + ) + logger.info(f"Launch task: job_id {task_job_id}, log_dir {log_dir}") + ++ cur_resume_cfg = auto_tuner.get_cfg_from_resume(cur_cfg) ++ if cur_resume_cfg: ++ cur_cfg = cur_resume_cfg ++ cur_cfg['job_id'] = job_id ++ auto_tuner.history_cfgs.pop(-1) ++ auto_tuner.add_cfg(cur_cfg) ++ recorder.add_cfg(**cur_cfg) ++ cur_best_cfgs, err = recorder.get_best( ++ metric=tuner_cfg['metric_cfg']['name'], ++ direction=tuner_cfg['metric_cfg']['OptimizationDirection'], ++ ) ++ if not err: ++ ctx.logger.info(f"Current best config: {cur_best_cfgs}") ++ logger.info(f"Current best config: {cur_best_cfgs}") ++ else: ++ ctx.logger.info( ++ "Get best config failed. Currently no config can be run." ++ ) ++ logger.info( ++ "Get best config failed. Currently no config can be run." ++ ) ++ if ( ++ "sharding_overlap" in cur_cfg ++ and cur_cfg["sharding_overlap"] ++ ): ++ add_overlap_performance( ++ cur_cfg, tuner_cfg, recorder.history ++ ) ++ ++ if cur_cfg["error_info"]: ++ error_task_nums += 1 ++ error_info = cur_cfg["error_info"] ++ task_nums = len(auto_tuner.algo.all_tasks) ++ cur_task_id = auto_tuner.algo.idx ++ ctx.logger.info( ++ "Auto Tuner Schedule: [{}/{}], Pruned nums {}, Error nums {}, Error info {}, Remaining time {} min".format( ++ cur_task_id, ++ task_nums, ++ cur_task_id - job_id, ++ error_task_nums, ++ error_info, ++ round( ++ (task_nums - cur_task_id) * max_time_per_task / 60, ++ 2, ++ ), ++ ) ++ ) ++ logger.info( ++ "Auto Tuner Schedule: [{}/{}], Pruned nums {}, Error nums {}, Error info {}, Remaining time {} min".format( ++ cur_task_id, ++ task_nums, ++ cur_task_id - job_id, ++ error_task_nums, ++ error_info, ++ round( ++ (task_nums - cur_task_id) * max_time_per_task / 60, ++ 2, ++ ), ++ ) ++ ) ++ recorder.store_history(history_file_path) ++ # generate a new config ++ new_cfg = auto_tuner.search_once() ++ cur_cfg = copy.deepcopy(new_cfg) ++ auto_tuner.add_cfg(cur_cfg) ++ continue ++ + # in single dp estimation scene, just some nodes not all nodes run + ctx = gen_new_ctx(ctx, cur_cfg, tuner_cfg) + actual_nnodes = int(ctx.args.nnodes.split(":")[0]) diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/tests/test.patch b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/tests/test.patch new file mode 100644 index 000000000..5aacd7017 --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/tests/test.patch @@ -0,0 +1,293 @@ +diff --git a/test/swe_test/test_auto_tuner_resume.py b/test/swe_test/test_auto_tuner_resume.py +new file mode 100644 +index 0000000000000000000000000000000000000000..e13801bf9b3a5e5c451c831bace1f55912aea092 +--- /dev/null ++++ b/test/swe_test/test_auto_tuner_resume.py +@@ -0,0 +1,287 @@ ++import csv ++import importlib.util ++import json ++import logging ++import sys ++import types ++import uuid ++from pathlib import Path ++ ++import pytest ++ ++ ++REPO_ROOT = Path(__file__).resolve().parents[2] ++TUNER_SOURCE = ( ++ REPO_ROOT / "python/paddle/distributed/auto_tuner/tuner.py" ++) ++LAUNCH_SOURCE = REPO_ROOT / "python/paddle/distributed/launch/main.py" ++ ++ ++def _package(name): ++ module = types.ModuleType(name) ++ module.__path__ = [] ++ sys.modules[name] = module ++ return module ++ ++ ++def _module(name): ++ module = types.ModuleType(name) ++ sys.modules[name] = module ++ return module ++ ++ ++def _load_checkout_module(name, source): ++ spec = importlib.util.spec_from_file_location(name, source) ++ module = importlib.util.module_from_spec(spec) ++ sys.modules[name] = module ++ spec.loader.exec_module(module) ++ return module ++ ++ ++def _load_tuner(task_cfg): ++ root = f"_swe_pr60417_{uuid.uuid4().hex}" ++ _package(root) ++ _package(f"{root}.distributed") ++ _package(f"{root}.distributed.auto_tuner") ++ ++ utils = _module(f"{root}.distributed.auto_tuner.utils") ++ utils.default_candidates = lambda cfg: cfg.get("candidates", []) ++ utils.gbs_default_candidates = lambda cfg: cfg.get("candidates", []) ++ ++ search = _module(f"{root}.distributed.auto_tuner.search") ++ ++ class GridSearch: ++ def __init__(self, tuner_cfg): ++ self.all_tasks = [dict(task_cfg)] ++ self.idx = 0 ++ ++ def search_once(self, history_cfgs): ++ if self.idx >= len(self.all_tasks): ++ return None ++ result = dict(self.all_tasks[self.idx]) ++ self.idx += 1 ++ return result ++ ++ search.GridSearch = GridSearch ++ search.DpEstimationSearch = GridSearch ++ search.GBSSearch = GridSearch ++ search.CustomizeSearch = GridSearch ++ ++ tuner = _load_checkout_module( ++ f"{root}.distributed.auto_tuner.tuner", TUNER_SOURCE ++ ) ++ return root, tuner ++ ++ ++def _base_config(): ++ return { ++ "mp_degree": 1, ++ "sharding_degree": 1, ++ "pp_degree": 1, ++ "dp_degree": 1, ++ "sharding_stage": 1, ++ "micro_batch_size": 2, ++ "vpp_degree": 1, ++ "use_recompute": False, ++ "recompute_granularity": "full", ++ "num_gpus": 1, ++ "nodes": 1, ++ "global_batch_size": 8, ++ "sharding_overlap": False, ++ } ++ ++ ++def _tuner_config(): ++ return { ++ "task_limit": 10, ++ "search_algo": {"name": "grid"}, ++ "metric_cfg": { ++ "name": "step_time", ++ "OptimizationDirection": "Minimize", ++ }, ++ "model_cfg": {"global_batch_size": 8}, ++ "run_best": False, ++ } ++ ++ ++def _write_history(path, rows): ++ with path.open("w", newline="", encoding="utf-8") as stream: ++ writer = csv.DictWriter(stream, fieldnames=list(rows[0])) ++ writer.writeheader() ++ writer.writerows(rows) ++ ++ ++def test_existing_search_and_history_flow_is_unchanged(): ++ task = _base_config() ++ _, tuner_module = _load_tuner(task) ++ tuner = tuner_module.AutoTuner(_tuner_config()) ++ ++ found = tuner.search_once() ++ tuner.add_cfg(found) ++ ++ assert found == task ++ assert tuner.history_cfgs == [task] ++ assert tuner.search_once() is None ++ ++ ++def test_resume_history_loads_values_and_preserves_a_copy(tmp_path): ++ task = _base_config() ++ _, tuner_module = _load_tuner(task) ++ tuner = tuner_module.AutoTuner(_tuner_config()) ++ history = tmp_path / "training.history.csv" ++ row = { ++ **task, ++ "step_time": 1.25, ++ "max_mem_usage": "", ++ "error_info": "", ++ } ++ _write_history(history, [row]) ++ ++ tuner.resume_form_history(history) ++ ++ assert len(tuner.resume_cfgs) == 1 ++ resumed = tuner.resume_cfgs[0] ++ assert resumed["micro_batch_size"] == 2 ++ assert resumed["use_recompute"] is False ++ assert resumed["step_time"] == pytest.approx(1.25) ++ assert resumed["max_mem_usage"] is None ++ assert resumed["time"] == pytest.approx(1.25) ++ copied = tmp_path / "training_copy.csv" ++ assert copied.exists() ++ with copied.open(newline="", encoding="utf-8") as stream: ++ assert list(csv.DictReader(stream))[0]["step_time"] == "1.25" ++ ++ ++def test_resume_lookup_returns_first_matching_configuration(): ++ task = _base_config() ++ _, tuner_module = _load_tuner(task) ++ tuner = tuner_module.AutoTuner(_tuner_config()) ++ tuner.resume_cfgs = [ ++ {**task, "marker": "first", "step_time": 1.25}, ++ {**task, "marker": "second", "step_time": 1.10}, ++ ] ++ ++ matched = tuner.get_cfg_from_resume(dict(task)) ++ different = tuner.get_cfg_from_resume({**task, "mp_degree": 2}) ++ ++ assert matched["marker"] == "first" ++ assert different is None ++ ++ ++def test_launch_reuses_history_without_starting_training(tmp_path): ++ task = _base_config() ++ history_row = { ++ **task, ++ "acc_steps": 4, ++ "step_time": 1.25, ++ "max_mem_usage": 1024, ++ "error_info": "", ++ } ++ history = tmp_path / "resume.csv" ++ _write_history(history, [history_row]) ++ config = _tuner_config() ++ config["resume_csv_file_path"] = str(history) ++ config_path = tmp_path / "tuner.json" ++ config_path.write_text(json.dumps(config), encoding="utf-8") ++ training_script = tmp_path / "train.py" ++ training_script.write_text("raise AssertionError('must not run')\n") ++ ++ root, tuner_module = _load_tuner(task) ++ _package(f"{root}.distributed.launch") ++ ++ class Logger: ++ def info(self, *args, **kwargs): ++ pass ++ ++ def warning(self, *args, **kwargs): ++ pass ++ ++ class FakeContext: ++ def __init__(self): ++ self.args = types.SimpleNamespace( ++ auto_tuner_json=str(config_path), ++ training_script=str(training_script), ++ training_script_args=[], ++ devices="0", ++ nnodes="1", ++ log_dir=str(tmp_path / "logs"), ++ max_restart=0, ++ master=None, ++ job_id="test", ++ ) ++ self.logger = Logger() ++ ++ def is_legacy_mode(self): ++ return False ++ ++ def is_auto_tuner_mode(self): ++ return True ++ ++ def set_envs(self, envs): ++ self.envs = envs ++ ++ context_module = _module(f"{root}.distributed.launch.context") ++ context_module.Context = FakeContext ++ ++ recorder_instances = [] ++ recorder_module = _module( ++ f"{root}.distributed.auto_tuner.recorder" ++ ) ++ ++ class HistoryRecorder: ++ def __init__(self, *args, **kwargs): ++ self.history = [] ++ self.store_calls = [] ++ recorder_instances.append(self) ++ ++ def add_cfg(self, **cfg): ++ self.history.append(dict(cfg)) ++ ++ def get_best(self, **kwargs): ++ return min(self.history, key=lambda cfg: cfg["time"]), False ++ ++ def store_history(self, path): ++ self.store_calls.append(path) ++ ++ def clean_history(self): ++ self.history.clear() ++ ++ recorder_module.HistoryRecorder = HistoryRecorder ++ sys.modules[f"{root}.distributed.auto_tuner.tuner"] = tuner_module ++ ++ utils = sys.modules[f"{root}.distributed.auto_tuner.utils"] ++ utils.add_overlap_performance = lambda *args, **kwargs: None ++ utils.find_error_from_log = lambda *args, **kwargs: [] ++ utils.gen_new_args = lambda args, cfg, tuner_cfg, **kwargs: list(args) ++ utils.gen_new_ctx = lambda ctx, cfg, tuner_cfg: ctx ++ utils.read_log = lambda *args, **kwargs: (None, None, 0) ++ utils.read_step_time_log = lambda *args, **kwargs: None ++ ++ controller_calls = [] ++ controllers = _module(f"{root}.distributed.launch.controllers") ++ ++ def init_controller(ctx): ++ controller_calls.append(ctx) ++ raise AssertionError("a resumed task must not start a controller") ++ ++ controllers.init = init_controller ++ ++ launch_module = _load_checkout_module( ++ f"{root}.distributed.launch.main", LAUNCH_SOURCE ++ ) ++ ++ try: ++ with pytest.raises(SystemExit): ++ launch_module.launch() ++ finally: ++ logger = logging.getLogger("auto_tuner") ++ for handler in list(logger.handlers): ++ handler.close() ++ logger.removeHandler(handler) ++ ++ assert controller_calls == [] ++ assert len(recorder_instances) == 1 ++ assert recorder_instances[0].history[0]["step_time"] == pytest.approx( ++ 1.25 ++ ) ++ assert recorder_instances[0].history[0]["job_id"] == 1 diff --git a/swe-paddle/tasks/PaddlePaddle__Paddle-60417/tests/test.sh b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/tests/test.sh new file mode 100644 index 000000000..7feaf1025 --- /dev/null +++ b/swe-paddle/tasks/PaddlePaddle__Paddle-60417/tests/test.sh @@ -0,0 +1,4 @@ +#!/usr/bin/env bash + +set -euo pipefail +python -m pytest test/swe_test/test_auto_tuner_resume.py -q