Multiproc v3 - #839

Draft

nkrah wants to merge 180 commits into

Commits on Oct 9, 2024

Correct typo in comment
nkrah
committed
fe4c659
Implement DynamicGateObject.reassign_subset_of_dynamic_params()
nkrah
committed
b66c665
Add attribute process_index to SimulationEngine (not used yet)
nkrah
committed
0456d04
First steps towards multi processing
nkrah
committed
30526c8
Add test080_multiprocessing_1.py
nkrah
committed
a7434e5
Add test030_dose_motion_dynamic_param_multiproc.py
nkrah
committed
605192f
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
e4bc3a7

Commits on Oct 13, 2024

Implement MultiProcessingHandler classes
nkrah
committed
523fdc9
create test080_multiprocessing_handler.py
nkrah
committed
cea24cf
Implement import_data_from_actor_output()
nkrah
committed
ae37675
Implement import_user_output_from_actor()
nkrah
committed
9c74aa2
Implement import_user_output_from_actor in ActorBase
nkrah
committed
87af53e
Change local variable name in reassign_dynamic_params_for_process()
nkrah
committed
196d954
Implement FinalizeSimulation() in VoxelDepositActor
nkrah
committed
138a44f
Add simulation_id to SimulationOutput
nkrah
committed
a796bec
Implement SimulationOutput.store_output_from_simulation_engine()
nkrah
committed
1375e4f
Simplify code in run_engine()
nkrah
committed
fc90573
Implement SimulationMetaData class
nkrah
committed
91f6b7f
Use SimulationMetaData in Simulation
nkrah
committed
722552a
Update MultiProcessingHandlerBase class
nkrah
committed
c7a317b
Remove obsolete generate_run_timing_interval_map() method
nkrah
committed
52bde4f
Update imports in managers.py
nkrah
committed
3d1448b
Update run_in_process()
nkrah
committed
fc0d155
Store number_of_sub_processes and start_new_process in simulation_meta_data
nkrah
committed
8b43fb2
Introduce avoid_write_to_disk_in_subprocess kwarg in Simulation.run()
nkrah
committed
a544da9
Use multi_proc_handler in Simulation.run()
nkrah
committed
19c9b17
Trigger import_user_output_from_actor() after multi_proc run
nkrah
committed
23fb55e
store meta_data after run
nkrah
committed
a1ce476
Get parameters from sources after multiproc run
nkrah
committed
a120987
Remove obsolete code
nkrah
committed
78bcd34
Trigger FinalizeSimulation() at the end of a run
nkrah
committed
39a45c8
remove obsolete code
nkrah
committed
7e47bec
Add test008_dose_actor_multiproc.py
nkrah
committed
b8581ce
Update test080_multiprocessing_1.py
nkrah
committed
3d4fe1e
Update GateObject to rename property 'warnings' in Simulation
nkrah
committed
dcebecc
Update run_engine to use Simulation.meta_data
nkrah
committed
c804709
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
27b49c6
Rename FinalizeSimulation() to EndOfMultiProcessAction()
nkrah
committed
0a25f0d

Commits on Oct 14, 2024

Implement explicit inplace_merge_with in ItkImageDataItem
nkrah
committed
c4cf4a4
Adapt merge_data to try to accelerate it (WIP)
nkrah
committed
d94c0e4
Implement sum_itk_images based on SimpleITK (workaround)
nkrah
committed
7d919d8
remove debug prints and clean code
nkrah
committed
7bfa8b7
In BaseUserInterfaceToActorOutput.__getstate__: always use return_dict = self.__dict__.copy()
nkrah
committed
11f6740
Implement reset_user_output() and reset_data()
nkrah
committed
1383057
remove debug print
nkrah
committed
6bba1b7
Update comment
nkrah
committed
56061bc
Update test008_dose_actor_multiproc.py
nkrah
committed
aebcc40
Update test009_voxels_dynamic.py (not relevant for test result)
nkrah
committed
557e679
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
15d72cd

Commits on Oct 24, 2024

Correct typo in comment
nkrah
committed
53f5ff0
Implement DynamicGateObject.reassign_subset_of_dynamic_params()
nkrah
committed
1463013
Add attribute process_index to SimulationEngine (not used yet)
nkrah
committed
b3e8f0c
First steps towards multi processing
nkrah
committed
0b7569b
Add test080_multiprocessing_1.py
nkrah
committed
d17682f
Add test030_dose_motion_dynamic_param_multiproc.py
nkrah
committed
fb3cb56
[pre-commit.ci] Automatic python and c++ formatting

pre-commit-ci[bot]
authored and
nkrah
committed
aa5e710
Implement MultiProcessingHandler classes
nkrah
committed
76fbfba
create test080_multiprocessing_handler.py
nkrah
committed
6973f13
Implement import_data_from_actor_output()
nkrah
committed
0260522
Implement import_user_output_from_actor()
nkrah
committed
3b93451
Implement import_user_output_from_actor in ActorBase
nkrah
committed
4d141c3
Change local variable name in reassign_dynamic_params_for_process()
nkrah
committed
f06e627
Implement FinalizeSimulation() in VoxelDepositActor
nkrah
committed
3ea4bc7
Add simulation_id to SimulationOutput
nkrah
committed
504673f
Implement SimulationOutput.store_output_from_simulation_engine()
nkrah
committed
76b0f86
Simplify code in run_engine()
nkrah
committed
fd799dd
Implement SimulationMetaData class
nkrah
committed
3b6d94d
Use SimulationMetaData in Simulation
nkrah
committed
c4cf0f0
Update MultiProcessingHandlerBase class
nkrah
committed
8b28b15
Remove obsolete generate_run_timing_interval_map() method
nkrah
committed
b1f2d21
Update imports in managers.py
nkrah
committed
2a033b7
Update run_in_process()
nkrah
committed
9a6f871
Store number_of_sub_processes and start_new_process in simulation_meta_data
nkrah
committed
18b49a0
Introduce avoid_write_to_disk_in_subprocess kwarg in Simulation.run()
nkrah
committed
0a4b237
Use multi_proc_handler in Simulation.run()
nkrah
committed
805468a
Trigger import_user_output_from_actor() after multi_proc run
nkrah
committed
fa5ece3
store meta_data after run
nkrah
committed
af06335
Get parameters from sources after multiproc run
nkrah
committed
002c000
Remove obsolete code
nkrah
committed
43ae7d2
Trigger FinalizeSimulation() at the end of a run
nkrah
committed
78eecbb
remove obsolete code
nkrah
committed
c6b9ae1
Add test008_dose_actor_multiproc.py
nkrah
committed
d6d5b37
Update test080_multiprocessing_1.py
nkrah
committed
53ceb61
Update GateObject to rename property 'warnings' in Simulation
nkrah
committed
38896f0
Update run_engine to use Simulation.meta_data
nkrah
committed
0e9928d
[pre-commit.ci] Automatic python and c++ formatting

pre-commit-ci[bot]
authored and
nkrah
committed
0020307
Rename FinalizeSimulation() to EndOfMultiProcessAction()
nkrah
committed
2612857
Implement explicit inplace_merge_with in ItkImageDataItem
nkrah
committed
956be1c
Adapt merge_data to try to accelerate it (WIP)
nkrah
committed
7bb0000
Implement sum_itk_images based on SimpleITK (workaround)
nkrah
committed
d5c7c40
remove debug prints and clean code
nkrah
committed
481dbc4
In BaseUserInterfaceToActorOutput.__getstate__: always use return_dict = self.__dict__.copy()
nkrah
committed
bffb06c
Implement reset_user_output() and reset_data()
nkrah
committed
fecd1e1
remove debug print
nkrah
committed
845544e
Update comment
nkrah
committed
2450ef5
Update test008_dose_actor_multiproc.py
nkrah
committed
ee5b1df
Update test009_voxels_dynamic.py (not relevant for test result)
nkrah
committed
2f49d87
Increment random_seed in run_in_process
nkrah
committed
5ccba4e
Extend inplace_merge_with() to accept *other, i.e. an unpackaged list
nkrah
committed
775ddb2
Fix ItkImageDataItem.inplace_merge_with()
nkrah
committed
ccc141d
Fix typo in error message
nkrah
committed
2df161a
Update test008_dose_actor_multiproc.py
nkrah
committed
5dcb80b
Deprecate unused kwarg 'stat' in assert_images()
nkrah
committed
79788b8
Update multiproc logic in Simulation class
nkrah
committed
22411d4
Merge remote-tracking branch 'origin/multiproc' into multiproc
nkrah
committed
d21bcbf
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
fb36286
Implement reset_data() in DataItem base class
nkrah
committed
27f4204
update imports in dataitems.py
nkrah
committed
994fded
Implement StatisticsDataItem
nkrah
committed
1e706b4
Implement StatisticsItemContainer
nkrah
committed
b17a98c
Let SimulationStatisticsActor use new ActorOutputStatisticsActor
nkrah
committed
09a95c4
Implement ActorOutputStatisticsActor
nkrah
committed
a2cd755
Implement UserInterfaceToActorOutputStatisticsActor
nkrah
committed
abdaacd

Commits on Oct 25, 2024

let default_suffix be an instance attribute of the ActorOutput classes rather than a class attribute.
nkrah
committed
37b6bba
Implement ActorOutputUsingDataItemContainer.reset_data()
nkrah
committed
3b0e499
Add process_cls(ActorOutputStatisticsActor)
nkrah
committed
6ce548c
Remove unused store_output_data() method
nkrah
committed
fd4d7fc
Update test008_dose_actor_multiproc.py
nkrah
committed
e1a6713
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
3c4e7b5

Commits on Oct 28, 2024

Update physics and sources documentation

dsarrut
authored and
nkrah
committed
51733c1
Add test080_multiprocessing_1.py
nkrah
committed
d76b64f
[pre-commit.ci] Automatic python and c++ formatting

pre-commit-ci[bot]
authored and
nkrah
committed
98752bf
Implement MultiProcessingHandler classes
nkrah
committed
8217a0b
create test080_multiprocessing_handler.py
nkrah
committed
3193890
Implement FinalizeSimulation() in VoxelDepositActor
nkrah
committed
552fae4
Add test008_dose_actor_multiproc.py
nkrah
committed
47bfa06
[pre-commit.ci] Automatic python and c++ formatting

pre-commit-ci[bot]
authored and
nkrah
committed
352ac37
Rename FinalizeSimulation() to EndOfMultiProcessAction()
nkrah
committed
cb90f29
Update test008_dose_actor_multiproc.py
nkrah
committed
d4d0f4b
[pre-commit.ci] Automatic python and c++ formatting

pre-commit-ci[bot]
authored and
nkrah
committed
8cb80b8
Merge remote-tracking branch 'origin/master' into multiproc
nkrah
committed
092dd69
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
ad36e56
rename test080 on multi processing to test082_xxx
nkrah
committed
216209f
Add merge_root to processing.py
nkrah
committed
8063a4d
rename import_data_from_actor_output to merge_data_from_actor_output
nkrah
committed
3729d25
Extend signature of import_user_output_from_actor by **kwargs
nkrah
committed
fafbb32
Add process_index attribute to SimulationEngine and SimulationOutput
nkrah
committed
69d9649
Implement clear_output_dir_before_run
nkrah
committed
33fdb50
Implement sub_process_registry
nkrah
committed
26d6c54
Make multi_proc_handler an instance attribute
nkrah
committed
9fe4915
Add output_dir as argument to run_in_process()
nkrah
committed
e1d58bf
Let each process store a json in the subfolder
nkrah
committed
a70ba52
Capsulate code into merge_simulations_from_multiprocessing()
nkrah
committed
f5f0d44
check global variable __spec__ in run() to avoid breaking when running multiproc in interactive session
nkrah
committed
c9b9b66
Remove merge_root() and unicity(): now in ActorOutputRoot class
nkrah
committed
43a8292
Implement ActorOutputRootmerge_data_from_actor_output()
nkrah
committed
facc182
Add test019_phsp_actor_multiproc.py
nkrah
committed
5d06d5b
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
01d0ea5
Remove obsolete imports in processing.py
nkrah
committed
245b73b

Commits on Oct 31, 2024

adapt read_stat_file_json() to updated StatsActor
nkrah
committed
451e7da
remove redundant code in test008_dose_actor_multiproc.py
nkrah
committed
f6f348b
Merge remote-tracking branch 'origin/master' into multiproc
nkrah
committed
09b4830
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
9e69420
activate ssh session on github
nkrah
committed
0eb6b81
update test082_multiprocessing_1.py
nkrah
committed
bbec500
pick up simulation_id as meta_data after a simulation
nkrah
committed
ec56329
add explicit process_cls() for MultiProcessHandler classes
nkrah
committed
039ef82
Update file names in test081_simulation_optigan_with_random_seed.py
nkrah
committed
dd3068c
Adapt test006_runs.py to updated StatsActorOutput
nkrah
committed
0ac5519

Commits on Nov 29, 2024

Merge remote-tracking branch 'origin/automatic_docstring_for_actoroutput' into multiproc_v2
nkrah
committed
6d8c0a2
[pre-commit.ci] Automatic python and c++ formatting
pre-commit-ci[bot]
committed
3860798
Remove obsolete variable available_data_container_classes
nkrah
committed
fe4bae0
remove obsolete imports
nkrah
committed
2ed23a7
remove obsolete method available_data_container_classes()
nkrah
committed
611ee81
Put methods back in order after merge
nkrah
committed
b2af2d7
Remove obsolete merge_into_merged_data
nkrah
committed
2060cae
Correct wrongly merged code
nkrah
committed
51e0bc9
renamed test082 to test084
nkrah
committed
4186880

Commits on Nov 7, 2025