「view this page in B3 βῆτα server」

Revisions №61021

branch: master 「№61021」
Commited by: Vikram K. Mulligan
GitHub commit link: 「410380c2d518abd3」 「№4089」
Difference from previous tested commit:  code diff
Commit date: 2019-11-04 16:14:37
linux.clang linux.gcc linux.srlz mac.clang
debug
release
unit
PyRosetta4.notebook linux.clang.cxx11thread.serialization.python37.PyRosetta4.unit linux.gcc.python36.PyRosetta4.unit mac.PyRosetta.unit build.clean.debug cppcheck mysql postgres linux.clang.python36.build.debug linux.zeromq.debug mpi mpi.serialization linux.icc.build.debug OpenCL mac.clang.python36.build.debug build.header build.levels ninja graphics static linux.ui mac.ui build.xcode beautification code_quality.clang_analysis serialization integration.mpi integration.release_debug integration.tensorflow integration.thread integration performance profile release.source scientific.mp_f19_sequence_recovery linux.clang.score linux.gcc.score mac.clang.score linux.scripts.pyrosetta scripts.rosetta.parse scripts.rosetta.validate scripts.rosetta.verify linux.clang.unit.release linux.gcc.unit.release util.apps

Merge pull request #4089 from RosettaCommons/vmullig/multithreaded_packer_setup2 Multi-thread packer setup using the RosettaThreadManager work vector interface This pull request replaces pull request #3964. A current challenge is figuring out how to get all of the existing Rosetta protocols to take advantage of parallel processing, with as little burden placed on the user as possible. Much of the time in a typical Rosetta run is spent on: - Computing interaction graphs (easy to parallelize). - Running packing trajectories (hard to parallelize). - Computing gradient vectors and inverse Hessian approximations for minimization (easy to parallelize). - Computing the scoring function (somewhat hard to parallelize). - Computing kinematics (may be parallelizable?) This pull request aims to address the first point: it will allow interaction graphs to be calculated using the `RosettaThreadManager`'s thread pool, implemented in pull request #3946. The `RosettaThreadManager` is intended to allow us to experiment with parallelizing things on many different levels in Rosetta (from apps to job distributors to movers, down to low-level components like the packer, minimizer, and scoring machinery), without having things that call things that call things create thread explosions. (So, for example, Rosetta could launch 16 threads, the job distributor could request parallel job execution in 4 threads, and each parallel job thread could request up to 16 threads for packing, receiving 1 to 13 depending on availability so that the global pool never exceeds 16 running threads. Note that switching the `MultithreadedJobDistributor` to use the `RosettaThreadManager` will be for a future pull request.) It also allows very finely-grained threading, without having to encapsulate atomistic tasks in jobs that might have more overhead than the tasks themselves (as would be the case if we wanted to consider individual entries in the interaction graph calculation as "jobs"). Tasks: - [ ] **FIX PYROSETTA DISTRIBUTED ISSUE** - [x] Pull request #4310 must be merged before this one. - [x] Pull request #3946 must be merged before this one. - [x] Pull request #3957 must be merged before this one. - [x] Add option (flag) for default number of threads that the packer will request. - [x] Ensure that `fixbb` respects the option. - [x] Test `fixbb`. - [x] Ensure that the `PackRotamersMover` respects the option by way of TaskOperations. - [x] Test `PackRotamersMover`. (Update test). - [x] Ensure that the test checks that the single- and multi-threaded PDB output is identical. - [x] Ensure that `FastRelax` respects the option by way of TaskOperations. - [x] Ensure that `FastDesign` respects the option by way of TaskOperations. - [x] Test `FastDesign`. (Update test). - [x] Update the no-threads case in the `FastDesign` and `PackRotamersMover` tests. - [x] ~~Modify `PackRotamersMover`:~~ - [x] ~~Add `MultithreadedMover` class that derives from `Mover` class.~~ - [x] ~~Have `MultithreadedMover` take an XML-configurable option for number of threads to request.~~ - [x] ~~Make the `PackRotamersMover` a `MultithreadedMover`, with its "number of threads" option overriding the global default if and only if provided by the user.~~ - [x] ~~Have the `PackRotamersMover` call the multithreaded `core::pack::pack_rotamers()` if number of threads is greater than 1.~~ - [x] Parallelize interaction graph setup. - [x] Pre-identify (in a single thread) the one- and two-body interactions to be computed. - [x] Pre-create the nodes and edges needed (using a single thread) in the interaction graph. - [x] Handle the symmetric case, too -- need virtual function overrides in `SymmetricRotamerSets`. - [x] Modify the interaction graph to allow parallel, threadsafe computation of elements. - [x] Make a work vector of all onebody and twobody energies to compute, and pass it to the `RosettaThreadManager`. - [x] Handle the lazy IG case -- probably not much that multithreading can do, but take a look anyway. --> Can only multithread the onebody energy precalculation. Meh. Better than nothing. - [x] Integration/unit test the 1-thread case. - [x] Integration/unit test the multi-threaded case. - In the future, integration test multi-threaded packing in the context of multi-threaded job distribution. - [x] Looks like I've got the same rare segfault issue. Track it down. --> Got it. Can't call functions that finalize edges from threads. - [x] Integration test: confirm that interaction graphs computed in parallel are identical to interaction graphs computed in serial. - [x] Symmetric case, too. - [x] Move the `-packing:packing_threads` option to `-multi_threading:packing_threads` to keep all of these options together. - [x] Same for the optH threads option. - [x] Pilot app for performance testing. - [x] Switch tracer output to debug output, or remove entirely. - [x] Special-case the logic for 1 thread requested to use the non-threaded code. - [x] Do this for `SymmetricRotamerSets`, too. - [x] Fix the twobody energy calculation for the symmetric case. - [x] Handle the case of threads requested in a non-threaded build. - [x] **FIX PYROSETTA DISTRIBUTED ISSUE.** - [ ] Developer documentation. - [ ] User documentation. - [x] Beauty. TODO: - [x] Make commandline flag more specific (`-interaction_graph_threads` instead of `packing_threads`.) - [x] Take out new options from hydrate, FastRelax, FastDesign, PackRotamerMover, etc. for threads. Instead... - [x] Add a new TaskOperation for setting number or threads. - [x] PackerTask stores number of threads to request for interaction graph setup. - [x] Maintain commutativity: taskop can only REDUCE the number of threads. - [x] Have default behaviour be to use all available threads. - [x] Switch integration tests to use this. - [x] See note about Andrew's suggestion for removing code duplication; using work vector even in non-threaded build. - [x] Allow thread manager to be called in threaded or non-threaded build. - [x] Ensure that thread manager just executes the work that it is given (single-threaded) in the non-threaded build. - ~~Add to correctness test at least one score value. (Especially in symmetric case.)~~ --> Already in unit tests. - [x] Pull construction of temporary rotamer sets in symmetry code out of the precomputation and into the work vector work units (so that they don't persist). - [x] Scrap the MultithreadedMover base class. For now, TaskOperations are the way to go; later, if we have multithreaded movers, we can add utility functions for their setup rather than having a base class for it. At some point in the future: - At some point, remove the default thread counts from `core/util/disulfide_util.hh`, find everything that fails to compile, and switch it all to explicitly request threads. - At some point, remove the default thread counts from `core/pack/pack_rotamers.hh`, find everything that fails to compile, and switch it all to explicitly request threads. - At some point, remove the default thread counts from `core/util/SwitchResidueTypeSet.hh`, find everything that fails to compile, and switch it all to explicitly request threads.

Vikram K. Mulligan 1 week
Note: the unit test failure here is stochastic, and is due to lack of thread safety of the score12 scorefunction. Switching this test to ref2015 stabilizes it. At some point, I'll track down the issue in score12 that makes it non-threadsafe, but for now it's a low priority.
...
Test: linux.clang.integration.release_debug_no_symbols

Failed sub-tests (click for more details):
ConsensusLoopDesign add_helix_sequence_constraints add_helix_sequence_constraints_advanced_setup antibody_H3 assemble_domains_jd2 beta_strand_homodimer buried_unsat_voids_hbnet_design buried_unsat_voids_hbnet_design_symm central_class_modification cycpep_design_pipeline d_workflow ddG_of_mutation ddG_scan design_glycans design_w_custom_palette-CAAs design_w_custom_palette-NCAAs design_w_custom_palette-RNA doug_dock_design_min_mod2_cal_cal duplicate_header_guards enzdes fiber_diffraction_fad fold_and_dock hbnet hbnet_asymm hbnet_energy hbnet_energy_rosettascripts_linear hbnet_energy_symm hbnet_use_input_rot helix_from_sequence hotspot_hashing hydrate hydrate_relax jd2test_mmCIFIO jd2test_mmCIFin_PDBout min_pack_min mirror_symm mp_find_interface mp_mutate_relax mp_range_relax multithreaded_fastdesign multithreaded_fixbb multithreaded_interaction_graph_accuracy multithreaded_interaction_graph_accuracy_symm multithreaded_packrotamersmover netcharge_design oligourea_design ppi_v3_suiteA range_relax_w_cst remodel_disulfides_rosettascripts remodel_helical_repeat rna_denovo_RNP_low_res rna_denovo_dna_bridge rna_design sequence_recovery simple_cycpep_predict_design simple_cycpep_predict_square_pyramidal_metal simple_cycpep_predict_tetrahedral_metal_asp simple_cycpep_predict_trigonal_planar_metal simple_cycpep_predict_trigonal_pyramidal_metal splice_out_H1_H2_longer splice_out_H1_H2_same splice_out_H1_H2_shorter splice_out_L1_L2_longer splice_out_L1_L2_same splice_out_L1_L2_shorter stored_residue_subset swa_protein_prepack swm_general_polymer_sampler test1_benchmark threefold_symm_peptide_design voids_penalty_energy_design voids_penalty_energy_design_symmetry zinc_heterodimer
Test: mac.clang.integration

Failed sub-tests (click for more details):
ConsensusLoopDesign add_helix_sequence_constraints add_helix_sequence_constraints_advanced_setup antibody_H3 assemble_domains_jd2 buried_unsat_voids_hbnet_design buried_unsat_voids_hbnet_design_symm central_class_modification cycpep_design_pipeline d_workflow ddG_of_mutation ddG_scan design_glycans design_w_custom_palette-CAAs design_w_custom_palette-NCAAs design_w_custom_palette-RNA doug_dock_design_min_mod2_cal_cal duplicate_header_guards enzdes fiber_diffraction_fad fold_and_dock hbnet hbnet_asymm hbnet_energy hbnet_energy_rosettascripts_linear hbnet_energy_symm hbnet_use_input_rot helix_from_sequence hotspot_hashing hydrate hydrate_relax jd2test_mmCIFIO jd2test_mmCIFin_PDBout mirror_symm mp_find_interface mp_mutate_relax mp_range_relax multithreaded_fastdesign multithreaded_fixbb multithreaded_interaction_graph_accuracy multithreaded_interaction_graph_accuracy_symm multithreaded_packrotamersmover netcharge_design oligourea_design ppi_v3_suiteA range_relax_w_cst remodel remodel_disulfides remodel_disulfides_rosettascripts remodel_helical_repeat rna_denovo_RNP_low_res rna_denovo_dna_bridge rna_design rosetta_scripts_jd3 simple_cycpep_predict_design splice_out_H1_H2_longer splice_out_H1_H2_same splice_out_H1_H2_shorter splice_out_L1_L2_longer splice_out_L1_L2_same splice_out_L1_L2_shorter stored_residue_subset swa_protein_prepack swm_general_polymer_sampler test1_benchmark threefold_symm_peptide_design voids_penalty_energy_design voids_penalty_energy_design_symmetry zinc_heterodimer
Test: linux.clang.profile

Failed sub-tests (click for more details):
AnchoredDesign
Test: linux.clang.scientific.mp_f19_sequence_recovery

 View log

Loading...

 View log in dialog  View log in log in separate window