Merge pull request #2768 from RosettaCommons/aleaverfay/revive_resource_manager
merging branch aleaverfay/revive_resource_manager into master.
New implementation of the ResourceManager.
Instead of being this all seeing, all knowing piece of global data that knows what job is running and what resource you mean when you say "3mers", the new ResourceManager instead is a significantly more humble contraption that's responsible for loading data that should stay in memory between jobs and deallocate that memory when it's told it is no longer needed.
The ResourceManager has three components:
1. The ResourceManager itself,
2. A set of ResourceLocators that can open a stream from some sort of input source (e.g. a file, or a database query), and
3. A set of ResourceLoaders that are responsible for loading a kind of Resrouce (e.g. a PoseResourceLoader responsible for loading Poses* into memory)
A Resource, then, is a bitwise constant object that can be shared between concurrently running jobs. E.g., a loop-hash library, a fragment set, or a Pose*, or the SEWING graph. The ResourceManager only grants const access to the Resources it holds. A Resource is what you want to use when you have large, load-once data that is either expensive to create from disk, or occupies so much memory that you want to ensure there is only one of (but where different jobs might use different versions of it, so a Singleton would be inappropriate).
The ResourceManager is accessible currently through rosetta_scripts_jd3. There is a new top-level \<Resources\> block to the parser::protocol XML file to say "These are resources that will be used for this job" so that the RosettaScriptsJobQueen (RSJQ) will know to put those resources into a special const-only section of the DataMap before the Movers and Filters and whatnot are loaded. Movers who want to use resources will ask the DataMap for the resources they need by name during "parse_my_tag". The RSJQ will figure out at startup when each resource can be deallocated and will tell the ResourceManager "you may now deallocate resource X" at the appropriate time. (The use of script_vars will make it trivially easy to have different resources used for different jobs).
My eventual goal is to make abinitio work in RosettaScripts without the corrupting influence of global data (i.e. the sin against humanity of the frag_cache_ static data in core::fragment::FragmentIO).
I would like people to start thinking more carefully about the lifetime of the data that they load and to begin integrating the ResourceManager into their protocol development.
In the near future, I will be looking for more clients for the ResourceManager -- Movers and Filters that need load-once data -- and will be adding new attributes to these Movers and Filters to retrieve resources from the DataMap instead of loading them from scratch at each parse_my_tag.
I will also be adding the ResourceManager to the (JD2) rosetta_scripts application so that it can be used there as well. I will also have to figure out how to incorporate it into Jack's new MultistageRosettaScripts.
This PR will create quite a few cosmetic integration test changes. The old resource manager, though not directly used by anyone, was being indirectly used by lots of things, and the convoluted process of it's wake up did work that wasn't necessary and sent output to Tracers while doing so. I have also removed several of the integration tests for the old resource manager, and adapted several others to use the new resource manager. I paid close attention to all of the tests that were changing on my machine, fixing a handful of bugs that I had introduced along the way.
(The testing server is showing me far more trajectory changes than I see locally. They are also inconsistent from revision to revision, so I think there is just something funny about the testing server.)
*Pose itself is not a bitwise-constant class, and so instead of creating Pose objects and putting them into the ResourceManager, the PoseResourceLoader will create a wrapper class called a PoseResource which will give consumers a *copy* of the Pose that was loaded. This will hopefully be the outlier where the resource is loaded once, but where there might be multiple copies of the Resource in memory at one time.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2904 from RosettaCommons/JackMaguire/MCHBNetCorrection
MC HBNet Correction. There was a small bug in the MC HBNet code that did not impact the legitimacy of results but strongly hindered the sampling ability of the MC HBNet protocol (credit goes to Scott for discovering there was a bug). That bug is now corrected and I took this opportunity to clean up the code in the class surrounding the bug.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2905 from RosettaCommons/benchmark
Updating benchmark python-install script so it now use default platform compiling when building Python. Updating scons .settings files to fix weekly release builds.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2898 from RosettaCommons/vmullig/fix_voids_penalty_config
Fixing a little problem in EnergyMethodsOptions setter functions that was preventing configuration of the voids_penalty energy term from RosettaScripts XML.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2895 from RosettaCommons/vmullig/fix_bluegene_build_again
Trying to fix Blue Gene/Q build again.
The Blue Gene/Q build was broken by the renumbering/renaming of protocols libraries. This corrects that. No test changes expected.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2890 from RosettaCommons/sergey/f2
Updating profile Benchmark script so it honor compiler platform setting and run on local machines instead of HPC.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2887 from RosettaCommons/revert-2886-revert-2883-vmullig/rotamer_read_refactor
Re-merge refactor of the rotamer library reader.
This was originally merged as pull request #2883. When I tested that PR, the Intel icc build was broken; the fix to that unfortunately broke the gcc build, and I didn't catch that before merging. I reverted that merge, and will fix the issue here and re-merge.
Reverts RosettaCommons/main#2886 (i.e. this is a reversion of the reversion of the initial merge, reinstating the initial merge). This should be merged before merging #2855.
notify author
notify list [rosetta-logs@googlegroups.com]