Merge pull request #2768 from RosettaCommons/aleaverfay/revive_resource_manager
merging branch aleaverfay/revive_resource_manager into master.
New implementation of the ResourceManager.
Instead of being this all seeing, all knowing piece of global data that knows what job is running and what resource you mean when you say "3mers", the new ResourceManager instead is a significantly more humble contraption that's responsible for loading data that should stay in memory between jobs and deallocate that memory when it's told it is no longer needed.
The ResourceManager has three components:
1. The ResourceManager itself,
2. A set of ResourceLocators that can open a stream from some sort of input source (e.g. a file, or a database query), and
3. A set of ResourceLoaders that are responsible for loading a kind of Resrouce (e.g. a PoseResourceLoader responsible for loading Poses* into memory)
A Resource, then, is a bitwise constant object that can be shared between concurrently running jobs. E.g., a loop-hash library, a fragment set, or a Pose*, or the SEWING graph. The ResourceManager only grants const access to the Resources it holds. A Resource is what you want to use when you have large, load-once data that is either expensive to create from disk, or occupies so much memory that you want to ensure there is only one of (but where different jobs might use different versions of it, so a Singleton would be inappropriate).
The ResourceManager is accessible currently through rosetta_scripts_jd3. There is a new top-level \<Resources\> block to the parser::protocol XML file to say "These are resources that will be used for this job" so that the RosettaScriptsJobQueen (RSJQ) will know to put those resources into a special const-only section of the DataMap before the Movers and Filters and whatnot are loaded. Movers who want to use resources will ask the DataMap for the resources they need by name during "parse_my_tag". The RSJQ will figure out at startup when each resource can be deallocated and will tell the ResourceManager "you may now deallocate resource X" at the appropriate time. (The use of script_vars will make it trivially easy to have different resources used for different jobs).
My eventual goal is to make abinitio work in RosettaScripts without the corrupting influence of global data (i.e. the sin against humanity of the frag_cache_ static data in core::fragment::FragmentIO).
I would like people to start thinking more carefully about the lifetime of the data that they load and to begin integrating the ResourceManager into their protocol development.
In the near future, I will be looking for more clients for the ResourceManager -- Movers and Filters that need load-once data -- and will be adding new attributes to these Movers and Filters to retrieve resources from the DataMap instead of loading them from scratch at each parse_my_tag.
I will also be adding the ResourceManager to the (JD2) rosetta_scripts application so that it can be used there as well. I will also have to figure out how to incorporate it into Jack's new MultistageRosettaScripts.
This PR will create quite a few cosmetic integration test changes. The old resource manager, though not directly used by anyone, was being indirectly used by lots of things, and the convoluted process of it's wake up did work that wasn't necessary and sent output to Tracers while doing so. I have also removed several of the integration tests for the old resource manager, and adapted several others to use the new resource manager. I paid close attention to all of the tests that were changing on my machine, fixing a handful of bugs that I had introduced along the way.
(The testing server is showing me far more trajectory changes than I see locally. They are also inconsistent from revision to revision, so I think there is just something funny about the testing server.)
*Pose itself is not a bitwise-constant class, and so instead of creating Pose objects and putting them into the ResourceManager, the PoseResourceLoader will create a wrapper class called a PoseResource which will give consumers a *copy* of the Pose that was loaded. This will hopefully be the outlier where the resource is loaded once, but where there might be multiple copies of the Resource in memory at one time.