Merge pull request #2913 from RosettaCommons/aleaverfay/layer_design_concise_logic
Aleaverfay/layer design concise logic
Per a discussion on slack over the difficulty in working with the layer-design residue selectors, I am creating some new code to let you combine residue selectors with boolean logic and pair those combinations with residue level task operations. This PR is not done, and I'm about to head out the door, but I wanted to give @sboyken and @cdbahl the heads up that this code is in progress.
Example new syntax below:
```
<RESIDUE_SELECTORS>
<Layer name="surface" select_core="false" select_boundary="false" select_surface="true" use_sidechain_neighbors="true"/>
<Layer name="boundary" select_core="false" select_boundary="true" select_surface="false" use_sidechain_neighbors="true"/>
<Layer name="core" select_core="true" select_boundary="false" select_surface="false" use_sidechain_neighbors="true"/>
<SecondaryStructure name="helix" overlap="0" minH="3" include_terminal_loops="false" use_dssp="true" ss="H" />
<SecondaryStructure name="sheet" overlap="0" minE="3" include_terminal_loops="false" use_dssp="true" ss="E" />
<SecondaryStructure name="loop" overlap="0" minH="3" minE="3" include_terminal_loops="true" use_dssp="true" ss="L" />
</RESIDUE_SELECTORS>
<TASKOPERATIONS>
<DesignRestrictions name="layer_design">
<Action selector_logic ="surface AND ( helix OR sheet) " aas="DEHKNQRST"/>
<Action selector_logic="surface AND loop" aas="DEGHKNPQRST"/>
<Action selector_logic="boundary AND helix" aas="ADEIKLMNQRSTVWY"/>
<Action selector_logic="boundary AND sheet" aas="DEFIKLNQRSTVWY"/>
<Action selector_logic="boundary AND loop" aas="ADEFGIKLMNPQRSTVWY"/>
<Action selector_logic="core AND helix" aas="AFILMVWY"/>
<Action selector_logic="core AND sheet" aas="FILVWY"/>
<Action selector_logic="core AND loop" aas="AFILMPVWY"/>
</DesignRestrictions>
</TASKOPERATIONS>
```
`!` has the highest priority
`AND` has the next highest priority
`OR` has the lowest priority
```
x AND ! y OR z
```
would convert into a boolean-function equivalent of
```
OR( AND( x, !y ), z )
```
Parentheses may be used to define alternate groupings:
```
x AND !( y or z )
```
would be equivalent to
```
AND( x, !( OR( y, z ) ) )
```
...
There is now a new top-level block for residue level task operations:
```
<RESIDUE_LEVEL_TASK_OPERATIONS>
```
that you can use in defining Actions in the DesignRestrictions task operation. These RLTOs can also be listed in the OperateOnResidueSubset task operation -- which now can take more than one RLTO (though it still takes exactly one ResidueSelector). The OperateOneResidueSubset task operation can *also* use the selector_logic attribute to construct a boolean combination of ResidueSelectors, as the DesignRestrictions TaskOp does.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2960 from RosettaCommons/JackMaguire/DuplicateHeaderGuardTest
We now have a unit test that checks for files that have duplicate header guards.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2861 from RosettaCommons/JackMaguire/icc_warnings2
Internal HBNet code is refactored. This should not affect the user or many classes outside of protocols/hbnet.
Also the header_using_check is updated. It now allows for `using X=Y` but still checks for any other instance of 'using'.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2956 from RosettaCommons/JackMaguire/StoreInterfaceAnalysisInPose
Making Interface Analyzer Mover Compatible With JD3 (very small PR)
The Interface Analyzer Mover currently can only report its findings to JD2. This PR changes the mover so that it stores its results in the pose so that the scores can survive in a JD3 environment.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2925 from RosettaCommons/jadolfbr/ab_design_optimize_interface
Jadolfbr/ab_design
Small tweaks to RAbD:
- Remove try/catch for sequence design. It should only be there for graft design.
- Fail if we have caught exceptions the whole time. I had some issues on a cluster that was being hammered hardcore with IO operations of another user and my runs still completed and output a file. We should never have that.
- Improve the speed at which we load structures from the database through a pre-made text file. This is a temporary solution until we have updated databases in raw txt form in the rosetta database. However, this significantly speeds up runs and is up-to-date with the current PyIgClassify db.
Etc:
- Throw exception in ProteinResidueFeatures instead of a utility_exit_with_message.
- Add option, `-read_only_ATOM_entries` to literally only read ATOM records in pdb_reader and cif_reader.
Vikram K. Mulligan 7 years Curious -- I got e-mails about a bunch of my integration tests changing with this commit, but looking at the integration test log, everything's fine. Was there a testing server hiccup of some sort, Sergey?[list]
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2948 from RosettaCommons/vmullig/fix_bluegene_crash_bug
Fix spiky memory usage in multi-threaded mode causing crashes on Blue Gene/Q system
The current implementation of the threadsafe lazy loading errs to the side of avoiding long locks, but thereby allows threads to waste effort concurrently loading large objects from disk that result in temporary spikes in memory usage (even though only one copy of the object is permanently stored). The old scheme for loading rotamer libraries looked like this:
- obtain read lock
- check for existence of library, and return it if it exists, releasing read lock.
- release read lock
- read library from disk (creating memory object) <-- Many threads might do this at once.
- obtain write lock
- add library to map if it isn't in there already <-- Only one thread does this. For other threads, the memory object that was created is eventually discarded.
- release write lock
- obtain read lock
- return pointer stored in map, releasing read lock
This meant that write-locking (which lets only one thread access the object) was as short as possible, but many threads might read a rotamer library from disk and set it up in memory before the first thread obtained its write-lock and added the created rotamer library to the map of rotamer libraries. This pull request revises this slightly, so that it now looks like this:
- obtain read lock
- check for existence of library, and return it if it exists, releasing read lock
- release read lock
- obtain write lock
- check again for existence of library in map, and return it if it exists, releasing write lock
- read library from disk (creating memory object) <-- Only one thread does this, now.
- add library to map
- release write lock
- obtain read lock
- return pointer stored in map, releasing read lock
This means that more threads might be sitting idle for slightly longer, while reads from disk are taking place, but that every read from disk is a productive one, and every memory object created temporarily is actually stored and used. It avoids big spikes in memory usage from transiently-duplicated memory objects. These spikes were causing crashes on limited-memory systems like the Blue Gene/Q system.
Tasks:
- [x] Fix the problem.
- [x] Confirm that rotamer libraries are only loaded once (by launching 40 threads on Jojo simultaneously and measuring the number of reads of the OU3_TRP library).
- [x] Confirm that this addresses the memory spike and crash issue on the Blue Gene/Q system.
- [x] Check whether this has any significant impact on multithreaded performance on Blue Gene/Q. (It ought not to. Initialization might be very slightly slower but with reduced disk access and fewer memory spikes; runs after lazily-loaded objects are loaded should be unaffected.)
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2936 from RosettaCommons/yhsia/MergePDBMover_no_design_option
new options for MergePDBMover; some edits to ResidueIndexSelector (symmetry reverse)
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2935 from RosettaCommons/vmullig/disable_rotamer_sorting
Adding a means of skipping the rotamer well order correction.
So it turns out that, now that I've implemented the Voronoi-based rotamer well interpolation for noncanonicals, it probably doesn't actually matter what order rotamer wells are in. The sorting step, which takes a while for large rotamer libraries, can therefore be skipped. I'm keeping the option to turn it back on in case there are unanticipated cases in which rotamer well order matters, but setting the default behaviour to be no sorting.
notify author
notify list [rosetta-logs@googlegroups.com]
Merge pull request #2934 from RosettaCommons/vmullig/disable_jd2
Adding a means for non-JD2 apps and protocols to avoid instantiating a JD2 JobDistributor
I noticed that the `simple_cycpep_predict` application, which uses its own job distributor that is not related to JD2 or JD3, was unnecessarily instantiating the JD2 job distributor (which was creating some problems in special cases -- large nstruct, for example). I traced this to the `ParsedProtocol` that I instantiate inside of `SimpleCycpepPredictApplication`, which invokes `protocol::jd2::jd2_used()` at several points, which determines whether JD2 is used by instantiating a JD2 job distributor and interrogating it.
To address this, this pull request adds an option to the `ParsedProtocol` constructor to disable JD2 access. This option is false by default, so that normal `ParsedProtocol` behaviour is unaffected except when JD2 access is explicitly disabled.
EDIT: At Rocco's suggestion, I abandoned the original approach and added a means of checking whether JD2's JobDistributor has already been instantiated, called in `jd2_used()`.
notify author
notify list [rosetta-logs@googlegroups.com]