Merge pull request #1813 from RosettaCommons/aleaverfay/xsd_xrw_merge_to_master
This is the big XSD XRW merge.
We have written complexTypes for all of the 800+ classes that are create-able through rosetta_scripts. These complexTypes, sewn together, form a schema describing valid input files for it. It serves two purposes. First, it ensures that your input scripts are on one level correct before any movers are created. Second, it provides a reminder to developers at the time that they add new code that they must document it, and enforces that reminder by preventing them from using their new code without doing so.
From here forward, when you add new options to your movers/filters/task operations/residue-level task operations/res filters//residue selectors/scoring grids/feature reporters/constraint generators/RDF functions/ligand areas/movemap definers/etc., you will have to update the schema.
A first-glance description of XML Schema, at least the way we use it, is that there are three things: there are elements (what we call tags), and there are complex types, which describe how the elements can be structured. Elements are described by complex types. Complex types can hold elements (i.e. sub-elements). The third thing: attributes (what we call options). When you say that "PackRotamers" can have an option named "taskoperations", the schema would say "the complex type 'mover_PackRotamers_type has an attribute 'taskoperations'" and elsewhere it would say "there is a subelement of the rosetta_scripts_MOVERS_type' that contains an element named 'PackRotamers', and its type is 'mover_PackRotamers_type.'"
```
<xs:complexType name="mover_PackRotamers_type">
<xs:attribute name="taskoperations" type="xs:string"/>
</xs:complexType>
</xs:complexType name="rosetta_scripts_MOVERS_type'>
<xs:choice>
<xs:element name="PackRotamers" type="mover_PackRotamers_type"/>
... other movers might go here
</xs:choice>
</xs:complexType>
```
The above example is a little hand wavy, but is intended to illustrate the complexType<-->element inter-relationship. You're responsibility as Mover/Filter developers is to define the complexType for your Mover/Filter. Fortunately, you now have 800 examples to learn from.
The code we developed here makes use of a set of classes in utility/tag/XMLSchemaGeneration.hh. These classes were designed to facilitate in-code creation of XML Schema. They are extensively documented. There is also the beginning of a wiki page describing XML Schema here.
About the pull request itself:
There are ~150 cosmetic and expected integration test changes. The only unexpected change is from the OrbitalsFeaturesReporter in the features integration test, which smells like an inlining difference (Some angles and distances that it measures and reports change ~10 digits past the decimal. None of the other features reporters show integration test changes; this is one of the few that involves a lot of code within the features reporter itself as opposed to it calling functions housed in other compilation units.)
Unit tests pass.
The 727 XML tests that were culled from the demos, integration tests, and the community at large all validate, after we went and threw out dozens of them that we could tell would not have worked in the current version of Rosetta, and after we commented out all but one of the LayerDesign operations, whose format is changing.
There is a really cool, really malleable python script that lives in the tools repository; it's in tools/xsd_xrw. It's named rewrite_rosetta_script.py. You give it your old XML with the --input flag and you give it the name of the file you want it to write to with the --output flag, and it turns your pre-XRW scripts into ones that should work post-XRW. You will definitely want to use this script.
Rocco has already begun extracting documentation from the schema output by rosetta_scripts (with the --output_schema flag) which is super cool.
This pull request also beautifies. If you hit a large number of merge conflicts, merge up until 76eb43e, carefully resolving any merge conflicts on the way, beautify your branch, and then merge the final commit of this PR (82349d7) with the assurance that your version should be taken at any point where you encounter a conflict. This is the same post-beautification merge process I outlined here but with different revisions.
On this Thanksgiving, I have to say I am extremely grateful for the hard work and camaraderie of the XRW team. Thank you Una, Sharon, Sam, Andy, Steven, Melanie, and Sebastian. It was a great week and I appreciate your having volunteered your time and energy.