Test: linux.clang.tensorflow.integration.tensorflow 「view this page in B3 βῆτα server」
Branch: master 「revision: №62046」
Test files: 「file-system-view」 「file-list-view」
Daemon: hojo-3     Started at: 2023-09-22 02:36:22     Run time: 0:59:00      State: failed

Failed sub-tests (click for more details):
esm_model_perplexity tensorflow_manager

{ "compared_with_test": { "full_name": "linux.clang.tensorflow.integration.tensorflow", "name": "integration.tensorflow", "platform": { "compiler": "clang", "extras": [ "tensorflow" ], "os": "linux" }, "platform_as_string": "linux.clang.tensorflow", "revision": { "branch": "master", "revision_id": 62045 }, "state": "passed", "test_id": 809234 }, "summary": { "failed": 2, "failed_tests": [ "esm_model_perplexity", "tensorflow_manager" ], "total": 21 }, "tests": { "PTMPrediction": { "log": "", "state": "passed" }, "abinitio_with_trRosetta": { "log": "", "state": "passed" }, "basic_gcn_tensorflow_test": { "log": "", "state": "passed" }, "database_md5": { "log": "", "state": "passed" }, "esm_model_perplexity": { "log": "Brief Diff:\nFiles /home/benchmark/working_dir/master:62045/esm_model_perplexity/command.tensorflow and /home/benchmark/working_dir/master:62046/esm_model_perplexity/command.tensorflow differ\r\nFiles /home/benchmark/working_dir/master:62045/esm_model_perplexity/command.tensorflow.sh and /home/benchmark/working_dir/master:62046/esm_model_perplexity/command.tensorflow.sh differ\r\n\n\nFull Diff:\ndiff -r '--exclude=command.sh' '--exclude=command.mpi.sh' '--exclude=observers' '--exclude=*.ignore' /home/benchmark/working_dir/master:62045/esm_model_perplexity/command.tensorflow /home/benchmark/working_dir/master:62046/esm_model_perplexity/command.tensorflow\r\n37a38\r\n> | grep -v \"Allocation of .* exceeds .* of system memory\" \\\r\ndiff -r '--exclude=command.sh' '--exclude=command.mpi.sh' '--exclude=observers' '--exclude=*.ignore' /home/benchmark/working_dir/master:62045/esm_model_perplexity/command.tensorflow.sh /home/benchmark/working_dir/master:62046/esm_model_perplexity/command.tensorflow.sh\r\n38a39\r\n> | grep -v \"Allocation of .* exceeds .* of system memory\" \\\r\n", "state": "failed" }, "smart_annealer": { "log": "", "state": "passed" }, "tensorflow_connection_test": { "log": "", "state": "passed" }, "tensorflow_manager": { "log": "Brief Diff:\nFiles /home/benchmark/working_dir/master:62045/tensorflow_manager/log and /home/benchmark/working_dir/master:62046/tensorflow_manager/log differ\r\nOnly in /home/benchmark/working_dir/master:62046/tensorflow_manager: ROSETTA_CRASH.log\r\nOnly in /home/benchmark/working_dir/master:62046/tensorflow_manager: .test_did_not_run.log\r\n\n\nFull Diff:\ndiff -r '--exclude=command.sh' '--exclude=command.mpi.sh' '--exclude=observers' '--exclude=*.ignore' /home/benchmark/working_dir/master:62045/tensorflow_manager/log /home/benchmark/working_dir/master:62046/tensorflow_manager/log\r\n62,74c62,96\r\n< apps.pilot.jackmaguire.tensorflow_manager_multi_input_test: Output from model: 0.5004\tExpected: 0.5004\tPASSED\r\n< apps.pilot.jackmaguire.tensorflow_manager_multi_input_test: Output from model: 0.40971\tExpected: 0.40971\tPASSED\r\n< apps.pilot.jackmaguire.tensorflow_manager_multi_input_test: Output from model: 0.4043\tExpected: 0.4043\tPASSED\r\n< apps.pilot.jackmaguire.tensorflow_manager_multi_input_test: Output from model: 0.33565\tExpected: 0.33565\tPASSED\r\n< apps.pilot.jackmaguire.tensorflow_manager_multi_input_test: Output from model: 0.71121\tExpected: 0.71121\tPASSED\r\n< apps.pilot.jackmaguire.tensorflow_manager_multi_input_test: Success!\r\n< basic.citation_manager.CitationManager: \r\n< The following UNPUBLISHED Rosetta modules were used during this run of Rosetta. Their authors should be included in the author list when this work is published:\r\n< \r\n< RosettaTensorflowManager Singleton's author(s):\r\n< Vikram K. Mulligan, Systems Biology, Center for Computational Biology, Flatiron Institute [vmulligan@flatironinstitute.org] (Created the RosettaTensorflowManager.)\r\n< Jack Magure, Menten AI [jack.maguire@menten.ai] (Expanded RosettaTensorflowManager capabilities for multi-head jobs and wrote tests.)\r\n< Sergey Lyskov, Gray Lab, Department of Chemical & Biomolecular Engineering, Johns Hopkins University [Sergey.Lyskov@jhu.edu] (Added testing infrastructure and helped to create the Rosetta-Tensorflow linked build.)\r\n---\r\n> \r\n> ERROR: MISMATCH!!!\tOriginal[2](1):\t-nan\tCombined(1+1):\t-nan\r\n> \r\n> \r\n> Error in RosettaTensorflowTensorContainer< T >::combine_tensors(): Failed to copy tensor properly! See error messages above.\r\n> ERROR:: Exit from: src/basic/tensorflow_manager/RosettaTensorflowTensorContainer.tmpl.hh line: 313\r\n> \r\n> [ ERROR ]: Caught exception:\r\n> \r\n> \r\n> File: src/basic/tensorflow_manager/RosettaTensorflowTensorContainer.tmpl.hh:313\r\n> [ ERROR ] UtilityExitException\r\n> ERROR: MISMATCH!!!\tOriginal[2](1):\t-nan\tCombined(1+1):\t-nan\r\n> \r\n> \r\n> Error in RosettaTensorflowTensorContainer< T >::combine_tensors(): Failed to copy tensor properly! See error messages above.\r\n> \r\n> \r\n> ------------------------ Begin developer's backtrace ------------------------- \r\n> BACKTRACE:\r\n> ROSETTA/source/build/src/debug/linux/3.10/64/x86/clang/3.4/tensorflow/libutility.so(backtrace_string(int)+0x41) [0x7f04cedd4d01]\r\n> ROSETTA/source/build/src/debug/linux/3.10/64/x86/clang/3.4/tensorflow/libutility.so(utility::excn::Exception::Exception(char const*, int, std::string const&)+0xb1) [0x7f04cee15501]\r\n> ROSETTA/source/build/src/debug/linux/3.10/64/x86/clang/3.4/tensorflow/libutility.so(utility::UtilityExitException::UtilityExitException(char const*, int, std::string const&)+0x7e) [0x7f04cedd864e]\r\n> ROSETTA/source/build/src/debug/linux/3.10/64/x86/clang/3.4/tensorflow/libutility.so(utility::exit(char const*, int, std::string const&, int)+0x56) [0x7f04cedd8246]\r\n> ROSETTA/source/bin/tensorflow_manager_multi_input_test.tensorflow.linuxclangdebug(basic::tensorflow_manager::RosettaTensorflowTensorContainer<float>::combine_tensors(utility::vector1<basic::tensorflow_manager::RosettaTensorflowTensorContainer<float>, std::allocator<basic::tensorflow_manager::RosettaTensorflowTensorContainer<float> > > const&, basic::tensorflow_manager::RosettaTensorflowTensorContainer<float>&)+0xd58) [0x426258]\r\n> ROSETTA/source/bin/tensorflow_manager_multi_input_test.tensorflow.linuxclangdebug() [0x4250ad]\r\n> ROSETTA/source/bin/tensorflow_manager_multi_input_test.tensorflow.linuxclangdebug() [0x4240fa]\r\n> ROSETTA/source/bin/tensorflow_manager_multi_input_test.tensorflow.linuxclangdebug() [0x421451]\r\n> ROSETTA/source/bin/tensorflow_manager_multi_input_test.tensorflow.linuxclangdebug() [0x422310]\r\n> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f04c166d555]\r\n> ROSETTA/source/bin/tensorflow_manager_multi_input_test.tensorflow.linuxclangdebug() [0x41dfb3]\r\n> ------------------------- End developer's backtrace -------------------------- \r\n> \r\n> \r\n> AN INTERNAL ERROR HAS OCCURED. PLEASE SEE THE CONTENTS OF ROSETTA_CRASH.log FOR DETAILS.\r\nOnly in /home/benchmark/working_dir/master:62046/tensorflow_manager: ROSETTA_CRASH.log\r\nOnly in /home/benchmark/working_dir/master:62046/tensorflow_manager: .test_did_not_run.log\r\n\nCompare(...): Marking as \"Script failed\" due to presense of .test_did_not_run.log or .test_got_timeout_kill.log file!\n", "state": "script failed" }, "tensorflow_simple_model_load_and_evaluate": { "log": "", "state": "passed" }, "trRosetta": { "log": "", "state": "passed" }, "trRosettaConstraintGenerator": { "log": "", "state": "passed" }, "trRosettaConstraintGenerator_rosettascripts": { "log": "", "state": "passed" }, "trRosettaProtocolMover": { "log": "", "state": "passed" }, "trRosettaProtocolMover_rosettascripts": { "log": "", "state": "passed" }, "trRosettaProtocolMover_rosettascripts_diskwrite": { "log": "", "state": "passed" }, "trRosettaProtocolMover_rosettascripts_diskwrite_only": { "log": "", "state": "passed" }, "trRosetta_test_predict": { "log": "", "state": "passed" }, "trRosetta_test_predict_ubiquitin": { "log": "", "state": "passed" }, "trRosetta_test_predict_ubiquitin_cst_file_write": { "log": "", "state": "passed" }, "trRosetta_test_predict_ubiquitin_cst_file_write_only": { "log": "", "state": "passed" }, "trRosetta_test_predict_ubiquitin_init_by_bins": { "log": "", "state": "passed" } } }