The present invention extends to methods, systems, and computer program products for isolating the evaluation of actual test results against expected test results from the test module that generates the actual test results. A results evaluation sub-system receives actual test results resulting from the execution of a test in a test environment. A results retrieval sub-system receives environmental data indicating that a test was performed in the test environment. The results retrieval sub-system selects one or more expected results from a results database based on the received environmental data. The results retrieval sub-system sends the selected one or more expected results to the results evaluation sub-system. The results evaluation sub-system receives the one or more selected results. The results evaluation sub-system evaluates the actual test results against the one or more expected results to determine if the test was successful.
Systems, methods, and/or techniques ("tools") that relate to an architecture for automated detection and analysis of security issues are described herein. One aspect of the tools provides a watchdog application that generates fizzing signals that enable or disable fuzzing of test cases that are to be received by the watchdog application. The watchdog application also receives notifications of test cases that are to be executed on a system under test with which the watchdog application is associated. Finally, the watchdog application monitors the system under test when it is executing the test case.
