Test and Evaluation Approach

Requirements decomposition drives:

  • Component Test Cases
  • Integration Test Threads
  • Functional Test Threads

Agile development process with continuous feedback from the test results. Test cases and test threads captured in a Test Management System.

  • Provide traceability to requirements and development activities
  • Capture details on test setup, procedures, and pass/fail criteria
  • Can be linked to specific test events
  • TMS example includes Redcase for Redmine


I&T Continuous Integration

Continuous integration (CI) tools used to automate builds and testing:

  • Provide rapid feedback to developers based on latest changes
  • Regression testing to maintain existing functionality
  • Tracking of test results over time
  • Reduces risk of technology insertion
  • Verifies component functionality before integration

DAS experience with CI Tools:

Jenkins CI, Bamboo, Sikuli GUI automation, Selenium Web Driver



I&T Orchestration Tools

Orchestration Tools Capabilities

  • Maintains proper test configuration and operation
  • Enables consistent and well documented test environment
  • Ensures test repeat-ability
  • Properly collects all test artifacts

Test Configuration

  • Chef, Puppet, Salt's tools available
  • Automate system configuration and update
  • Provides means for strong configuration via version control

Test Execution

  • Sikuli and Selenium can be used to drive GUI and web applications
  • EASE is capable of orchestrating component configuration, startup, shutdown, and artifact collection
  • Particularly useful for Integration Events


Incremental Test Events

Integration Events and Functional Verification

  • Incrementally test integration of system
  • Isolated component test cases for entry in IE
  • Orchestration tools to set up consistent testing
  • Test automation incorporated

System Measurement Performance

  • Capture system performance metrics under load to identify bottlenecks
  • Utilize system agents and network statistics
  • Cisco NetFlow and Performance Monitor
  • Apache ActiveMQ statistics