Skip to main content

ADR-019: Consumer Engineering Quality/Testing Best Practices

  • Status: Proposed
  • Deciders: Web and Mobile
  • Author: Chris McCue, Sr. Quality Engineer

Context​

The Consumer Engineering department at UnitedHealth Group develops a variety of consumer-focused healthcare products, including native mobile apps and websites for UnitedHealthCare and Optum. With teams working on diverse applications, there is a need to standardize quality engineering and testing practices. This ADR aims to unify these practices, enhance efficiency, streamline testing efforts, and promote consistency. It is being submitted to the Abyss teams for initial review and feedback.

Decision​

To implement standardized quality engineering/testing practices across Consumer Engineering, the following tools, frameworks, and guidelines are recommended:

Testing Tools and Frameworks​

Unit and Snapshot Testing​

End-to-End Testing​

  • Web: Playwright (primary), alternatives: Cypress, Selenium WebDriver
  • Mobile: Appium (primary), alternative: Detox
    • Appium is preferred for both mobile web and native apps due to its extensive documentation and versatility.

API Testing​

  • Abyss does not require API tests but additional research is needed for broader Consumer Engineering projects to identify suitable tools based on current usage.

Component Testing​

  • Web: Playwright (recommended by Abyss for its ability to mount components in isolation or for integration testing)
  • Mobile: Appium
    • Best practices include creating a component library app and testing native components within a native app instead of a web browser.

Note: Tools should be selected based on project needs, but the recommended options should be prioritized, with justification provided if alternatives are chosen.

Bot/Conversational AI Testing​

  • Botium
    • Currently not in use by Abyss, however the Botium testing tool looks promising for future consideration.

Test Coverage​

  • Goal: Aim for high test coverage—near 100% for unit tests and test cases in automated tests—while focusing on meaningful and effective testing.
  • Snapshot Tests: Ensure UI changes are intentional and regressions are prevented. Regularly update snapshots to reflect intentional changes.
  • Edge Cases: Add tests as needed to improve coverage and ensure critical edge cases are addressed.
  • Acknowledge: 100% coverage is not always possible on a project, nor is there a guarantee that calling your coverage "100% coverage" will cover all code paths.

Automation and Metrics​

  • Automate as many tests as possible for faster cycles and broader coverage.
  • Integrate code coverage tools into CI/CD pipelines to monitor test coverage and identify gaps.
  • Maintain a test plan document to benchmark accurate metrics and measure coverage (e.g., document test cases to create and track automation progress).

Testing Focus: Timeline and Environment​

  • Prioritize early testing throughout the Software Development Life Cycle (SDLC) to identify and address issues as soon as possible. This proactive approach reduces costly fixes later in the development process.
  • Encourage developers to adopt Test Driven Development (TDD) as a fundamental practice. By writing tests before the code, developers can clarify requirements, enhance code quality, and catch issues early, leading to more efficient and reliable development cycles.
  • Conduct the majority of testing in non-production environments to maximize the ability to detect and resolve issues without the risks associated with production settings. This allows for more comprehensive testing scenarios and faster feedback loops.
  • While production environments should have a baseline of tests to ensure stability, testing may be limited due to security and integration challenges.
  • It's essential to balance the need for stability with the necessity of maintaining a robust testing strategy.

Recommendations​

  1. Unit Tests: Focus on core business logic and critical functions. Regularly update tests to match codebase changes.
  2. Snapshot Tests: Use to detect unintended UI changes, and ensure snapshots reflect intended updates.
  3. Automation: Incorporate CI/CD pipelines for automated testing on every code commit.
  4. Test Plans: Document and track common test cases, using test plans as benchmarks for automation and coverage metrics.
  5. Test Timeline: Promote testing throughout the entire Software process, with emphasis on early testing to identify issues as early as possible.

Impact​

Adopting these standardized practices will:

  • Increase testing efficiency.
  • Enhance the reliability, stability, and quality of consumer products.
  • Foster consistency and collaboration across teams.
  • Accelerate defect identification and resolution.

Implementation Plan​

  1. Review and Feedback: Collaborators to review the draft and provide feedback.
  2. Finalization: Finalize the ADR based on feedback and begin implementation.

Collaborators​

  • Identify key stakeholders and collaborators from each team to review the ADR and provide input.
  • Ensure cross-team representation to address specific needs and concerns.

Communication and Training​

  • Conduct training sessions/meetings to familiarize teams with agreed-upon testing tools and practices.
  • Provide documentation and resources as needed.

References and Supplementary Documentation​

For detailed information on tool setup, usage, and the testing process lifecycle, refer to the following resources:

Unit and Snapshot Testing with Jest​

End-to-End Testing​

Component Testing with Playwright​

Bot/Conversational AI Testing with Botium​

CI/CD Pipeline Starting Guide​

These documents provide comprehensive guidance on the setup, usage, and best practices for each noted testing tool and framework.

Monitoring and Evaluation​

  • Monitor the adoption and effectiveness of the new practices.
  • Collect feedback from teams to understand what is and isn't working, in order to make iterative improvements.

By standardizing quality engineering and testing practices, we aim to create a cohesive and efficient testing environment that supports the delivery of high-quality consumer products.

Table of Contents