Dedication to quality

On Abyss, ensuring quality is an ongoing, continuous effort. The Abyss team prioritizes quality in every stage of development, with dedicated Quality Engineers and Developers ensuring that high standards are consistently met.

Testing with AI

Testing with AI continues to be a fast growing field in software quality assurance. With the emergence of AI-driven development and testing tools, development and testing roles are becoming increasingly review-based of AI code generation and output, rather than traditional development and testing processes, requiring a shift in how quality assurance is approached.

• As AI generated development code becomes more prevalent, testing strategies and assurance of quality need to adapt accordingly
• Enhanced test case generation using AI-driven insights
• Experimenting with AI-driven testing tools and methodologies to improve and strengthen software quality.
• Playwright SKILL aids in Playwright test automation, keeping tests structured and maintainable to our high standards.
• Testing of LLM prompts and expected replies using Promptfoo

Automation testing

Automation testing of Abyss components is a top priority. Currently, our automation tests consist of the following:

Web

• Functional and Visual Testing with Playwright
  • Functionality components tests, such as click, drag, type, text compare, etc.
  • Playwright SKILL aids in Playwright test automation, keeping tests structured and maintainable to our high standards.
    • Playwright Skill Documentation
  • Visual testing integrated with the functionality tests.
    • Visual Test Case Example
  • A11y Deque Axe Integration.
    • Scanner integrated with existing Abyss Playwright framework to validate and flag potential accessibility issues.
    • A11y Scan Example

Mobile

• Functional Testing with Appium
  • Mobile test automation framework for the Abyss component native app.
  • Appium Tests in GitHub

Unit testing

• Jest Unit Tests for Abyss Projects
• Abyss Web Unit Tests
• Abyss Mobile Native Unit Tests
• Abyss CLI Unit Tests
• Abyss Shared Unit Tests

Miscellaneous

• LLM Eval Testing with Promptfoo
• Abyss Document Web Crawler

Manual testing

While automation testing helps provide strong coverage of existing components and regression testing, manual testing is also performed during sprints, before a release goes to production. To better understand the Abyss testing process, see the QE Review Task Process Flowchart

FAQ

How can I report an issue/bug/defect with an Abyss component?

1. Go to the Contact Us page.
2. Scroll to the form at the bottom of the page.
3. Select the Abyss Product you are reporting the issue for.
4. Under Request Type, select the “Bug” option that best fits the issue.
5. Fill out the rest of the form fields as accurately as possible.
6. In the description, include as many details as possible, such as clear and concise steps to reproduce the issue.
   • Include information such as your browser, browser version, mobile device, operating system, etc. This helps the Abyss team best troubleshoot the issue.
7. Add any helpful attachments: screenshots, videos, etc.
8. Click the Submit button and wait for the success message.
9. The Abyss Team will review the issue and plan next steps.