Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://docs.gitlab.com/development/testing_guide/testing_ai_features/ below:

Testing AI features | GitLab Docs

Help us learn about your current experience with the documentation.

.

This document highlights AI-specific testing considerations that complement GitLab standard testing guidelines. It focuses on the challenges AI features bring to testing, such as non-deterministic responses from third-party providers. Examples are included for each testing level.

AI-powered features depend on system components outside the GitLab monolith, such as the AI Gateway and IDE extensions. In addition to these guidelines, consult any testing guidelines documented in each component project.

Follow standard unit testing guidelines. For AI features, always mock third-party AI provider calls to ensure fast, reliable tests.

• GitLab: ee/spec/lib/code_suggestions/tasks/code_completion_spec.rb
• VS Code extension: code_suggestions/code_suggestions.test.ts

Use integration tests to verify request construction and response handling for AI providers. Mock AI provider responses to ensure predictable, fast tests that handle various responses, errors, and status codes.

• GitLab: ee/spec/requests/api/code_suggestions_spec.rb
• VS Code extension: main/test/integration/chat.test.js

Use frontend feature tests to validate AI features from an end-user perspective. Mock AI providers to maintain speed and reliability. Focus on happy paths with selective negative path testing for high-risk scenarios.

• GitLab Duo Chat: ee/spec/features/duo_chat_spec.rb

Use end-to-end tests sparingly to verify AI features work with real provider responses. Key considerations:

• Keep tests minimal due to slower execution and potential provider outages.
• Account for non-deterministic AI responses in test design. For example, use deterministic assertions on controlled elements like chatbot names, not AI-generated content.

• GitLab: specs/features/ee/browser_ui/3_create/web_ide/code_suggestions_in_web_ide_spec.rb
• JetBrains: test/kotlin/com/gitlab/plugin/e2eTest/tests/CodeSuggestionTest.kt

• GitLab.com: We run minimal E2E tests continuously against staging and production environments. For example, Code Suggestions smoke tests.
• GitLab Self-Managed: We use the gitlab-qa orchestrator with AI Gateway scenarios to test AI features on GitLab Self-Managed instances.

Perform exploratory testing before significant milestones to uncover bugs outside expected workflows and UX issues. This is especially important for AI features as they progress through experiment, beta, and GA phases.

We dogfood everything. This is especially important for AI features given the rapidly changing nature of the field. See the dogfooding process for details.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4