AI project acceptance adds an 'effectiveness' dimension beyond traditional software—100% functional pass is not enough; core scenario accuracy must be ≥95%, hallucination rate ≤3%, P99 latency ≤5s, and security must pass 100%. According to the China Academy of Information and Communications Technology's '2025 AI Application Development Report', the lack of systematic acceptance criteria is the primary cause of AI project disputes. This article offers a complete four-dimensional acceptance criteria template: functionality, performance, security, and effectiveness, making acceptance evidence-based.
How to Pass Functional Acceptance?
Basic Functional Acceptance
| Acceptance Item | Acceptance Criterion | Test Method |
|---|---|---|
| All features implemented | 100% of contracted features implemented | Verify item by item against the functional test checklist |
| Authorization controls effective | Different roles see different content | Multi-role testing |
| Data flow normal | Data correctly synchronized across systems | End-to-end process testing |
| Exception handling correct | Prompts and fallback mechanisms for exceptions | Exception scenario testing |
AI-Specific Functional Acceptance
| Acceptance Item | Acceptance Criterion | Test Method |
|---|---|---|
| Intent recognition | Core intent recognition accuracy ≥90% | Verify with 200+ test cases |
| Knowledge retrieval | Recall@10 ≥85% | Evaluate on standard test set |
| Answer generation | Answer accuracy ≥85% | Human annotation of 100+ real questions |
| Human handoff | Seamless handoff process with complete context | Simulate low-confidence scenarios |
What Are the Performance Acceptance Criteria?
| Metric | Standard Value | Test Condition |
|---|---|---|
| Average response time | ≤2s | Normal load |
| P99 response time | ≤5s | Normal load |
| Peak throughput | ≥ contracted value | Stress test |
| System availability | ≥99.9% | 7-day continuous run |
| GPU memory usage | ≤ contracted value | Sustained running |
| Concurrency support | ≥ contracted concurrency | Concurrency test |
What Are the Mandatory Security Acceptance Checks?
Data Security
| Acceptance Item | Standard | Test Method |
|---|---|---|
| Data transmission encryption | TLS 1.2+ | Verify via packet capture |
| Data storage encryption | AES-256 | Configuration check |
| Sensitive data masking | ID numbers/phone numbers/bank card numbers | 100+ test cases |
| Access control | RBAC + document-level permissions | Unauthorized access testing |
AI Security
| Acceptance Item | Standard | Test Method |
|---|---|---|
| Prompt injection defense | Malicious commands are not executed | 50+ injection attack tests |
| Hallucination control | Core scenario hallucination rate ≤5% | Human annotation verification |
| Output filtering | Non-compliant content is not output | Sensitive word + non-compliant content testing |
| Operation audit | All critical operations are logged | Log integrity check |
Security Test Checklist
How to Evaluate Effectiveness Acceptance? This Is a Dimension Unique to AI Projects
Effectiveness Indicators
| Scenario | Accuracy Target | Hallucination Rate Target |
|---|---|---|
| Core scenarios | ≥95% | ≤3% |
| General scenarios | ≥85% | ≤10% |
| Edge scenarios | "I don't know" allowed | — |
Effectiveness Test Methods
| Method | Sample Size | Performed By |
|---|---|---|
| Automated evaluation | 500+ items | Technical team |
| Human-annotated evaluation | 100+ items | Business team |
| Real user testing | 50+ users | Target users |
| A/B comparison | Compare with old system | Operations team |
Effectiveness Decay Testing
Run continuously for 7 days; accuracy fluctuations must not exceed ±3%.
What Does Documentation Acceptance Include?
| Document Type | Must Include |
|---|---|
| Operation manual | User operation steps, screenshots, FAQs |
| O&M manual | System architecture, deployment steps, monitoring metrics, emergency plans |
| API documentation | Interface descriptions, request/response examples, error codes |
| Training materials | Training PPT, video tutorials, assessment questions |
| Knowledge base management | Document update process, templates, quality standards |
How Is the Acceptance Process Conducted?
```
Pre-acceptance (Internal) → Fix Issues → Formal Acceptance (Client Participation)
↓
Functional Acceptance → Performance Acceptance → Security Acceptance → Effectiveness Acceptance → Documentation Acceptance
↓
Acceptance Report → Outstanding Issues List → Rectification within Deadline → Go-Live
```
Acceptance Pass Criteria
Frequently Asked Questions
What Is the Difference Between 'Effectiveness Acceptance' in AI Projects and 'Functional Testing' in Traditional Software?
Functional testing in traditional software is binary—a feature either exists or not, and outcomes are predictable. Effectiveness acceptance in AI projects is probabilistic—the same input may produce different outputs, with accuracy being 95% rather than 100%. Therefore, AI projects must agree on 'accuracy thresholds' and 'test sample sizes,' rather than simply checking whether a feature is implemented.
How Are 'Core Scenarios' and 'General Scenarios' Defined in Effectiveness Acceptance?
Core scenarios are those that directly impact business revenue or compliance (e.g., risk control approval, compliance queries), with an accuracy target of ≥95%. General scenarios are supplementary (e.g., product recommendations, usage guidance), where ≥85% accuracy is sufficient. It is recommended to agree in writing on scenario classification and corresponding standards with the business side at project initiation.
What Should Be Done If Effectiveness Declines After AI Project Acceptance?
During acceptance, an 'effectiveness warranty period' should be specified—usually 3–6 months. Within this period, if accuracy drops by more than 5%, the vendor should optimize for free. Common causes of decline include outdated knowledge bases, changing business rules, and data distribution drift. The acceptance documentation should include periodic optimization mechanisms and responsibility assignment.
Looking to establish AI project acceptance criteria? Book a Free Acceptance Consultation