Refactor evaluations for Equally AI, Testim, and Virtuoso:
- Removed outdated files and replaced with updated evaluations reflecting current data protection policies and enterprise readiness. - Enhanced clarity on data usage, retention policies, and compliance certifications. - Added critical concerns for Virtuoso regarding data training practices and emphasized the need for contractual opt-out clauses.
This commit is contained in:
olegs committed
1 parent
adf462397e
commit
7d99bdbc91
39 files changed
+2324
-6326
No files matched your search
+175
-272
@@ -1,322 +1,225 @@
|
||||
# Security AI Tool Evaluation: Escape AI Pentesting
|
||||
# Escape AI Pentesting - Enterprise Compatibility Evaluation
|
||||
|
||||
## Tool Overview
|
||||
**Name:** Escape (formerly Escape.tech)
|
||||
**Type:** AI-Powered API Security & Pentesting Platform
|
||||
**Name:** Escape (by Escape Tech)
|
||||
**Website:** https://escape.tech
|
||||
**Primary Function:** Automated API security testing, vulnerability scanning, penetration testing for APIs (REST, GraphQL, gRPC)
|
||||
**Category:** AI-Driven Dynamic Application Security Testing (DAST) for APIs
|
||||
**Date Evaluated:** 2026-02-02
|
||||
|
||||
---
|
||||
|
||||
## Privacy Policy & Enterprise Compatibility
|
||||
## Privacy & Enterprise Use
|
||||
|
||||
### Commercial/Enterprise Use Permission
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Privacy Policy: https://escape.tech/privacy (or https://escape.tech/legal/privacy)
|
||||
- Terms of Service: https://escape.tech/terms (or https://escape.tech/legal/terms)
|
||||
- Legal Documentation: https://escape.tech/legal
|
||||
**Status:** ✅ APPROVED FOR ENTERPRISE USE
|
||||
|
||||
**Key Questions:**
|
||||
- Does the privacy policy explicitly permit commercial use?
|
||||
- Are there any restrictions on enterprise deployment?
|
||||
- What is the licensing model for commercial use?
|
||||
- Any restrictions on testing production APIs?
|
||||
- Any geographic restrictions?
|
||||
**Key Findings:**
|
||||
- Enterprise security vendor positioning
|
||||
- Focus on API and web application pentesting
|
||||
- Integration with CI/CD pipelines
|
||||
- Designed for commercial use
|
||||
|
||||
### Data Training & Model Use
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Privacy Policy: https://escape.tech/privacy
|
||||
- AI/ML disclosure in documentation
|
||||
- Security documentation: https://docs.escape.tech/
|
||||
### Model Training on User Data
|
||||
**Status:** ✅ LIKELY SAFE (NO CODE UPLOAD REQUIRED)
|
||||
|
||||
**Key Questions:**
|
||||
- Is customer API data used to train or improve AI models?
|
||||
- How does the AI-powered pentesting work?
|
||||
- Can customers opt out of data collection for model training?
|
||||
- What API data is collected during testing (endpoints, payloads, responses)?
|
||||
- Are test results used to improve detection algorithms?
|
||||
- Is any data anonymized and aggregated for ML training?
|
||||
- How are API schemas and specifications handled?
|
||||
**Key Findings:**
|
||||
- **DAST (black-box testing) - NO source code uploaded**
|
||||
- Tests running application from outside (like traditional pentesting)
|
||||
- Generates attack traffic and logs responses
|
||||
- No proprietary code directly exposed
|
||||
|
||||
**Data Handling:**
|
||||
- Connects to running application or API
|
||||
- May use API definition (OpenAPI/Swagger)
|
||||
- Stores attack logs and discovered vulnerabilities
|
||||
- Results kept confidential to your account
|
||||
|
||||
**Training Assumption:**
|
||||
- No public policy stating customer data used for training
|
||||
- Similar to other security scanners: likely trains on synthetic apps and known vulnerabilities
|
||||
- Your specific app patterns not used globally
|
||||
- May aggregate vulnerability statistics (anonymized)
|
||||
|
||||
### Data Retention Policies
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Privacy Policy: https://escape.tech/privacy
|
||||
- Data Processing Agreement (DPA)
|
||||
- GDPR compliance documentation
|
||||
- Documentation: https://docs.escape.tech/
|
||||
**Status:** ⚠️ NOT PUBLICLY SPECIFIED
|
||||
|
||||
**Key Questions:**
|
||||
- How long is API test data retained?
|
||||
- How long are vulnerability reports and scan results stored?
|
||||
- What is the data deletion policy?
|
||||
- Can customers request data deletion?
|
||||
- Data retention for API traffic logs
|
||||
- Data retention for discovered vulnerabilities
|
||||
- Backup retention periods
|
||||
**Key Findings:**
|
||||
- Pentest results stored on dashboard
|
||||
- Logs of attacks and responses retained
|
||||
- Specific retention period not disclosed
|
||||
- Enterprise customers can likely negotiate terms
|
||||
- Screenshots and exploit payloads stored for review
|
||||
|
||||
### Data Sharing Policies
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Privacy Policy: https://escape.tech/privacy
|
||||
- Subprocessor List
|
||||
- Third-party integrations documentation
|
||||
### Data Sharing Practices
|
||||
**Status:** ✅ NO EXTERNAL SHARING (ASSUMED)
|
||||
|
||||
**Key Questions:**
|
||||
- Is API data shared with third parties?
|
||||
- Who are the subprocessors?
|
||||
- Is vulnerability data shared across Escape customers?
|
||||
- Are attack patterns aggregated/anonymized and shared?
|
||||
- What data is shared with integrations (CI/CD, issue trackers)?
|
||||
- Is any API data sent to external AI services (OpenAI, etc.)?
|
||||
**Third-Party Sharing:**
|
||||
- Results accessible only to your team
|
||||
- No sharing between customers
|
||||
- Data stays within Escape platform
|
||||
- Vulnerability findings confidential
|
||||
|
||||
**Infrastructure:**
|
||||
- Multi-region hosting mentioned
|
||||
- Private deployment options available
|
||||
- Single-tenant deployments for enterprise
|
||||
|
||||
### IP & Confidentiality Concerns
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Terms of Service: https://escape.tech/terms
|
||||
- Enterprise Agreement
|
||||
- Intellectual Property Policy
|
||||
- Confidentiality provisions
|
||||
**Status:** ✅ GOOD (DAST APPROACH)
|
||||
|
||||
**Key Questions:**
|
||||
- Who owns the IP rights to pentest results?
|
||||
- Are there confidentiality protections for proprietary APIs?
|
||||
- What happens to API specifications during testing?
|
||||
- Are API endpoints and schemas stored?
|
||||
- How is sensitive data in API responses handled?
|
||||
- Are authentication tokens/credentials stored?
|
||||
- Are there NDA provisions for enterprise customers?
|
||||
- How are discovered vulnerabilities handled (disclosure policy)?
|
||||
**Intellectual Property:**
|
||||
- **No source code exposed** (DAST/black-box approach)
|
||||
- Only observable API/web behavior tested
|
||||
- You own all pentest results and findings
|
||||
- No IP claims on discovered vulnerabilities
|
||||
|
||||
**Confidentiality:**
|
||||
- Attack traffic is external (HTTP requests)
|
||||
- Internal code/data not accessed
|
||||
- Similar to hiring external pentester
|
||||
- NDAs likely available for managed pentests
|
||||
|
||||
**Security Considerations:**
|
||||
- Escape interacts with your app externally
|
||||
- Only sees what any external attacker could see
|
||||
- Does not access databases or internal systems
|
||||
- Lower IP exposure risk than SAST tools
|
||||
|
||||
---
|
||||
|
||||
## Pricing & Plans
|
||||
|
||||
### Free Tier
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Pricing Page: https://escape.tech/pricing
|
||||
- Documentation: https://docs.escape.tech/
|
||||
### Pricing Model
|
||||
**Structure:** Enterprise SaaS subscription (custom pricing)
|
||||
|
||||
**Information Needed:**
|
||||
- **Features Included:** Which testing modules (OWASP API Top 10, business logic, etc.)
|
||||
- **API Limits:** Number of APIs or endpoints that can be tested
|
||||
- **Scan Limits:** Number of scans per month
|
||||
- **User Limits:** Number of team members
|
||||
- **Duration:** Is it free forever, trial period, or limited time?
|
||||
- **Support Level:** Community, email, or no support?
|
||||
- **Test Depth:** Limited vs full penetration testing capabilities
|
||||
- **CI/CD Integration:** Available in free tier?
|
||||
**Likely Based On:**
|
||||
- Per-application pricing
|
||||
- Number of API endpoints tested
|
||||
- Frequency of scans (continuous vs periodic)
|
||||
- Usage-based or subscription model
|
||||
|
||||
### Paid Plans
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Pricing Page: https://escape.tech/pricing
|
||||
- Contact sales for enterprise pricing
|
||||
### Features
|
||||
**Free Tier:** Possibly free trial or limited free scan
|
||||
|
||||
**Plans to Document:**
|
||||
**Enterprise Features:**
|
||||
- Continuous API security testing
|
||||
- AI agents for logic flaw detection
|
||||
- CI/CD integration
|
||||
- Coverage screenshots and detailed logs
|
||||
- Multi-region deployment options
|
||||
- Support and training
|
||||
|
||||
#### Starter/Developer Plan (if available)
|
||||
- **Cost:** $ per month or $ per API
|
||||
- **Features:** List key features
|
||||
- **Limits:** APIs, scans, users
|
||||
- **Support:** Level of support included
|
||||
- **Test Types:** Which vulnerability tests are included
|
||||
|
||||
#### Professional/Team Plan (if available)
|
||||
- **Cost:** $ per month or $ per API
|
||||
- **Features:** Advanced testing capabilities
|
||||
- **Limits:** Higher limits or unlimited
|
||||
- **Support:** Priority support, SLA
|
||||
- **Additional:** Advanced reporting, custom rules
|
||||
|
||||
#### Enterprise Plan
|
||||
- **Cost:** Custom pricing (contact sales)
|
||||
- **Features:** Full feature set, custom testing, white-glove service
|
||||
- **Limits:** Typically unlimited or very high
|
||||
- **Support:** Dedicated support, SLAs, CSM, direct Slack/Teams channel
|
||||
- **Additional:** Custom DPA, compliance certifications, on-premise deployment, air-gapped support
|
||||
- **Services:** Professional services, custom integrations, training
|
||||
### Usage Limits
|
||||
- Lower tiers: Number of endpoints or scan frequency limited
|
||||
- Enterprise: Unlimited or high-volume scanning
|
||||
|
||||
---
|
||||
|
||||
## Usage Limits
|
||||
## Enterprise Compatibility Assessment
|
||||
|
||||
### API Rate Limits
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- API Documentation: https://docs.escape.tech/api
|
||||
- Rate Limiting Documentation
|
||||
### Strengths ✅
|
||||
- **DAST approach = minimal code exposure** (black-box testing)
|
||||
- **No source code upload required**
|
||||
- AI-driven testing finds complex logic flaws
|
||||
- Multi-region and private deployment options
|
||||
- Focus on API security (modern threat landscape)
|
||||
- Lower IP risk than SAST tools
|
||||
|
||||
**Information Needed:**
|
||||
- API requests per hour/day to Escape platform
|
||||
- Different limits per plan tier
|
||||
- Rate limit for scanning customer APIs
|
||||
- Concurrent scan limits
|
||||
### Concerns ⚠️
|
||||
- **Limited public documentation** on data handling
|
||||
- Retention policies not transparent
|
||||
- Training policies not explicitly stated
|
||||
- Relatively newer player in market
|
||||
- **Requires verification** before enterprise deployment
|
||||
|
||||
### API/Application Limits
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Information Needed:**
|
||||
- Number of APIs per organization
|
||||
- Number of endpoints per API
|
||||
- API specification size limits (OpenAPI, GraphQL schema size)
|
||||
- Limits vary by plan tier
|
||||
### Recommendations
|
||||
|
||||
### Scan Limits
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Information Needed:**
|
||||
- Number of scans per month
|
||||
- Scan frequency (continuous vs scheduled)
|
||||
- Concurrent scan limits
|
||||
- Scan duration/timeout limits
|
||||
- Test depth limits (number of requests per scan)
|
||||
**✅ PROVISIONALLY APPROVED WITH VERIFICATION**
|
||||
|
||||
### User/Seat Limits
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Information Needed:**
|
||||
- Minimum seats for paid plans
|
||||
- Maximum team members
|
||||
- Read-only user limits
|
||||
- Guest/external user policies
|
||||
**Why DAST is Lower Risk:**
|
||||
- Only tests externally observable behavior
|
||||
- No access to proprietary source code
|
||||
- Similar risk profile to external pentesting
|
||||
- Internal business logic not exposed
|
||||
|
||||
### Integration Limits
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Information Needed:**
|
||||
- Number of CI/CD integrations
|
||||
- Webhook delivery limits
|
||||
- Issue tracker integration limits
|
||||
**Action Required BEFORE Use:**
|
||||
1. **Request written confirmation:** App-specific data not used for training
|
||||
2. **Clarify data retention:** Get vulnerability finding retention policies
|
||||
3. **Review security:** Ensure findings stay confidential
|
||||
4. **Test environment:** Use on staging/test environments first
|
||||
5. **NDA:** Ensure confidentiality agreement covers findings
|
||||
|
||||
**Best For:**
|
||||
- API security testing (REST, GraphQL, etc.)
|
||||
- Organizations wanting AI-powered DAST
|
||||
- Teams with modern cloud-native applications
|
||||
- Continuous security testing in CI/CD
|
||||
|
||||
**Not Ideal For:**
|
||||
- Testing production with real customer data (standard pentesting caution)
|
||||
- Organizations requiring complete SAST + DAST (Escape is DAST only)
|
||||
- Companies uncomfortable with any cloud-based security testing
|
||||
|
||||
**For Hack Days:**
|
||||
- ✅ **SAFER THAN SAST TOOLS** - doesn't access source code
|
||||
- Point it at test/staging environment
|
||||
- Ensure test environment doesn't contain real customer data
|
||||
- Monitor what endpoints are tested
|
||||
- Review findings before wider deployment
|
||||
|
||||
### Comparison to Alternatives
|
||||
- **vs SAST (Snyk Code, etc.):** Escape has LOWER IP exposure (no code upload)
|
||||
- **vs Aikido:** Similar AI pentesting approach, similar verification needs
|
||||
- **vs Traditional DAST:** Escape adds AI for logic flaw detection
|
||||
- **vs Manual Pentesting:** Automated, continuous, more cost-effective
|
||||
|
||||
**Unique Advantage:** AI finds complex logic flaws that traditional scanners miss, without requiring source code access
|
||||
|
||||
---
|
||||
|
||||
## Enterprise-Specific Considerations
|
||||
## Sources & References
|
||||
|
||||
### Compliance & Certifications
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Sources to Check:**
|
||||
- Security/Trust page: https://escape.tech/security or https://escape.tech/trust
|
||||
- Compliance documentation
|
||||
|
||||
**Check for:**
|
||||
- SOC 2 Type II certification
|
||||
- ISO 27001
|
||||
- GDPR compliance
|
||||
- CCPA compliance
|
||||
- PCI-DSS relevance (for payment API testing)
|
||||
- OWASP API Security Top 10 coverage
|
||||
- Penetration testing certifications (CREST, OSCP, etc.)
|
||||
|
||||
### Deployment Options
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Check for:**
|
||||
- Cloud-hosted (SaaS) - standard offering
|
||||
- Self-hosted/on-premise options
|
||||
- Private cloud deployment
|
||||
- Air-gapped environment support (critical for pentesting)
|
||||
- Hybrid deployment (control plane cloud, scanning on-prem)
|
||||
- Data residency options (EU, US, etc.)
|
||||
|
||||
### Security Features
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Check for:**
|
||||
- SSO/SAML support
|
||||
- SCIM provisioning
|
||||
- Role-based access control (RBAC)
|
||||
- Audit logging
|
||||
- Data encryption (in transit and at rest)
|
||||
- Two-factor authentication (2FA)
|
||||
- IP whitelisting
|
||||
- API key management
|
||||
- Secure credential storage for API authentication
|
||||
|
||||
### Testing Capabilities
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Check for:**
|
||||
- OWASP API Security Top 10 coverage
|
||||
- Business logic testing
|
||||
- Authentication/authorization testing
|
||||
- Rate limiting testing
|
||||
- Input validation testing
|
||||
- GraphQL-specific tests
|
||||
- gRPC testing support
|
||||
- REST API testing
|
||||
- WebSocket testing
|
||||
- Custom rule creation
|
||||
- Compliance testing (PCI-DSS, HIPAA)
|
||||
|
||||
### Integration Ecosystem
|
||||
**Status:** REQUIRES MANUAL VERIFICATION
|
||||
**Check for:**
|
||||
- CI/CD platforms (GitHub Actions, GitLab CI, Jenkins, CircleCI)
|
||||
- Issue trackers (Jira, Linear, GitHub Issues)
|
||||
- Communication (Slack, Microsoft Teams, email)
|
||||
- API Gateways (Kong, Apigee, AWS API Gateway)
|
||||
- Documentation platforms (Swagger, Postman)
|
||||
- Cloud providers (AWS, Azure, GCP)
|
||||
|
||||
---
|
||||
|
||||
## Research Checklist
|
||||
|
||||
- [ ] Read complete privacy policy
|
||||
- [ ] Review terms of service
|
||||
- [ ] Check pricing page for current costs and plans
|
||||
- [ ] Review API documentation for rate limits
|
||||
- [ ] Check for Data Processing Agreement (DPA)
|
||||
- [ ] Verify GDPR/compliance information
|
||||
- [ ] Review documentation for deployment options
|
||||
- [ ] Check security/trust center for certifications
|
||||
- [ ] Test free tier if possible (understand sign-up requirements)
|
||||
- [ ] Contact sales for enterprise-specific information
|
||||
- [ ] Review integration documentation
|
||||
- [ ] Check for customer case studies or testimonials
|
||||
- [ ] Understand AI/ML capabilities and data usage
|
||||
- [ ] Review vulnerability disclosure policy
|
||||
- [ ] Check for bug bounty program
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- Documentation: https://docs.escape.tech/
|
||||
- Blog: https://escape.tech/blog
|
||||
- GitHub: Check for open-source components, examples, or client libraries
|
||||
- Status Page: Check for uptime/status page
|
||||
- Support: Check for support portal or contact information
|
||||
- Community: Discord, Slack, or forum
|
||||
1. **ChatGPT Evaluation Report** - Escape AI analysis
|
||||
2. **Escape Website**: https://escape.tech
|
||||
3. **Industry Context**: DAST security testing standards
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
- Last Updated: 2026-02-02
|
||||
- Evaluator: Claude Code based on ChatGPT findings
|
||||
- **Key Finding:** Escape's DAST approach means lower IP exposure than SAST tools (no source code upload)
|
||||
- **Advantage:** Black-box testing approach naturally protects proprietary code
|
||||
- **Status:** Provisionally approved with verification recommended
|
||||
|
||||
**Important Considerations for Escape:**
|
||||
1. **AI-Powered Testing:** Understand exactly how AI is used and what data feeds the models
|
||||
2. **API Data Sensitivity:** APIs often contain business-critical data - verify how test traffic is handled
|
||||
3. **Testing Production APIs:** Clarify if/how production API testing is supported safely
|
||||
4. **Authentication Handling:** Critical to understand how API credentials and tokens are managed
|
||||
5. **Vulnerability Disclosure:** Check if discovered vulnerabilities are reported externally or kept confidential
|
||||
6. **Test Traffic Generation:** Understand volume and nature of test requests sent to customer APIs
|
||||
7. **False Positive Rate:** Check if AI reduces false positives compared to traditional scanners
|
||||
8. **Schema Requirements:** Verify if OpenAPI/GraphQL schemas are required or if discovery is automatic
|
||||
---
|
||||
|
||||
**Critical Security Questions:**
|
||||
- Does pentesting generate traffic that could trigger rate limits or WAF rules?
|
||||
- How are API keys and credentials securely provided to the platform?
|
||||
- Is there a way to test staging/development environments only?
|
||||
- What happens if a critical vulnerability is discovered?
|
||||
- Are pentest results encrypted at rest?
|
||||
## DAST vs SAST - Privacy Comparison
|
||||
|
||||
**Competitive Positioning:**
|
||||
- Compare to: StackHawk, Astra Security, Veracode, HCL AppScan
|
||||
- Note AI-specific advantages or limitations
|
||||
- Evaluate API-specific features vs general application security
|
||||
### DAST (Escape, traditional scanners)
|
||||
- ✅ **No source code uploaded**
|
||||
- ✅ Tests external behavior only
|
||||
- ✅ Lower IP exposure risk
|
||||
- ⚠️ May miss internal code issues
|
||||
|
||||
**Company Background:**
|
||||
- Verify company funding, stability, and reputation
|
||||
- Check for any security incidents in the past
|
||||
- Review customer testimonials specific to enterprise use
|
||||
### SAST (Snyk Code, etc.)
|
||||
- ⚠️ Source code uploaded to cloud
|
||||
- ✅ Finds internal code vulnerabilities
|
||||
- ⚠️ Higher IP exposure risk (mitigated by no-training policies)
|
||||
- ✅ Catches more vulnerability types
|
||||
|
||||
**Last Updated:** [DATE TO BE ADDED AFTER RESEARCH]
|
||||
**Researcher:** [TO BE COMPLETED]
|
||||
**Verification Status:** Template - Requires Manual Research
|
||||
**For Maximum Privacy:** Use DAST (Escape) for external testing + On-prem SAST for internal scanning
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ VERIFICATION CHECKLIST
|
||||
|
||||
Before using Escape:
|
||||
- [ ] Request confirmation: App data not used for AI training
|
||||
- [ ] Clarify: Vulnerability finding retention period
|
||||
- [ ] Ensure: Test environment has no real customer data
|
||||
- [ ] Verify: Results stay confidential to your team
|
||||
- [ ] Check: NDA/confidentiality agreement available
|
||||
- [ ] Review: Which endpoints/APIs will be tested
|
||||
|
||||
**Recommendation:** Escape is LOWER RISK than SAST tools for IP protection. Good choice for API security testing after basic verification.
|
||||
Reference in new issue
Block a user