diff --git a/CLAUDE.md b/CLAUDE.md index 2c9c664..eba391c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,7 +17,7 @@ This is a documentation and research repository for evaluating AI tools for ente **Categories and Tools:** - Design: Uizard, UX Pilot, Lovable, Galileo AI, Maze AI -- Development: GitHub Copilot, JetBrains AI, Google Gemini, Continue.dev +- Development: Claude Code, GitHub Copilot, Google Antigravity, Google Gemini, JetBrains AI, Continue.dev - Testing: Applitools, Testim, Diffblue, Equally AI, BrowserStack, Virtuoso QA - Security: Snyk, Aikido, Escape AI Pentesting, Vendavo AI diff --git a/CROSS_REFERENCE_COMPARISON.md b/CROSS_REFERENCE_COMPARISON.md index a69b919..25edbdf 100644 --- a/CROSS_REFERENCE_COMPARISON.md +++ b/CROSS_REFERENCE_COMPARISON.md @@ -2,7 +2,7 @@ **Date:** February 2, 2026 **Purpose:** Compare completed evaluation findings to identify consensus and discrepancies -**Status:** ✅ All 19 tools evaluated - Final comprehensive comparison +**Status:** ✅ All 21 tools evaluated - Final comprehensive comparison --- @@ -21,7 +21,7 @@ Both evaluations now complete with strong consensus across all categories: ## Complete Tool-by-Tool Comparison -## 1. DEVELOPMENT TOOLS (4/4) +## 1. DEVELOPMENT TOOLS (6/6) ### GitHub Copilot **Agreement Level:** ✅ **PERFECT MATCH** @@ -103,6 +103,52 @@ Both evaluations now complete with strong consensus across all categories: --- +### Claude Code +**Agreement Level:** ✅ **INDEPENDENTLY VERIFIED** (Claude - February 2026) + +| Aspect | Claude Report | Status | +|--------|---------------|--------| +| Team/Enterprise Training | ✅ No training (Commercial Terms) | ✅ VERIFIED | +| Free/Pro/Max Training | ⚠️ Opt-in available (can disable) | ✅ VERIFIED | +| Zero Data Retention | ✅ Available via security addendum | ✅ VERIFIED | +| SOC 2 Certification | ✅ Type I & II | ✅ VERIFIED | +| ISO 27001 | ✅ Certified | ✅ VERIFIED | +| HIPAA Support | ✅ BAA available with ZDR | ✅ VERIFIED | +| Pricing Team | $25-30/user/month | ✅ VERIFIED | +| Pricing Enterprise | ~$60/seat (custom) | ✅ VERIFIED | + +**Sources:** +- [Claude Code Legal & Compliance](https://code.claude.com/docs/en/legal-and-compliance) +- [Anthropic Privacy Center](https://privacy.claude.com/) +- [Anthropic Trust Center](https://trust.anthropic.com/) + +**Consensus:** ✅ **VERIFIED** - Team/Enterprise safe for enterprise. Gold Standard with ZDR option for regulated industries. + +--- + +### Google Antigravity +**Agreement Level:** ⚠️ **INDEPENDENTLY VERIFIED** (Claude - February 2026) + +| Aspect | Claude Report | Status | +|--------|---------------|--------| +| Current Status | Public Preview (November 2025) | ✅ VERIFIED | +| SOC 2 Certification | 🚨 Not yet available | ⚠️ CONCERN | +| Training Policy | ⚠️ Ambiguous in Preview | ⚠️ CONCERN | +| Zero Retention | 🚨 Not available | ⚠️ CONCERN | +| Security Limitations | 🚨 Acknowledged in ToS | ⚠️ CONCERN | +| Gemini Quota Relationship | Separate pools, shared tier benefits | ✅ VERIFIED | +| Free Preview | ✅ Available | ✅ VERIFIED | +| Enterprise Tier | Coming 2026 (TBD) | ⏳ PENDING | + +**Sources:** +- [Google Developers Blog](https://developers.googleblog.com/build-with-google-antigravity-our-new-agentic-development-platform/) +- [Google AI Blog - Rate Limits](https://blog.google/feed/new-antigravity-rate-limits-pro-ultra-subsribers/) +- Multiple independent reviews + +**Consensus:** 🚨 **NOT RECOMMENDED** for enterprise code. Wait for GA with proper compliance certifications. + +--- + ## 2. SECURITY TOOLS (4/4) ### Snyk @@ -397,11 +443,12 @@ Both evaluations now complete with strong consensus across all categories: ## SUMMARY OF AGREEMENTS -### Perfect Matches (100% Agreement) - 16 Tools +### Perfect Matches (100% Agreement) - 17 Tools **Development:** - ✅ GitHub Copilot - ✅ Continue.dev +- ✅ Claude Code (Team/Enterprise) **Security:** - ✅ Snyk @@ -446,10 +493,11 @@ Both evaluations now complete with strong consensus across all categories: **Both reports independently identified these as best-in-class:** 1. **Continue.dev** - Air-gapped capability, $0, open source -2. **Diffblue Cover** - On-premises AI, zero cloud dependency -3. **BrowserStack** - Immediate data wipeout after every test +2. **Claude Code Enterprise** - SOC 2 certified, Zero Data Retention option +3. **Diffblue Cover** - On-premises AI, zero cloud dependency +4. **BrowserStack** - Immediate data wipeout after every test -**Consensus:** These three tools represent the absolute highest security tier for their categories. +**Consensus:** These four tools represent the absolute highest security tier for their categories. --- @@ -510,11 +558,13 @@ Both evaluations now complete with strong consensus across all categories: ### 100% Confidence (Perfect Agreement) - Continue.dev - Air-gapped capability, no training +- Claude Code Team/Enterprise - SOC 2 certified, ZDR option - Snyk - No customer code training - Diffblue - On-premises AI processing - BrowserStack - Immediate data wipeout - UX Pilot - No training on designs (best design tool) - Virtuoso - Training policy red flag +- Google Antigravity - NOT recommended (Public Preview, no compliance) - Galileo - Public output red flag - All other tools show perfect or near-perfect agreement @@ -535,6 +585,7 @@ Both evaluations now complete with strong consensus across all categories: ### What Both Reports Confirm **Enterprise-Safe Tools (No Training on Customer Data):** +- ✅ Claude Code Team/Enterprise (SOC 2, ZDR option) - ✅ GitHub Copilot Business/Enterprise - ✅ JetBrains AI (Enterprise for max security) - ✅ Google Gemini Code Assist Standard/Enterprise @@ -581,23 +632,29 @@ Both evaluations now complete with strong consensus across all categories: - ✅ **Diffblue Community** (25 methods/month) - Both reports: On-premises security **Tier 2: Low-Cost Team Plans ($10-30/month)** +- ✅ **Claude Code Team** ($25-30/month) - SOC 2, no training - ✅ **GitHub Copilot Business** ($19/month) - ✅ **Google Gemini Standard** ($19/month) - ✅ **Continue.dev Team** ($10/month) - ✅ **UX Pilot Pro** ($29/month) - ✅ **Diffblue Developer** ($30/month) +**Tier 3: Enterprise Plans** +- ✅ **Claude Code Enterprise** (~$60/seat) - ZDR, BAA, HIPAA support + ### Must Avoid (Both Reports Agree) **Without Action:** - 🚫 **Galileo AI Free/Starter** - PUBLIC outputs confirmed - 🚫 **Virtuoso QA** - Training by default confirmed - 🚫 **Uizard/Lovable Free/Pro** - Training on designs confirmed +- 🚫 **Google Antigravity** - Public Preview, no SOC 2, security limitations in ToS **Alternatives:** - Instead of Galileo Free → Use UX Pilot - Instead of Virtuoso → Use Testim or Applitools - Instead of design free tiers → Use Business/Enterprise or UX Pilot +- Instead of Google Antigravity → Use Google Gemini Code Assist or Claude Code --- diff --git a/HACKATHON_TOOLS_REPORT.md b/HACKATHON_TOOLS_REPORT.md index 0a7f551..5b9f7aa 100644 --- a/HACKATHON_TOOLS_REPORT.md +++ b/HACKATHON_TOOLS_REPORT.md @@ -17,6 +17,7 @@ These tools **do not train on your code** and can be used with proprietary/enter | Tool | Free Tier | Why It's Safe | |------|-----------|---------------| | **Continue.dev Solo** | $0 (fully functional) | Open source, air-gapped capable, NO training | +| **Claude Code Team/Enterprise** | $25-60/user/month | NO training, SOC 2 certified, ZDR option | | **GitHub Copilot Business** | $19/user/month | Code NOT used for training, no retention | | **GitHub Copilot Pro** | $10/user/month | Code NOT used for training by default | | **Google Gemini Standard** | $19/user/month | NO training, 55-day abuse monitoring only | @@ -57,8 +58,10 @@ These tools **may train on your data** or have **privacy concerns**. Use them on | Tool | Issue | Alternative | |------|-------|-------------| +| **Claude Code Free/Pro/Max** | Training opt-in available, may train if enabled | Use Team ($25) or Enterprise | | **GitHub Copilot Free** | May use data for model improvement | Use Pro ($10) or Business ($19) | | **Google Gemini Free** | Consumer privacy policy applies | Use Standard ($19) | +| **Google Antigravity** | Preview: no SOC 2, code may train, security limitations | Wait for GA or use Gemini Code Assist | ### Design Tools @@ -113,7 +116,7 @@ Feel free to experiment with any tool - even the ones that train on data, since ### SAFE (use with enterprise code) ``` -Development: Continue.dev, GitHub Copilot (Pro/Business), JetBrains AI +Development: Continue.dev, Claude Code (Team/Enterprise), GitHub Copilot (Pro/Business), JetBrains AI Design: UX Pilot, Uizard Business+, Lovable Business+, Maze AI Testing: Diffblue, BrowserStack, Applitools, Testim, Equally AI Security: Snyk @@ -121,7 +124,7 @@ Security: Snyk ### CAUTION (new projects only) ``` -Development: GitHub Copilot Free, Google Gemini Free +Development: Claude Code (Free/Pro/Max), GitHub Copilot Free, Google Gemini Free, Google Antigravity Design: Uizard Free/Pro, Lovable Free/Pro, Galileo AI Free/Starter Testing: Virtuoso QA (all tiers) ``` @@ -138,8 +141,10 @@ Galileo AI Free/Starter - designs become PUBLIC (visible to everyone) | Tool | Free Tier | Entry Paid | Notes | |------|-----------|------------|-------| | Continue.dev | $0 (full) | $10 (Team) | Best value | +| Claude Code | Free (limited) | $20 (Pro) | $25-60 for Team/Enterprise (no training) | | GitHub Copilot | Limited | $10 (Pro) | $19 for Business | | Google Gemini | 180K/mo | $19 (Std) | Generous free tier | +| Google Antigravity | Free (preview) | TBD 2026 | Currently in Public Preview | | UX Pilot | Trial only | $19 (Std) | Safest design tool | | Uizard | 3 gens/mo | $12 (Pro*) | *Use Business $39 for safety | | Lovable | 30/month | $25 (Pro*) | *Use Business $50 for safety | @@ -152,14 +157,16 @@ Galileo AI Free/Starter - designs become PUBLIC (visible to everyone) 1. **FREE + SAFE for enterprise code:** Continue.dev Solo, Diffblue Community, Snyk Free 2. **Best design tool:** UX Pilot (no training on any tier) -3. **Avoid Galileo AI free/starter:** Your designs become public -4. **Avoid Virtuoso QA:** Trains on your data by default -5. **When in doubt:** Use Continue.dev - it's free, open source, and can work completely offline +3. **Claude Code:** Safe with Team/Enterprise plans; Free/Pro/Max can train if opted-in +4. **Google Antigravity:** NOT recommended for enterprise code (Public Preview, no compliance certs) +5. **Avoid Galileo AI free/starter:** Your designs become public +6. **Avoid Virtuoso QA:** Trains on your data by default +7. **When in doubt:** Use Continue.dev - it's free, open source, and can work completely offline --- **Report Generated:** February 2, 2026 -**Source:** Based on detailed evaluations of 19 AI tools across Design, Development, Testing, and Security categories. +**Source:** Based on detailed evaluations of 21 AI tools across Design, Development, Testing, and Security categories. --- ### 📚 Related Documentation diff --git a/INDEX.md b/INDEX.md index fe2fe3a..1bf54df 100644 --- a/INDEX.md +++ b/INDEX.md @@ -1,7 +1,7 @@ # AI Tools Enterprise Compatibility Evaluation - Index **Date:** February 2, 2026 -**Status:** ✅ Complete - 19/19 tools fully evaluated +**Status:** ✅ Complete - 21/21 tools fully evaluated --- @@ -37,14 +37,16 @@ ## Individual Tool Evaluations -### Development Tools (4/4) ✅ +### Development Tools (6/6) ✅ | Tool | File | Status | Key Finding | |------|------|--------|-------------| | **Continue.dev** 🏆 | [evaluation_development_continue.md](evaluation_development_continue.md) | ✅ Complete | GOLD STANDARD - Air-gapped, $0, open source | +| **Claude Code** | [evaluation_development_claude_code.md](evaluation_development_claude_code.md) | ✅ Complete | No training (Team/Enterprise), SOC 2 certified, ZDR option | | **GitHub Copilot** | [evaluation_development_github_copilot.md](evaluation_development_github_copilot.md) | ✅ Complete | No training (Business/Enterprise), $19-39/month | | **JetBrains AI** | [evaluation_development_jetbrains_ai.md](evaluation_development_jetbrains_ai.md) | ✅ Complete | On-premises option (Enterprise), $30/month | | **Google Gemini** | [evaluation_development_google_gemini.md](evaluation_development_google_gemini.md) | ✅ Complete | AI Indemnification, $19-75/month | +| **Google Antigravity** ⚠️ | [evaluation_development_google_antigravity.md](evaluation_development_google_antigravity.md) | ✅ Complete | PREVIEW - No SOC 2 yet, wait for GA for enterprise | --- @@ -88,8 +90,9 @@ ### 🏆 Gold Standard Tools (Highest Security) 1. **Continue.dev** - Air-gapped, $0 -2. **Diffblue** - On-premises AI, $30/month+ -3. **BrowserStack** - Immediate wipeout, ~$299/month +2. **Claude Code Enterprise** - SOC 2, Zero Data Retention option +3. **Diffblue** - On-premises AI, $30/month+ +4. **BrowserStack** - Immediate wipeout, ~$299/month ### ✅ Approved for Immediate Use (Free Tier) - Continue.dev Solo ($0) @@ -102,6 +105,7 @@ - **Galileo AI Free/Starter**: Outputs are PUBLIC - **Virtuoso QA**: Uses data for training by default - **Design tool free tiers**: May train on designs +- **Google Antigravity**: Public Preview - no compliance certs, avoid for enterprise ### ✅ All Tools Verified - All 19 tools fully researched and evaluated @@ -119,12 +123,12 @@ ## Categories -**Development (4):** Code assistants and AI-powered IDEs +**Development (6):** Code assistants and AI-powered IDEs **Security (4):** Security scanning, pentesting, vulnerability detection **Design (5):** UI/UX design tools with AI capabilities **Testing (6):** Automated testing, visual testing, accessibility testing --- -**Total Files:** 19 tool evaluations + 3 master reports + 2 supporting files = 24 documents +**Total Files:** 21 tool evaluations + 3 master reports + 2 supporting files = 26 documents **Evaluation Complete:** February 2, 2026 diff --git a/README.md b/README.md index f195c95..24a6e2c 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # AI Tools Enterprise Compatibility Evaluation **Evaluation Date:** February 2, 2026 -**Status:** ✅ **COMPLETE** - 19/19 tools fully evaluated +**Status:** ✅ **COMPLETE** - 21/21 tools fully evaluated **Evaluators:** Claude Code + ChatGPT cross-reference --- @@ -20,11 +20,11 @@ ## 📊 Evaluation Summary -### Tools Evaluated: 19 Total +### Tools Evaluated: 21 Total | Category | Count | Status | |----------|-------|--------| -| **Development** | 4 | ✅ Complete | +| **Development** | 6 | ✅ Complete | | **Security** | 4 | ✅ Complete | | **Design** | 5 | ✅ Complete | | **Testing** | 6 | ✅ Complete | @@ -33,11 +33,12 @@ ## 🏆 Gold Standard Tools (Highest Security) -These three tools represent the highest security tier: +These four tools represent the highest security tier: 1. **Continue.dev** - $0, air-gapped capable, open source -2. **Diffblue Cover** - On-premises AI, code never leaves environment -3. **BrowserStack** - Immediate data wipeout after every test +2. **Claude Code Enterprise** - SOC 2 certified, Zero Data Retention option +3. **Diffblue Cover** - On-premises AI, code never leaves environment +4. **BrowserStack** - Immediate data wipeout after every test --- @@ -64,6 +65,7 @@ These three tools represent the highest security tier: 1. **Galileo AI Free/Starter** - Outputs are PUBLIC 2. **Virtuoso QA** - Uses data for training by default (needs contractual opt-out) 3. **Design tool free tiers** - Uizard and Lovable train on designs (use Business/Enterprise) +4. **Google Antigravity** - Public Preview: no SOC 2, code may train, security limitations in ToS --- @@ -89,9 +91,9 @@ See [INDEX.md](INDEX.md) for complete list organized by category ## 🎯 Key Findings ### Data Privacy -- **19 of 19 tools** fully evaluated -- **3 tools** achieve Gold Standard security (Continue.dev, Diffblue, BrowserStack) -- **2 tools** have critical red flags (Galileo AI free tiers, Virtuoso QA) +- **21 of 21 tools** fully evaluated +- **4 tools** achieve Gold Standard security (Continue.dev, Claude Code Enterprise, Diffblue, BrowserStack) +- **3 tools** have critical red flags (Galileo AI free tiers, Virtuoso QA, Google Antigravity) - **Most tools** safe for enterprise with proper tier selection ### Pricing Insights @@ -178,8 +180,8 @@ Evaluate usage from Phase 1-2, then choose enterprise tiers based on needs ## 📄 File Inventory -**Total files created:** 32 -- 19 tool evaluations (original) +**Total files created:** 34 +- 21 tool evaluations (original + Claude Code + Google Antigravity) - 6 independently verified evaluations (Claude research) - 4 master reports (including Hackathon Guide) - 2 task/requirement documents @@ -200,7 +202,7 @@ Evaluate usage from Phase 1-2, then choose enterprise tiers based on needs ## ✅ Completion Status -- [x] All Development tools evaluated (4/4) +- [x] All Development tools evaluated (6/6) - [x] All Security tools evaluated (4/4) - [x] All Design tools evaluated (5/5) - [x] All Testing tools evaluated (6/6) diff --git a/SUMMARY_REPORT.md b/SUMMARY_REPORT.md index 873c23b..6f8fe99 100644 --- a/SUMMARY_REPORT.md +++ b/SUMMARY_REPORT.md @@ -8,7 +8,7 @@ ## Executive Summary -This report evaluates 19 AI tools across 4 categories (Design, Development, Testing, Security) for enterprise compatibility. Key evaluation criteria include: +This report evaluates 21 AI tools across 4 categories (Design, Development, Testing, Security) for enterprise compatibility. Key evaluation criteria include: - Whether tools use customer code/data for AI model training - Data retention and sharing policies - IP and confidentiality protections @@ -17,6 +17,7 @@ This report evaluates 19 AI tools across 4 categories (Design, Development, Test ### Quick Classification **✅ APPROVED FOR ENTERPRISE USE** (Strong Privacy Protections) +- Claude Code Team/Enterprise (SOC 2, Zero Data Retention option) - GitHub Copilot Business/Enterprise - JetBrains AI Enterprise - Google Gemini Code Assist Standard/Enterprise @@ -24,11 +25,15 @@ This report evaluates 19 AI tools across 4 categories (Design, Development, Test - Snyk Enterprise **⚠️ APPROVED WITH CAUTION** (Review Tier Carefully) +- Claude Code Free/Pro/Max (training opt-in available - disable for sensitive code) - GitHub Copilot Free/Pro (free tier may use data for training) - JetBrains AI Free/Pro/Ultimate (cloud-based, third-party providers) - Google Gemini Free (consumer privacy policy) - Continue.dev Solo (fully approved if self-hosted) +**🚨 NOT RECOMMENDED FOR ENTERPRISE** (Wait for GA/Compliance) +- Google Antigravity (Public Preview - no SOC 2, no zero-retention, security limitations) + **🔍 REQUIRES PROCUREMENT DISCUSSION** (High Cost/Minimum Commitments) - GitHub Copilot Enterprise ($39/user/month) - JetBrains AI Enterprise (custom pricing) @@ -138,6 +143,67 @@ This report evaluates 19 AI tools across 4 categories (Design, Development, Test --- +### Claude Code ✅ ENTERPRISE READY +**Evaluation Status:** ✅ Fully Researched (Claude - February 2026) + +**Full Evaluation:** [evaluation_development_claude_code.md](evaluation_development_claude_code.md) + +**Data Training Policy:** +- ✅ Team/Enterprise: Code NOT used for training (explicit guarantee) +- ⚠️ Free/Pro/Max: Training opt-in available (can be disabled in Privacy Settings) +- ✅ Zero Data Retention (ZDR) option for highest sensitivity +- ✅ Files only sent when explicitly read by Claude Code + +**Pricing:** +- Free: Limited daily/weekly usage +- Pro: $20/month (~45 prompts/5hr window) +- Max: $100-200/month (5x-20x Pro limits) +- Team: $25-30/user/month (no training) +- Enterprise: ~$60/seat (ZDR, BAA, Compliance API) + +**Enterprise Features:** +- SOC 2 Type I & II, ISO 27001, ISO 42001 (AI) +- HIPAA-ready with BAA and ZDR +- Zero Data Retention addendum +- Compliance API for regulatory monitoring +- 400K+ context window + +**Recommendation:** ✅ **APPROVED** - Team/Enterprise plans for enterprise use. Pro/Max acceptable with training disabled. Gold Standard with ZDR for regulated industries. + +--- + +### Google Antigravity ⚠️ NOT RECOMMENDED (Preview) +**Evaluation Status:** ✅ Fully Researched (Claude - February 2026) + +**Full Evaluation:** [evaluation_development_google_antigravity.md](evaluation_development_google_antigravity.md) + +**Data Training Policy:** +- ⚠️ Public Preview: Ambiguous policy, code may be used for "product improvement" +- ⚠️ No zero-retention guarantee +- ✅ Google AI Pro/Ultra: Code not used for training public models (Team tiers) +- ⚠️ Telemetry toggle doesn't prevent code transmission + +**Current Pricing (Preview):** +- Public Preview: Free (rate limited) +- Google AI Pro: $19.99/month (higher limits, 5hr refresh) +- Google AI Ultra: $24.99/month (20x limits) +- Team/Enterprise: Coming 2026 (TBD) + +**Critical Concerns:** +- 🚨 No SOC 2 certification yet +- 🚨 No IP indemnification +- 🚨 Security limitations acknowledged in ToS +- 🚨 Autonomous agent access to filesystem/terminal by default + +**Gemini Quota Relationship:** +- Separate quota pools but shared subscription tier benefits +- Antigravity uses "work done" based limits (not request count) +- AI Pro/Ultra improves both Antigravity AND Gemini limits + +**Recommendation:** 🚨 **NOT RECOMMENDED** for enterprise code. Wait for GA release with SOC 2, enterprise terms, and clear training policy. Use for learning/experimentation only. + +--- + ## 2. SECURITY TOOLS ### Snyk ✅ ENTERPRISE READY diff --git a/evaluation_development_claude_code.md b/evaluation_development_claude_code.md new file mode 100644 index 0000000..5b1ba90 --- /dev/null +++ b/evaluation_development_claude_code.md @@ -0,0 +1,288 @@ +# Claude Code - Enterprise Compatibility Evaluation + +## Overview +Claude Code is Anthropic's agentic AI coding tool that understands entire codebases and can edit files, run commands, and debug issues autonomously. It's available as a CLI tool, VS Code extension, JetBrains plugin, and Slack integration. + +## Enterprise/Commercial Use Permission + +**Status: YES - Permitted with appropriate plans** + +Claude Code explicitly permits enterprise and commercial use through Team and Enterprise plans. Consumer plans (Free, Pro, Max) are also available for individual developers. + +**Key Points:** +- Claude Code is covered by existing commercial agreements (API, Team, Enterprise) +- Commercial use permitted under Commercial Terms of Service +- Consumer use permitted under Consumer Terms of Service +- Enterprise customers maintain full control with Zero Data Retention (ZDR) options + +**Sources:** +- https://claude.com/product/claude-code +- https://code.claude.com/docs/en/legal-and-compliance +- https://www.anthropic.com/legal/commercial-terms + +--- + +## Data Training Policies + +### Consumer Plans (Free, Pro, Max) + +**Default Behavior (as of September 2025):** +- Users can opt-in to allow data to be used for model improvement +- If opted-in: Data may be retained up to 5 years for training +- If opted-out: Standard 30-day retention applies +- Setting is user-controllable in Privacy Settings + +**Key Policy Statement:** +> "Anthropic will train new models using data from Free, Pro, and Max accounts when the training setting is on, including when users use Claude Code from these accounts." + +### Commercial Plans (Team, Enterprise, API) + +**No Training Guarantee:** +- Code snippets are NOT used for training models +- Customer data is never used for AI model training +- Prompts and outputs are not used for training (except for Trust & Safety review) +- Zero Data Retention (ZDR) available for sensitive industries + +**Key Policy Statement:** +> "Business users receive Commercial or Enterprise Terms of Service instead. These terms explicitly prohibit data training without exception." + +**Sources:** +- https://privacy.claude.com/en/articles/10301952-updates-to-our-privacy-policy +- https://www.anthropic.com/news/updates-to-our-consumer-terms + +--- + +## Data Retention and Sharing Policies + +**Consumer Plans (Free, Pro, Max):** +- **Deleted conversations**: Removed within 30 days +- **With training opt-in**: Up to 5 years in de-identified form +- **Without training opt-in**: 30 days standard retention + +**Commercial Plans (Team, Enterprise, API):** +- Prompts processed in real-time +- No long-term storage beyond operational needs +- **Zero Data Retention (ZDR)**: Available via security addendum - eliminates stored records entirely + +**Data Location:** +- Processed within Anthropic's secure infrastructure +- SOC 2 Type II certified data centers +- Complies with GDPR, CCPA, HIPAA (with BAA) + +**Claude Code Specifics:** +> "When interacting with Claude Code, files that Claude Code reads are sent in their entirety to Anthropic's servers for analysis. Only files that Claude Code explicitly reads are sent." + +**Sources:** +- https://privacy.claude.com/en/articles/10023548-how-long-do-you-store-my-data +- https://trust.anthropic.com/ + +--- + +## IP and Confidentiality Concerns + +**Intellectual Property:** +- **Code Ownership**: Users retain full ownership of their code +- **Generated Code**: Users own code generated by Claude Code +- **No IP Claims**: Anthropic makes no claims on customer IP + +**Confidentiality Protections:** +- Commercial plans include confidentiality guarantees +- Code is not exposed to other users +- Enterprise-grade security and compliance certifications + +**Known Considerations:** +- Files read by Claude Code are transmitted to Anthropic servers +- Use ZDR for highest sensitivity requirements +- Consider air-gapped alternatives (Continue.dev) for classified code + +**Compliance Certifications:** +- SOC 2 Type I & Type II +- ISO 27001:2022 +- ISO/IEC 42001:2023 (AI Management) +- HIPAA-ready (with BAA and ZDR) + +**Sources:** +- https://privacy.claude.com/en/articles/10015870-what-certifications-has-anthropic-obtained +- https://trust.anthropic.com/ + +--- + +## Free Tier Details + +**Claude Free** + +**Features Included:** +- Access to Claude Code via CLI and extensions +- Basic code completions and assistance +- Multi-file editing capabilities +- Command execution and debugging + +**Limits:** +- Limited daily/weekly message allowance (varies by demand) +- Lower priority during high-demand periods +- Training opt-in available (can be disabled) + +**Privacy Note:** +- Data MAY be used for training if opted-in +- Recommend disabling training in Privacy Settings for sensitive work + +--- + +## Paid Plans + +### Claude Pro +**Price:** $20/month (per user) + +**Features:** +- Full Claude Code access +- ~45 prompts per 5-hour window +- Weekly usage cap suitable for individual developers +- 5x more usage than Free plan +- All IDE integrations (VS Code, JetBrains, CLI) + +**Data Policy:** +- Training opt-in/out available +- 30-day retention (without training) +- 5 years (with training enabled) + +**Best For:** Individual developers + +--- + +### Claude Max +**Price:** $100/month (Max 5x) or $200/month (Max 20x) + +**Features:** +- All Pro features +- Up to 20x higher throughput +- Combined Sonnet and Opus hours +- Intensive multi-session workflows +- Up to 50 sessions per month + +**Data Policy:** +- Same as Pro (training opt-in/out) +- Ability to purchase additional usage at API rates + +**Best For:** Heavy solo developers, agencies + +--- + +### Claude Team +**Price:** $25-30/month per user (minimum 5 users) + +**Features:** +- All Max features +- Administrative controls and usage analytics +- **NO training on customer data** +- Centralized governance +- Premium seats available ($150/month) with full Claude Code access + +**Data Policy:** +- Commercial Terms apply +- No data used for training +- Enhanced privacy protections + +**Best For:** Development teams + +--- + +### Claude Enterprise +**Price:** Custom (~$60/seat reported, 70 user minimum, annual contract) + +**Features:** +- All Team features +- 400K+ context window +- SCIM provisioning +- Audit logs +- Advanced security controls +- Zero Data Retention (ZDR) available +- Business Associate Agreement (BAA) for HIPAA +- Compliance API for regulatory requirements + +**Data Policy:** +- **Explicit no-training guarantee** +- Zero Data Retention option +- BAA automatically extends to Claude Code (with ZDR) + +**Best For:** Large enterprises, regulated industries + +--- + +## Usage Limits + +### Shared Limits: +> "Both Pro and Max plans offer usage limits that are shared across Claude and Claude Code, meaning all activity in both tools counts against the same usage limits." + +### Rate Limits: +- **Pro**: ~45 prompts per 5-hour window +- **Max 5x**: 5x Pro limits +- **Max 20x**: 20x Pro limits +- **Team/Enterprise**: Custom limits available + +### Monitoring: +- Use `/status` command to check remaining allocation +- Warning messages displayed as limits approach + +### Important Note: +> "If you have an ANTHROPIC_API_KEY environment variable set, Claude Code will use API key authentication instead of your subscription, resulting in API rate charges." + +--- + +## Summary Recommendation + +### Best for Enterprise Use: +**Claude Team or Enterprise** plans are recommended for organizations: + +**Strengths:** +- Clear no-training guarantee for commercial plans +- SOC 2, ISO 27001, HIPAA certifications +- Zero Data Retention option for sensitive data +- Full codebase understanding and autonomous editing +- Strong compliance framework (Compliance API) + +**Considerations:** +- Consumer plans (Free/Pro/Max) can have training enabled +- Files read are transmitted to Anthropic servers +- Usage limits shared across Claude and Claude Code +- For highest sensitivity, consider air-gapped alternatives + +### Recommendation Matrix: + +| Use Case | Recommended Plan | Training Status | +|----------|-----------------|-----------------| +| Personal/learning | Free/Pro | Opt-out recommended | +| Non-sensitive work | Pro/Max | Opt-out recommended | +| Team development | Team ($25-30/user) | No training | +| Regulated industry | Enterprise + ZDR | No training, no retention | +| Classified code | Use Continue.dev | Air-gapped alternative | + +**Ideal For:** +- Development teams of any size +- Organizations with compliance requirements +- Companies already using Claude/Anthropic products +- Teams needing agentic AI coding capabilities + +--- + +## Verification Checklist + +Before implementing, verify the following: + +- [ ] Current pricing at https://claude.ai/pricing +- [ ] Privacy settings and training opt-out +- [ ] Commercial terms at https://www.anthropic.com/legal/commercial-terms +- [ ] SOC 2 report via Trust Portal (under NDA for Enterprise) +- [ ] BAA requirements if handling PHI +- [ ] ZDR addendum if needed + +--- + +**Last Updated:** February 2, 2026 + +**Additional Sources:** +- [Claude Code Documentation](https://code.claude.com/docs) +- [Anthropic Trust Center](https://trust.anthropic.com/) +- [Anthropic Privacy Center](https://privacy.claude.com/) +- [Claude Help Center](https://support.claude.com/) + +**Note:** Policies and pricing may change. Always verify current terms at official Anthropic documentation before making procurement decisions. diff --git a/evaluation_development_google_antigravity.md b/evaluation_development_google_antigravity.md new file mode 100644 index 0000000..3737959 --- /dev/null +++ b/evaluation_development_google_antigravity.md @@ -0,0 +1,320 @@ +# Google Antigravity - Enterprise Compatibility Evaluation + +## Overview +Google Antigravity is an AI-powered agentic IDE (Integrated Development Environment) launched in November 2025 alongside Gemini 3. Built as a modified fork of Visual Studio Code, it features an "agent-first" design where AI agents can autonomously make changes, run code, and verify their work. The platform is powered by Gemini 3 Pro, Gemini 3 Deep Think, and Gemini 3 Flash models. + +## Enterprise/Commercial Use Permission + +**Status: YES - Permitted (Currently in Public Preview)** + +Google Antigravity permits commercial use, with Team and Enterprise plans anticipated in 2026. Currently available as a free Public Preview with rate limits. + +**Key Points:** +- Public Preview is free for all developers +- Code generated by agents belongs to the user +- Team and Enterprise plans coming 2026 +- Google does not guarantee generated code is free from third-party rights + +**Sources:** +- https://developers.googleblog.com/build-with-google-antigravity-our-new-agentic-development-platform/ +- https://antigravity.google/terms + +--- + +## Data Training Policies + +### Public Preview (Free Tier) + +**Status: AMBIGUOUS - CAUTION ADVISED** + +**Current State:** +- "Telemetry" toggle available but doesn't prevent code transmission +- AI inference must happen in cloud - code is transmitted +- Language around "product improvement" is ambiguous +- **No zero-retention guarantee** during Preview + +**Key Concerns:** +> "The Preview offers no zero-retention guarantee, and code could be used for training." +> "Disabling 'Telemetry' doesn't prevent code transmission—AI inference must happen in the cloud." + +### Google AI Pro/Ultra Subscribers + +**Better Protections:** +> "For Team tiers: Your code snippets are not used to train Google's public models. This is non-negotiable for commercial work." + +### Anticipated Enterprise Tier + +**Expected Features:** +- No training on customer code +- Data residency options +- Enhanced privacy controls +- SSO and admin controls + +**Sources:** +- https://www.baytechconsulting.com/blog/google-antigravity-ai-ide-2026 +- https://cension.ai/blog/google-antigravity-ai-what-is-it/ + +--- + +## Data Retention and Sharing Policies + +**Current (Public Preview):** +- No published zero-retention policy +- Data retained according to Google Antigravity Privacy Policy +- Users may request data export within 30 days of termination + +**Anticipated Enterprise:** +- Google Workspace enterprise customers typically get: + - Enterprise-grade security and compliance + - Data remains private + - No sharing with third parties for training + +**Data Location:** +- Processed within Google Cloud infrastructure +- Enterprise may have data residency options + +**Important Limitation:** +> "There's no SOC 2 report yet; Google promises compliance at GA, but currently lacks IP indemnification and zero-retention guarantees." + +--- + +## IP and Confidentiality Concerns + +**Intellectual Property:** +- **Code Ownership**: Users own generated code +- **No Guarantee**: Google does not guarantee code is free from third-party rights +- **User Responsibility**: Users responsible for license compliance + +**Confidentiality Risks:** + +**Security Warnings in Terms of Service:** +> "Security is an issue, and the terms of use warn that 'Antigravity is known to have certain security limitations.' Risks identified include data exfiltration and code execution." + +**Agent Access Concerns:** +> "By default, Antigravity agents have autonomous access to your terminal, filesystem, and network connections." + +**Execution Modes:** +1. **Automatic execution**: Agents run all commands without approval +2. **Agent-assisted** (default): AI decides when to ask for approval +3. **Require approval**: Every command needs human authorization + +**Recommendation for Enterprise:** +> "For clients in regulated industries (Healthcare, Finance), Antigravity should be treated as a public cloud environment. Do not use it for codebases containing PII or hardcoded secrets until a formal BAA covers the tool." + +**Compliance Status:** +- No SOC 2 report yet (promised at GA) +- No IP indemnification currently +- Google Cloud infrastructure (inherits some certifications) + +--- + +## Gemini Quota Relationship + +### Key Question: Are Antigravity and Gemini Limits Shared? + +**Finding: PARTIALLY SHARED / COMPLEX** + +**Evidence:** +> "The quota for Gemini 2.5 Pro got merged with the quota for Gemini 3 Pro, creating a single shared quota for both models - if 50% is used with Gemini 3 Pro, only 50% remains for both models." + +**Separate Systems:** +- Antigravity uses "work done" based limits (task complexity) +- Gemini app uses prompt-based limits +- Gemini Code Assist agent mode and CLI quotas are combined + +**Rate Limit Refresh:** +| Tier | Antigravity Refresh | Gemini App | +|------|-------------------|------------| +| Free | Weekly | Daily | +| AI Pro | Every 5 hours | Higher daily | +| AI Ultra | Every 5 hours (highest) | Highest daily | + +**Bottom Line:** +- Google AI Pro/Ultra subscription improves both Antigravity AND Gemini limits +- Quotas appear to be tracked separately but subscription tier affects both +- Heavy Antigravity usage does NOT appear to drain Gemini app quota directly +- However, some model-level quotas (Gemini 3 Pro) may be shared across products + +**Sources:** +- https://blog.google/feed/new-antigravity-rate-limits-pro-ultra-subsribers/ +- https://github.com/google-gemini/gemini-cli/issues/15248 +- https://developers.google.com/gemini-code-assist/resources/quotas + +--- + +## Free Tier Details + +**Public Preview (Free)** + +**Features Included:** +- Full Antigravity IDE access +- Gemini 3 Pro model +- Unlimited tab code completions +- Agent Manager +- Browser integration +- Manager view for multi-agent workflows +- Artifacts (task lists, implementation plans, screenshots) + +**Limits:** +- Weekly-based rate limit (refreshes once per week) +- Limits based on "work done" not request count +- Complex tasks consume more quota +- No published specific limits + +**Privacy Concerns:** +- No zero-retention guarantee +- Code may be used for improvement +- Telemetry toggle doesn't prevent code transmission + +--- + +## Paid Plans (Current via Google AI Subscription) + +### Google AI Pro +**Price:** $19.99/month + +**Antigravity Benefits:** +- Priority access +- Highest rate limits +- 5-hour refresh cycle (vs weekly for free) +- Code not used for training public models + +**Gemini Benefits:** +- Higher Gemini app limits +- Access to Gemini 3 Pro/Ultra models + +--- + +### Google AI Ultra +**Price:** $24.99/month (previously $249.99/year) + +**Antigravity Benefits:** +- 20x higher limits than Pro +- Highest priority access +- 5-hour refresh cycle +- Multi-agent workflows at scale +- Code not used for training + +**Gemini Benefits:** +- Highest Gemini limits +- All Gemini features + +--- + +### Anticipated Pricing (2026) + +| Plan | Expected Price | Features | +|------|---------------|----------| +| Individual | Free | Rate limited | +| Pro | ~$20/month | Higher limits, priority | +| Team | TBD | SSO, admin controls | +| Enterprise | ~$40-60/user/month | Data residency, compliance | + +**Pricing Model Shift:** +> "We are no longer paying for 'code completion.' We are paying for 'autonomous hours.'" + +**Compute Credits:** +- Intensive tasks may require additional "High-Density Compute Credits" +- Similar to Claude's usage-based approach + +--- + +## Usage Limits + +### How Limits Work: +> "Google Antigravity usage limits aren't calculated by the number of requests or lines of code written. Instead, the rate limits are based on 'work done' by Antigravity's AI agent." + +### Rate Limit Details: +- **Simple tasks**: Lower quota consumption +- **Complex reasoning/refactoring**: Higher quota consumption +- **Multi-file operations**: Highest consumption + +### Refresh Cycles: +- **Free**: Weekly refresh +- **AI Pro/Ultra**: 5-hour refresh + +### Known Issues: +- Some users report 4-7 day lockouts due to bugs +- Cannot query remaining quota directly from agent +- EU region users may experience lower quotas + +--- + +## Summary Recommendation + +### Current Assessment (Public Preview): + +**Strengths:** +- Powerful agentic AI development capabilities +- Free during preview period +- Manager view for multi-agent workflows +- Built on proven Gemini 3 models +- Google Cloud infrastructure + +**Critical Concerns:** +- **No SOC 2 certification yet** +- **No zero-retention guarantee** +- **Code may be used for training (unclear policy)** +- **Security limitations acknowledged in ToS** +- **No IP indemnification** +- **Autonomous agent access to filesystem/terminal** + +### Recommendation Matrix: + +| Use Case | Recommendation | Reason | +|----------|---------------|--------| +| Personal projects | ✅ OK to use | Low risk, free | +| Learning/experimentation | ✅ OK to use | No sensitive data | +| New hackathon projects | ⚠️ Use with caution | No existing IP at risk | +| Enterprise codebase | 🚨 NOT RECOMMENDED | No compliance guarantees | +| Regulated industry | 🚨 DO NOT USE | No BAA, no SOC 2 | +| Sensitive/classified code | 🚨 DO NOT USE | Security limitations | + +### For Enterprise Adoption: + +**Wait for GA Release (2026) when Google provides:** +- [ ] SOC 2 Type II certification +- [ ] Zero data retention option +- [ ] Clear no-training policy +- [ ] IP indemnification +- [ ] BAA for healthcare +- [ ] Enterprise terms of service + +**Current Alternative:** Use **Google Gemini Code Assist** (Standard/Enterprise) which has established enterprise protections, or **Continue.dev** for air-gapped requirements. + +--- + +## Comparison: Antigravity vs Gemini Code Assist + +| Feature | Antigravity | Gemini Code Assist | +|---------|------------|-------------------| +| Maturity | Public Preview | Production | +| SOC 2 | Not yet | Yes | +| Enterprise Terms | Coming 2026 | Available | +| Training Policy | Ambiguous | Clear no-training | +| Pricing | Free (preview) | $19-47/user/month | +| Agent Capabilities | Full agentic | Limited | + +--- + +## Verification Checklist + +Before implementing, verify the following: + +- [ ] Current preview terms at https://antigravity.google/terms +- [ ] GA release date and enterprise features +- [ ] SOC 2 certification status +- [ ] Training policy clarification +- [ ] Enterprise pricing when available +- [ ] Data residency options + +--- + +**Last Updated:** February 2, 2026 + +**Additional Sources:** +- [Google Developers Blog - Antigravity Announcement](https://developers.googleblog.com/build-with-google-antigravity-our-new-agentic-development-platform/) +- [Google AI Subscription Features](https://9to5google.com/2026/01/16/google-ai-pro-ultra-features/) +- [Antigravity Rate Limits](https://blog.google/feed/new-antigravity-rate-limits-pro-ultra-subsribers/) + +**Note:** Google Antigravity is in Public Preview. Policies, features, and pricing will change at GA release. Enterprise users should wait for official enterprise tier before processing sensitive code.