# ✅ Deployment Checklist - APK Scraper

**Version:** 2.0  
**Date:** December 3, 2025  
**Status:** Ready for Production

---

## 📋 Pre-Deployment

- [ ] **composer.json updated**
  - Verify Goutte and DomCrawler are in requires
  - Run: `composer install`

- [ ] **Dependencies installed**
  - Check: `vendor/autoload.php` exists
  - Check: `vendor/fabpot/goutte` exists
  - Check: `vendor/symfony/dom-crawler` exists

- [ ] **Code reviewed**
  - [ ] `app/Services/MultiSourceScraper.php` (v2.0)
  - [ ] `app/Services/HTMLParseHelper.php`
  - [ ] `app/Http/Controllers/APKImportController.php`

- [ ] **Documentation read**
  - [ ] SCRAPER_QUICKSTART.md
  - [ ] SCRAPER_IMPLEMENTATION_GUIDE.md
  - [ ] SCRAPER_ROUTES_EXAMPLE.php

---

## 🚀 Local Development

- [ ] **Test basic scraping**
  ```bash
  php artisan tinker
  > MultiSourceScraper::scrape('github', 'mozilla-mobile/fenix')
  ```

- [ ] **Test validation**
  ```bash
  > MultiSourceScraper::validate(['package_name' => 'invalid'])
  ```

- [ ] **Test all sources**
  - [ ] GitHub
  - [ ] APKPure
  - [ ] APKMirror

- [ ] **Check error handling**
  - Test invalid query
  - Test network timeout
  - Test malformed data

---

## 🔧 Configuration

- [ ] **Set timezone** in `.env`
  ```
  APP_TIMEZONE=UTC
  ```

- [ ] **Enable logging** in `config/logging.php`
  ```php
  'default' => env('LOG_CHANNEL', 'stack'),
  ```

- [ ] **Optional: GitHub token** (for higher rate limit)
  ```
  GITHUB_TOKEN=ghp_xxxxx
  ```

- [ ] **Set timeout appropriately**
  - For slow hosting: `private const TIMEOUT = 30;`
  - For standard hosting: `private const TIMEOUT = 15;` (default)

---

## 🛣️ Routes

- [ ] **Add routes to `routes/api.php`** (recommended)
  ```php
  Route::middleware(['auth:sanctum', 'admin'])->prefix('apk')->group(function () {
      Route::post('/search', [APKImportController::class, 'search']);
      Route::post('/import', [APKImportController::class, 'import']);
      Route::post('/bulk-import', [APKImportController::class, 'bulkImport']);
      Route::get('/sources', [APKImportController::class, 'sources']);
  });
  ```

- [ ] **Or add to `routes/platform.php`** (for admin panel)
  ```php
  Route::middleware(['auth', 'admin'])->prefix('admin')->group(function () {
      Route::post('/apk-search', [APKImportController::class, 'search']);
      // ... other routes
  });
  ```

- [ ] **Test routes**
  ```bash
  php artisan route:list | grep apk
  ```

---

## 🔐 Security

- [ ] **Middleware configured**
  - [ ] Authentication required (auth:sanctum or auth)
  - [ ] Admin-only access (admin middleware)
  - [ ] CSRF protection enabled

- [ ] **Input validation**
  - [ ] Request validation in controller
  - [ ] Package name format checked
  - [ ] URL format validated
  - [ ] File size limits enforced

- [ ] **Rate limiting** (optional)
  - [ ] Add throttle middleware if needed
  - [ ] Configure API rate limits

- [ ] **Logging enabled**
  - [ ] All imports logged
  - [ ] All errors logged
  - [ ] Verify logs write to `storage/logs/`

---

## 📊 Database

- [ ] **App model exists** (`app/Models/App.php`)
  ```php
  $app = App::create($appData);
  ```

- [ ] **Required fields in table**
  - [ ] package_name (unique)
  - [ ] name
  - [ ] developer
  - [ ] download_url
  - [ ] latest_version

- [ ] **Optional fields**
  - [ ] description
  - [ ] icon_url
  - [ ] size
  - [ ] rating
  - [ ] source
  - [ ] source_url

---

## 🧪 Testing

- [ ] **Test GitHub scraping**
  ```bash
  curl -X POST http://localhost:8000/api/apk/search \
    -H "Content-Type: application/json" \
    -d '{"source":"github","query":"mozilla-mobile/fenix"}'
  ```

- [ ] **Test APKPure scraping**
  ```bash
  curl -X POST http://localhost:8000/api/apk/search \
    -H "Content-Type: application/json" \
    -d '{"source":"apkpure","query":"telegram"}'
  ```

- [ ] **Test APKMirror scraping**
  ```bash
  curl -X POST http://localhost:8000/api/apk/search \
    -H "Content-Type: application/json" \
    -d '{"source":"apkmirror","query":"whatsapp"}'
  ```

- [ ] **Test import endpoint**
  ```bash
  curl -X POST http://localhost:8000/api/apk/import \
    -H "Content-Type: application/json" \
    -d '{
      "package_name":"com.example.app",
      "name":"Example",
      "developer":"Dev",
      "download_url":"https://example.com/app.apk",
      "latest_version":"1.0"
    }'
  ```

- [ ] **Test validation**
  - [ ] Invalid package name rejected
  - [ ] Missing fields rejected
  - [ ] Non-.apk URLs rejected
  - [ ] Duplicate package_name rejected

- [ ] **Test error scenarios**
  - [ ] Network timeout handled
  - [ ] Invalid HTML handled
  - [ ] Empty results handled
  - [ ] Database errors handled

---

## 📁 File Permissions

- [ ] **Storage directory writable**
  ```bash
  chmod -R 755 storage/
  chmod -R 755 bootstrap/cache/
  ```

- [ ] **Log directory exists**
  ```bash
  mkdir -p storage/logs
  touch storage/logs/laravel.log
  chmod 666 storage/logs/laravel.log
  ```

---

## 🔍 Logging & Monitoring

- [ ] **Logs configured** in `config/logging.php`
  ```php
  'channels' => [
      'single' => [
          'driver' => 'single',
          'path' => storage_path('logs/laravel.log'),
      ],
  ],
  ```

- [ ] **Monitor logs during testing**
  ```bash
  tail -f storage/logs/laravel.log
  ```

- [ ] **Check for errors**
  - [ ] No "Goutte not found" errors
  - [ ] No timeout errors (adjust if needed)
  - [ ] No database errors
  - [ ] No validation errors (except expected)

---

## 🌐 Shared Hosting Specifics

- [ ] **PHP version** >= 8.2
  - Check: `php -v`

- [ ] **Extensions installed**
  - [ ] curl
  - [ ] json
  - [ ] mbstring
  - [ ] openssl

- [ ] **composer available**
  - Check: `php composer.phar --version`
  - Or: `composer --version`

- [ ] **Disk space available**
  - Need ~10MB for vendor dependencies

- [ ] **Memory limit** >= 256MB
  - Check: `php -r "echo ini_get('memory_limit');"`

---

## 🚀 Deployment

- [ ] **Push to staging**
  ```bash
  git add .
  git commit -m "Add APK scraper v2.0 (Goutte)"
  git push origin develop
  ```

- [ ] **Deploy to staging**
  ```bash
  composer install
  php artisan migrate (if needed)
  php artisan config:cache
  ```

- [ ] **Test on staging**
  - [ ] Run all tests above
  - [ ] Test with real data
  - [ ] Monitor logs
  - [ ] Check performance

- [ ] **Deploy to production**
  ```bash
  git push origin main
  # Deployment script runs
  composer install
  php artisan cache:clear
  php artisan config:cache
  ```

- [ ] **Verify production**
  - [ ] Routes working
  - [ ] API endpoints responding
  - [ ] Logs being written
  - [ ] No errors in logs

---

## 📈 Performance Monitoring

- [ ] **Monitor response times**
  - GitHub: 2-5 seconds
  - APKPure: 5-10 seconds
  - APKMirror: 5-10 seconds

- [ ] **Monitor error rates**
  - Check logs for exceptions
  - Track validation failures
  - Monitor timeout errors

- [ ] **Monitor disk usage**
  - Check storage/logs size
  - Archive old logs if needed

- [ ] **Monitor database**
  - Track app import count
  - Watch for duplicate package_names
  - Monitor query performance

---

## 🔄 Post-Deployment

- [ ] **Create backup**
  ```bash
  # Database
  mysqldump -u user -p database > backup-$(date +%Y%m%d).sql
  ```

- [ ] **Document changes**
  - [ ] Add to CHANGELOG.md
  - [ ] Update README.md
  - [ ] Notify team

- [ ] **Monitor for 24 hours**
  - [ ] Check logs regularly
  - [ ] Monitor error rates
  - [ ] Verify data quality
  - [ ] Check performance metrics

- [ ] **Set up alerting** (optional)
  - [ ] Log errors to Sentry
  - [ ] Email on critical errors
  - [ ] Slack notifications

---

## 🔧 Troubleshooting During Deployment

| Issue | Solution |
|-------|----------|
| "Goutte not found" | Run: `composer install` |
| "Class not found" | Clear cache: `php artisan cache:clear` |
| Timeout errors | Increase `TIMEOUT` constant (15 → 30) |
| 403 Forbidden | Try different source or add delay |
| Database errors | Check table structure, run migrations |
| Permission denied | Fix permissions: `chmod 755 storage/` |
| Memory limit exceeded | Increase PHP memory_limit in .ini |

---

## ✅ Final Sign-Off

- [ ] **All code changes tested**
- [ ] **All routes working**
- [ ] **All APIs responding**
- [ ] **All errors handled**
- [ ] **Logs verified**
- [ ] **Performance acceptable**
- [ ] **Security validated**
- [ ] **Documentation updated**
- [ ] **Team notified**
- [ ] **Ready for production** ✓

---

## 📞 Support Contacts

- **Developer:** [Your Name]
- **DevOps:** [Your DevOps]
- **Project Lead:** [Your Project Lead]

**Deployment Date:** _______________  
**Deployed By:** _______________  
**Verified By:** _______________

---

## 📚 Documentation References

- SCRAPER_QUICKSTART.md
- SCRAPER_IMPLEMENTATION_GUIDE.md
- SCRAPER_ROUTES_EXAMPLE.php
- SCRAPER_REFERENCE_CARD.md

---

## 🎉 Deployment Complete!

Once all items are checked, the APK scraper is ready for production.

**Status:** ☐ Not Started | ☐ In Progress | ☑️ Complete

---

**Version:** 2.0  
**Date:** December 3, 2025  
**Last Updated:** [Date]
