domnitor/logs/README.md

Logs Directory

This directory contains detailed application logs for debugging and monitoring purposes.

Log Files

TLD Import Logs (tld_import_YYYY-MM-DD.log)

Comprehensive logs for TLD registry import operations including:

• Start/End Operations: Marked with === START: and === END: separators
• Batch Processing: Each batch's start time, TLDs processed, and duration
• Individual TLD Processing:
  • TLD name and ID
  • Fetch time (in milliseconds)
  • Database update time
  • Data found (WHOIS server, registry URL, dates)
  • Success/failure status
  • Error messages with exception details
• Progress Tracking:
  • Current step in workflow
  • Last processed ID
  • Completion percentage
  • Remaining TLDs
• Statistics:
  • Batch processing time
  • Average time per TLD
  • Success/failure counts
  • Overall progress

Cron Job Logs (cron.log)

Logs from automated domain checking cron jobs (created by cron/check_domains.php).

Log Levels

• DEBUG: Detailed diagnostic information
• INFO: General informational messages (normal operations)
• WARNING: Warning messages (non-critical issues)
• ERROR: Error messages (failures that don't stop execution)
• CRITICAL: Critical errors (failures that stop execution)

Log Format

[YYYY-MM-DD HH:MM:SS] [LEVEL] Message | Context: {"key":"value"}

Example:

[2025-10-08 10:04:23] [INFO] Processing TLD [1/50]: .aaa (ID: 1)
[2025-10-08 10:04:23] [INFO] TLD .aaa: SUCCESS - Found WHOIS server, registry URL | Context: {"fetch_time_ms":245.67,"update_time_ms":12.34,"data_fields":2}

Troubleshooting TLD Import Issues

Issue: Import Stuck at Same Progress

Check the logs for:

1. Look for "last_processed_id" - is it advancing?
2. Check for FAILED messages - which TLDs are failing?
3. Look at "batch_time_seconds" - is it taking too long?

Example log analysis:

# Find failed TLDs
grep "FAILED" logs/tld_import_2025-10-08.log

# Check last processed IDs
grep "Updating last processed ID" logs/tld_import_2025-10-08.log

# Find slow TLDs (over 5 seconds)
grep "fetch_time_ms" logs/tld_import_2025-10-08.log | grep -E '"fetch_time_ms":[5-9][0-9]{3}|[0-9]{5,}'

Issue: FastCGI Timeout

Symptoms in logs:

• Last log entry shows a TLD being processed
• No "Batch statistics" or "END:" marker
• Apache/Nginx error log shows "Connection reset by peer"

Solutions:

1. Reduce batch size in TldRegistryService.php:

   $tldsNeedingWhois = $this->getTldsNeedingWhoisData(25, $lastProcessedId); // Reduced from 50
   

2. Increase PHP/FastCGI timeout:

   ; php.ini
   max_execution_time = 300
   

Issue: Repeated Failures on Same TLDs

Check logs for:

# Find TLDs that consistently fail
grep "FAILED" logs/tld_import_*.log | sort | uniq -c | sort -rn

Common causes:

• TLD doesn't have IANA RDAP/WHOIS data
• Network timeout to IANA servers
• Invalid TLD format

Analyzing Performance

Average Processing Time

# Get average fetch time per TLD
grep "fetch_time_ms" logs/tld_import_2025-10-08.log | grep -oP '"fetch_time_ms":\K[0-9.]+' | awk '{sum+=$1; n++} END {print "Average: " sum/n " ms"}'

Slowest TLDs

# Find slowest 10 TLDs
grep "fetch_time_ms" logs/tld_import_2025-10-08.log | grep -oP 'TLD \.\w+.*fetch_time_ms":\K[0-9.]+' | sort -rn | head -10

Batch Performance

# Show all batch statistics
grep "Batch statistics" logs/tld_import_2025-10-08.log

Log Retention

Logs are organized by date and are not automatically deleted. You may want to:

1. Manual cleanup: Delete old logs periodically

   find logs/ -name "*.log" -mtime +30 -delete  # Delete logs older than 30 days
   

2. Set up logrotate (Linux):

   /path/to/Domain Monitor/logs/*.log {
       daily
       rotate 30
       compress
       missingok
       notifempty
   }
   

Accessing Logs

Via Command Line

# View latest TLD import log
tail -f logs/tld_import_$(date +%Y-%m-%d).log

# View last 100 lines
tail -100 logs/tld_import_2025-10-08.log

# Search for errors
grep ERROR logs/tld_import_2025-10-08.log

# Watch for specific TLD
grep ".example" logs/tld_import_2025-10-08.log

Via PHP

The Logger class provides a tail() method:

$logger = new Logger('tld_import');
$lastLines = $logger->tail(100); // Get last 100 lines

Important Notes

• Log files grow large: A complete TLD import (~1500 TLDs) generates ~2-5 MB of logs
• Context data is JSON: Can be parsed programmatically for analysis
• Timestamps are in server timezone: Check your PHP timezone setting
• Logs are NOT web-accessible: The .htaccess in the project root blocks access to this directory

Related Files

• app/Services/Logger.php - Logger implementation
• app/Services/TldRegistryService.php - Uses logger for TLD import operations
• cron/check_domains.php - Uses file_put_contents for cron.log

Support

If you're experiencing import issues:

1. Check the relevant log file for error messages
2. Look for patterns in failed TLDs
3. Check Apache/Nginx error logs for timeout issues
4. Verify network connectivity to IANA servers
5. Consider reducing batch size for slower servers