Fixed Encryption key generation and updated Readme
This commit is contained in:
Hosteroid
@@ -1,60 +0,0 @@
|
||||
# Quick Start: Using the TLD Import Logs
|
||||
|
||||
## 📁 Log File Location
|
||||
|
||||
Your TLD import logs are here:
|
||||
```
|
||||
logs/tld_import_2025-10-08.log (today's log)
|
||||
```
|
||||
|
||||
## 🔍 Quick Commands
|
||||
|
||||
### Watch import in real-time (Windows PowerShell):
|
||||
```powershell
|
||||
Get-Content logs\tld_import_2025-10-08.log -Wait -Tail 50
|
||||
```
|
||||
|
||||
### Check for errors:
|
||||
```powershell
|
||||
Select-String -Path logs\tld_import_*.log -Pattern "ERROR|CRITICAL"
|
||||
```
|
||||
|
||||
### Find last processed TLD:
|
||||
```powershell
|
||||
Select-String -Path logs\tld_import_2025-10-08.log -Pattern "Processing TLD" | Select-Object -Last 1
|
||||
```
|
||||
|
||||
### Check completion status:
|
||||
```powershell
|
||||
Select-String -Path logs\tld_import_2025-10-08.log -Pattern "Batch statistics|complete"
|
||||
```
|
||||
|
||||
## 🚨 If Import Fails
|
||||
|
||||
1. **Check the log file** for the date of your import
|
||||
2. **Look for the last line** - it should show which TLD was being processed
|
||||
3. **Search for "ERROR" or "FAILED"** to find problems
|
||||
4. **Check "last_processed_id"** to see where it stopped
|
||||
|
||||
## 📊 What the Logs Show
|
||||
|
||||
- ✅ Each TLD being processed with ID
|
||||
- ✅ Time taken per TLD (in milliseconds)
|
||||
- ✅ Success/failure status
|
||||
- ✅ Data found (WHOIS server, registry URL, dates)
|
||||
- ✅ Progress tracking (processed/remaining)
|
||||
- ✅ Batch statistics (total time, average per TLD)
|
||||
- ✅ Detailed error messages with context
|
||||
|
||||
## 🔧 Next Steps After Your Import
|
||||
|
||||
1. **Run your import again** - the logging is now active
|
||||
2. **Watch the logs** while it runs
|
||||
3. **If it times out** - check the log to see where
|
||||
4. **Share the log file** if you need help debugging
|
||||
|
||||
## 📖 Full Documentation
|
||||
|
||||
See `logs/README.md` for complete documentation and troubleshooting guide.
|
||||
See `LOGGING_SYSTEM.md` for technical details about the implementation.
|
||||
|
||||
193
logs/README.md
193
logs/README.md
@@ -1,193 +0,0 @@
|
||||
# Logs Directory
|
||||
|
||||
This directory contains detailed application logs for debugging and monitoring purposes.
|
||||
|
||||
## Log Files
|
||||
|
||||
### TLD Import Logs (`tld_import_YYYY-MM-DD.log`)
|
||||
|
||||
Comprehensive logs for TLD registry import operations including:
|
||||
|
||||
- **Start/End Operations**: Marked with `=== START:` and `=== END:` separators
|
||||
- **Batch Processing**: Each batch's start time, TLDs processed, and duration
|
||||
- **Individual TLD Processing**:
|
||||
- TLD name and ID
|
||||
- Fetch time (in milliseconds)
|
||||
- Database update time
|
||||
- Data found (WHOIS server, registry URL, dates)
|
||||
- Success/failure status
|
||||
- Error messages with exception details
|
||||
- **Progress Tracking**:
|
||||
- Current step in workflow
|
||||
- Last processed ID
|
||||
- Completion percentage
|
||||
- Remaining TLDs
|
||||
- **Statistics**:
|
||||
- Batch processing time
|
||||
- Average time per TLD
|
||||
- Success/failure counts
|
||||
- Overall progress
|
||||
|
||||
### Cron Job Logs (`cron.log`)
|
||||
|
||||
Logs from automated domain checking cron jobs (created by `cron/check_domains.php`).
|
||||
|
||||
## Log Levels
|
||||
|
||||
- **DEBUG**: Detailed diagnostic information
|
||||
- **INFO**: General informational messages (normal operations)
|
||||
- **WARNING**: Warning messages (non-critical issues)
|
||||
- **ERROR**: Error messages (failures that don't stop execution)
|
||||
- **CRITICAL**: Critical errors (failures that stop execution)
|
||||
|
||||
## Log Format
|
||||
|
||||
```
|
||||
[YYYY-MM-DD HH:MM:SS] [LEVEL] Message | Context: {"key":"value"}
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
[2025-10-08 10:04:23] [INFO] Processing TLD [1/50]: .aaa (ID: 1)
|
||||
[2025-10-08 10:04:23] [INFO] TLD .aaa: SUCCESS - Found WHOIS server, registry URL | Context: {"fetch_time_ms":245.67,"update_time_ms":12.34,"data_fields":2}
|
||||
```
|
||||
|
||||
## Troubleshooting TLD Import Issues
|
||||
|
||||
### Issue: Import Stuck at Same Progress
|
||||
|
||||
**Check the logs for:**
|
||||
1. Look for "last_processed_id" - is it advancing?
|
||||
2. Check for FAILED messages - which TLDs are failing?
|
||||
3. Look at "batch_time_seconds" - is it taking too long?
|
||||
|
||||
**Example log analysis:**
|
||||
```bash
|
||||
# Find failed TLDs
|
||||
grep "FAILED" logs/tld_import_2025-10-08.log
|
||||
|
||||
# Check last processed IDs
|
||||
grep "Updating last processed ID" logs/tld_import_2025-10-08.log
|
||||
|
||||
# Find slow TLDs (over 5 seconds)
|
||||
grep "fetch_time_ms" logs/tld_import_2025-10-08.log | grep -E '"fetch_time_ms":[5-9][0-9]{3}|[0-9]{5,}'
|
||||
```
|
||||
|
||||
### Issue: FastCGI Timeout
|
||||
|
||||
**Symptoms in logs:**
|
||||
- Last log entry shows a TLD being processed
|
||||
- No "Batch statistics" or "END:" marker
|
||||
- Apache/Nginx error log shows "Connection reset by peer"
|
||||
|
||||
**Solutions:**
|
||||
1. Reduce batch size in `TldRegistryService.php`:
|
||||
```php
|
||||
$tldsNeedingWhois = $this->getTldsNeedingWhoisData(25, $lastProcessedId); // Reduced from 50
|
||||
```
|
||||
|
||||
2. Increase PHP/FastCGI timeout:
|
||||
```ini
|
||||
; php.ini
|
||||
max_execution_time = 300
|
||||
```
|
||||
|
||||
### Issue: Repeated Failures on Same TLDs
|
||||
|
||||
**Check logs for:**
|
||||
```bash
|
||||
# Find TLDs that consistently fail
|
||||
grep "FAILED" logs/tld_import_*.log | sort | uniq -c | sort -rn
|
||||
```
|
||||
|
||||
**Common causes:**
|
||||
- TLD doesn't have IANA RDAP/WHOIS data
|
||||
- Network timeout to IANA servers
|
||||
- Invalid TLD format
|
||||
|
||||
## Analyzing Performance
|
||||
|
||||
### Average Processing Time
|
||||
```bash
|
||||
# Get average fetch time per TLD
|
||||
grep "fetch_time_ms" logs/tld_import_2025-10-08.log | grep -oP '"fetch_time_ms":\K[0-9.]+' | awk '{sum+=$1; n++} END {print "Average: " sum/n " ms"}'
|
||||
```
|
||||
|
||||
### Slowest TLDs
|
||||
```bash
|
||||
# Find slowest 10 TLDs
|
||||
grep "fetch_time_ms" logs/tld_import_2025-10-08.log | grep -oP 'TLD \.\w+.*fetch_time_ms":\K[0-9.]+' | sort -rn | head -10
|
||||
```
|
||||
|
||||
### Batch Performance
|
||||
```bash
|
||||
# Show all batch statistics
|
||||
grep "Batch statistics" logs/tld_import_2025-10-08.log
|
||||
```
|
||||
|
||||
## Log Retention
|
||||
|
||||
Logs are organized by date and are not automatically deleted. You may want to:
|
||||
|
||||
1. **Manual cleanup**: Delete old logs periodically
|
||||
```bash
|
||||
find logs/ -name "*.log" -mtime +30 -delete # Delete logs older than 30 days
|
||||
```
|
||||
|
||||
2. **Set up logrotate** (Linux):
|
||||
```
|
||||
/path/to/Domain Monitor/logs/*.log {
|
||||
daily
|
||||
rotate 30
|
||||
compress
|
||||
missingok
|
||||
notifempty
|
||||
}
|
||||
```
|
||||
|
||||
## Accessing Logs
|
||||
|
||||
### Via Command Line
|
||||
```bash
|
||||
# View latest TLD import log
|
||||
tail -f logs/tld_import_$(date +%Y-%m-%d).log
|
||||
|
||||
# View last 100 lines
|
||||
tail -100 logs/tld_import_2025-10-08.log
|
||||
|
||||
# Search for errors
|
||||
grep ERROR logs/tld_import_2025-10-08.log
|
||||
|
||||
# Watch for specific TLD
|
||||
grep ".example" logs/tld_import_2025-10-08.log
|
||||
```
|
||||
|
||||
### Via PHP
|
||||
The `Logger` class provides a `tail()` method:
|
||||
```php
|
||||
$logger = new Logger('tld_import');
|
||||
$lastLines = $logger->tail(100); // Get last 100 lines
|
||||
```
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **Log files grow large**: A complete TLD import (~1500 TLDs) generates ~2-5 MB of logs
|
||||
- **Context data is JSON**: Can be parsed programmatically for analysis
|
||||
- **Timestamps are in server timezone**: Check your PHP timezone setting
|
||||
- **Logs are NOT web-accessible**: The `.htaccess` in the project root blocks access to this directory
|
||||
|
||||
## Related Files
|
||||
|
||||
- `app/Services/Logger.php` - Logger implementation
|
||||
- `app/Services/TldRegistryService.php` - Uses logger for TLD import operations
|
||||
- `cron/check_domains.php` - Uses file_put_contents for cron.log
|
||||
|
||||
## Support
|
||||
|
||||
If you're experiencing import issues:
|
||||
1. Check the relevant log file for error messages
|
||||
2. Look for patterns in failed TLDs
|
||||
3. Check Apache/Nginx error logs for timeout issues
|
||||
4. Verify network connectivity to IANA servers
|
||||
5. Consider reducing batch size for slower servers
|
||||
|
||||
Reference in New Issue
Block a user