LingvAI/README.md

# LingvAI

<img src="public/logo.svg" width="128" align="right">

[![License](https://img.shields.io/github/license/thedaviddelta/lingva-translate)](./LICENSE)

**LingvAI** is an enhanced fork of [Lingva Translate](https://github.com/thedaviddelta/lingva-translate) — a privacy-respecting alternative front-end for Google Translate — extended with **Replicate AI-powered translation**, an **admin panel**, and **document translation** (PDF, Word, Excel, CSV) with full formatting preservation.

---

## Features

### Core (from Lingva Translate)
- 100+ languages via Google Translate scraper (no tracking)
- Audio playback for source and translated text
- Auto-translate mode
- GraphQL and REST API
- PWA support (installable)
- Dark/light mode

### New in LingvAI

#### Admin Panel (`/admin`)
- Password-protected settings dashboard (gear icon in header)
- Configure Replicate API token and JigsawStack API key
- Select/change the Replicate model version
- Enable/disable AI translation per-instance
- Test translation button
- Change admin password
- Settings stored server-side in `data/settings.json` (never committed)

#### Replicate AI Translation
- When enabled in admin, uses [Replicate](https://replicate.com) + [JigsawStack](https://jigsawstack.com) text-translate model
- Replaces Google Translate scraper with AI translation when active
- Falls back to original lingva-scraper when Replicate is disabled
- Batch translation with separator trick for efficiency

#### Document Translation (new "Document" tab)
Translate whole documents while preserving original formatting:

| Format | Formatting | Notes |
|--------|-----------|-------|
| `.xlsx` / `.xls` | **Fully preserved** | Cell styles, formulas, column widths intact. Select which columns to translate. |
| `.docx` | **Fully preserved** | Fonts, tables, images, paragraph styles preserved via XML manipulation |
| `.csv` | Structure preserved | Column selection supported |
| `.pdf` | Best-effort | Text extracted, translated, new formatted PDF generated |

- Drag-and-drop or click-to-upload (up to 50 MB)
- **Column selector** for Excel/CSV: choose individual columns, use All/None toggles per sheet
- Download translated file (named `original_<lang>.ext`)

---

## Getting Started

### Prerequisites
- Node.js 16+
- npm

### Installation

```bash
git clone https://devops.cloudhost.es/CloudHost/LingvAI.git
cd LingvAI
npm install
```

### Environment Variables

Create a `.env.local` file:

```env
# Admin panel
ADMIN_PASSWORD=your_secure_password       # Default: admin
ADMIN_JWT_SECRET=random_secret_string     # Used to sign admin session tokens

# Replicate AI (optional - can also be set via admin panel)
REPLICATE_API_TOKEN=r8_...
JIGSAWSTACK_API_KEY=sk_...

# Optional: override default languages
NEXT_PUBLIC_DEFAULT_SOURCE_LANG=auto
NEXT_PUBLIC_DEFAULT_TARGET_LANG=en
```

### Running

```bash
# Development
npm run dev

# Production
npm run build
npm start
```

The app runs on [http://localhost:3000](http://localhost:3000) by default.

---

## Configuration

### Setting up Replicate AI Translation

1. Open the app and click the **gear icon** (⚙) in the top-right header
2. Log in with your admin password (default: `admin`)
3. Enter your **Replicate API token** — get one at [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens)
4. Enter your **JigsawStack API key** — get one at [jigsawstack.com](https://jigsawstack.com)
5. Optionally change the **model version** (default is the JigsawStack text-translate model)
6. Toggle **Enable Replicate Translation** on
7. Click **Save Settings**
8. Use **Test Translation** to verify it works

Default model:
```
jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89
```

### Replicate API call format

```bash
curl -X POST \
  -H "Authorization: Bearer $REPLICATE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Prefer: wait" \
  -d '{
    "version": "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89",
    "input": {
      "text": "Hello, world!",
      "api_key": "<jigsawstack_api_key>",
      "target_language": "es"
    }
  }' \
  https://api.replicate.com/v1/predictions
```

---

## API Endpoints

### Original REST API
```
GET /api/v1/:source/:target/:query   → { translation, info }
GET /api/v1/audio/:lang/:text        → { audio: number[] }
GET /api/v1/languages                → { languages }
```

### GraphQL
```
POST /api/graphql
```

### New Endpoints

```
POST /api/translate/replicate
  Body: { text: string, targetLanguage: string }
  Returns: { translation: string }

POST /api/translate/document
  Body: multipart/form-data
    file: <file>
    targetLanguage: string
    action: "translate" | "getColumns"
    columnSelections?: JSON string (for Excel/CSV)
  Returns: file download (translate) or { columns } (getColumns)

GET  /api/admin/auth      → { authenticated: boolean }
POST /api/admin/auth      body: { password } → sets session cookie
DELETE /api/admin/auth    → clears session cookie

GET  /api/admin/settings  → { replicateApiToken, jigsawApiKey, modelVersion, replicateEnabled }
POST /api/admin/settings  body: { ...settings, newPassword? }
```

---

## Architecture

```
pages/
  [[...slug]].tsx          Main translation page (Text + Document tabs)
  admin/index.tsx          Admin settings panel
  api/
    admin/
      auth.ts              JWT-based admin authentication
      settings.ts          Settings read/write (requires admin auth)
    translate/
      replicate.ts         Replicate AI translation endpoint
      document.ts          Document upload & translation endpoint
    v1/[[...slug]].ts      Original REST API
    graphql.ts             Original GraphQL API

components/
  DocumentTranslator.tsx   File upload UI, progress, download
  ColumnSelector.tsx       Per-sheet column selection for Excel/CSV
  Header.tsx               + admin gear icon link

utils/
  settings-store.ts        Read/write data/settings.json
  admin-auth.ts            JWT sign/verify helpers
  replicate-translate.ts   Replicate API calls + batch helper
  document-processors/
    excel.ts               SheetJS Excel/CSV processor
    docx.ts                JSZip + XML DOCX processor
    pdf.ts                 pdf-parse + pdf-lib PDF processor
```

---

## Docker

```dockerfile
FROM node:18-alpine
WORKDIR /app
COPY . .
RUN npm install && npm run build
EXPOSE 3000
CMD ["npm", "start"]
```

Or using the included `Dockerfile`:
```bash
docker build -t lingvai .
docker run -p 3000:3000 \
  -e ADMIN_PASSWORD=secret \
  -e ADMIN_JWT_SECRET=random \
  -v ./data:/app/data \
  lingvai
```

> Mount `./data` as a volume to persist admin settings across container restarts.

---

## Tech Stack

| Layer | Technology |
|-------|-----------|
| Framework | Next.js 12, React 18, TypeScript |
| UI | Chakra UI 2, Framer Motion |
| Translation (default) | lingva-scraper (Google Translate) |
| Translation (AI) | Replicate + JigsawStack text-translate |
| Document processing | SheetJS (xlsx), JSZip, pdf-lib, pdf-parse |
| Admin auth | jose (JWT), HTTP-only cookie |
| File uploads | formidable v3 |
| API | REST + GraphQL (Apollo Server) |

---

## License

[AGPL-3.0](./LICENSE) — same as the upstream Lingva Translate project.

Original project by [thedaviddelta](https://github.com/thedaviddelta/lingva-translate).
LingvAI enhancements: admin panel, Replicate AI integration, document translation.