feat: add admin panel, Replicate AI translation, and document translation
- Admin panel (/admin) with JWT auth: configure Replicate API token, JigsawStack API key, model version, enable/disable AI translation, change admin password. Settings persisted in data/settings.json. - Replicate AI translation: POST /api/translate/replicate uses JigsawStack text-translate model via Replicate API. Main page switches to client-side AI translation when enabled. - Document translation tab: supports PDF, DOCX, XLSX, XLS, CSV. Excel/Word formatting fully preserved (SheetJS + JSZip XML manipulation). PDF uses pdf-parse extraction + pdf-lib reconstruction. Column selector UI for tabular data (per-sheet, All/None toggles). - Updated README with full implementation documentation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -45,3 +45,6 @@ cypress/screenshots
|
|||||||
**/public/workbox-*.js*
|
**/public/workbox-*.js*
|
||||||
**/public/sw.js*
|
**/public/sw.js*
|
||||||
**/public/worker-*.js*
|
**/public/worker-*.js*
|
||||||
|
|
||||||
|
# admin settings (contains API keys)
|
||||||
|
/data/settings.json
|
||||||
|
|||||||
395
README.md
395
README.md
@@ -1,241 +1,256 @@
|
|||||||
# Lingva Translate
|
# LingvAI
|
||||||
|
|
||||||
<img src="public/logo.svg" width="128" align="right">
|
<img src="public/logo.svg" width="128" align="right">
|
||||||
|
|
||||||
[](https://travis-ci.com/thedaviddelta/lingva-translate)
|
|
||||||
[](https://lingva.ml/)
|
|
||||||
[](https://dashboard.cypress.io/projects/qgjdyd/runs)
|
|
||||||
[](./LICENSE)
|
[](./LICENSE)
|
||||||
[](https://github.com/humanetech-community/awesome-humane-tech)
|
|
||||||
[<img src="https://www.datocms-assets.com/31049/1618983297-powered-by-vercel.svg" alt="Powered by Vercel" height="20">](https://vercel.com?utm_source=lingva-team&utm_campaign=oss)
|
|
||||||
|
|
||||||
Alternative front-end for Google Translate, serving as a Free and Open Source translator with over a hundred languages available
|
**LingvAI** is an enhanced fork of [Lingva Translate](https://github.com/thedaviddelta/lingva-translate) — a privacy-respecting alternative front-end for Google Translate — extended with **Replicate AI-powered translation**, an **admin panel**, and **document translation** (PDF, Word, Excel, CSV) with full formatting preservation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## How does it work?
|
## Features
|
||||||
|
|
||||||
Inspired by projects like [NewPipe](https://github.com/TeamNewPipe/NewPipe), [Nitter](https://github.com/zedeus/nitter), [Invidious](https://github.com/iv-org/invidious) or [Bibliogram](https://git.sr.ht/~cadence/bibliogram), *Lingva* scrapes through Google Translate and retrieves the translation without directly accessing any Google-related service, preventing them from tracking.
|
### Core (from Lingva Translate)
|
||||||
|
- 100+ languages via Google Translate scraper (no tracking)
|
||||||
|
- Audio playback for source and translated text
|
||||||
|
- Auto-translate mode
|
||||||
|
- GraphQL and REST API
|
||||||
|
- PWA support (installable)
|
||||||
|
- Dark/light mode
|
||||||
|
|
||||||
For this purpose, *Lingva* is built, among others, with the following Open Source resources:
|
### New in LingvAI
|
||||||
|
|
||||||
+ [Lingva Scraper](https://github.com/thedaviddelta/lingva-scraper), a Google Translate scraper built and maintained specifically for this project, which obtains all kind of information from this platform.
|
#### Admin Panel (`/admin`)
|
||||||
+ [TypeScript](https://www.typescriptlang.org/), the JavaScript superset, as the language.
|
- Password-protected settings dashboard (gear icon in header)
|
||||||
+ [React](https://reactjs.org/) as the main front-end framework.
|
- Configure Replicate API token and JigsawStack API key
|
||||||
+ [Next.js](https://nextjs.org/) as the complementary React framework, that provides Server-Side Rendering, Static Site Generation or serverless API endpoints.
|
- Select/change the Replicate model version
|
||||||
+ [ChakraUI](https://chakra-ui.com/) for the in-component styling.
|
- Enable/disable AI translation per-instance
|
||||||
+ [Jest](https://jestjs.io/), [Testing Library](https://testing-library.com/) & [Cypress](https://www.cypress.io/) for unit, integration & E2E testing.
|
- Test translation button
|
||||||
+ [Apollo Server](https://www.apollographql.com/docs/apollo-server/) for handling the GraphQL endpoint.
|
- Change admin password
|
||||||
+ [Inkscape](https://inkscape.org/) for designing both the logo and the banner.
|
- Settings stored server-side in `data/settings.json` (never committed)
|
||||||
|
|
||||||
|
#### Replicate AI Translation
|
||||||
|
- When enabled in admin, uses [Replicate](https://replicate.com) + [JigsawStack](https://jigsawstack.com) text-translate model
|
||||||
|
- Replaces Google Translate scraper with AI translation when active
|
||||||
|
- Falls back to original lingva-scraper when Replicate is disabled
|
||||||
|
- Batch translation with separator trick for efficiency
|
||||||
|
|
||||||
## Deployment
|
#### Document Translation (new "Document" tab)
|
||||||
|
Translate whole documents while preserving original formatting:
|
||||||
|
|
||||||
As *Lingva* is a [Next.js](https://nextjs.org/) project you can deploy your own instance anywhere Next is supported.
|
| Format | Formatting | Notes |
|
||||||
|
|--------|-----------|-------|
|
||||||
|
| `.xlsx` / `.xls` | **Fully preserved** | Cell styles, formulas, column widths intact. Select which columns to translate. |
|
||||||
|
| `.docx` | **Fully preserved** | Fonts, tables, images, paragraph styles preserved via XML manipulation |
|
||||||
|
| `.csv` | Structure preserved | Column selection supported |
|
||||||
|
| `.pdf` | Best-effort | Text extracted, translated, new formatted PDF generated |
|
||||||
|
|
||||||
The only requirement is to set an environment variable called `NEXT_PUBLIC_SITE_DOMAIN` with the domain you're deploying the instance under. This is used for the canonical URL and the meta tags.
|
- Drag-and-drop or click-to-upload (up to 50 MB)
|
||||||
|
- **Column selector** for Excel/CSV: choose individual columns, use All/None toggles per sheet
|
||||||
|
- Download translated file (named `original_<lang>.ext`)
|
||||||
|
|
||||||
Optionally, there are other environment variables available:
|
---
|
||||||
+ `NEXT_PUBLIC_FORCE_DEFAULT_THEME`: Force a certain theme over the system preference set by the user. The accepted values are `light` and `dark`.
|
|
||||||
+ `NEXT_PUBLIC_DEFAULT_SOURCE_LANG`: Set an initial *source* language instead of the default `auto`.
|
|
||||||
+ `NEXT_PUBLIC_DEFAULT_TARGET_LANG`: Set an initial *target* language instead of the default `en`.
|
|
||||||
|
|
||||||
### Docker
|
## Getting Started
|
||||||
|
|
||||||
An [official Docker image](https://hub.docker.com/r/thedaviddelta/lingva-translate) is available to ease the deployment using Compose, Kubernetes or similar technologies. Remember to also include the environment variables (simplified to `site_domain`, `force_default_theme`, `default_source_lang` and `default_target_lang`) when running the container.
|
### Prerequisites
|
||||||
|
- Node.js 16+
|
||||||
|
- npm
|
||||||
|
|
||||||
#### Docker Compose:
|
### Installation
|
||||||
|
|
||||||
```
|
|
||||||
version: '3'
|
|
||||||
|
|
||||||
services:
|
|
||||||
|
|
||||||
lingva:
|
|
||||||
container_name: lingva
|
|
||||||
image: thedaviddelta/lingva-translate:latest
|
|
||||||
restart: unless-stopped
|
|
||||||
environment:
|
|
||||||
- site_domain=lingva.ml
|
|
||||||
- force_default_theme=light
|
|
||||||
- default_source_lang=auto
|
|
||||||
- default_target_lang=en
|
|
||||||
ports:
|
|
||||||
- 3000:3000
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Docker Run
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run -p 3000:3000 -e site_domain=lingva.ml -e force_default_theme=light -e default_source_lang=auto -e default_target_lang=en thedaviddelta/lingva-translate:latest
|
git clone https://devops.cloudhost.es/CloudHost/LingvAI.git
|
||||||
|
cd LingvAI
|
||||||
|
npm install
|
||||||
```
|
```
|
||||||
|
|
||||||
### Vercel
|
### Environment Variables
|
||||||
|
|
||||||
Another easy way is to use the Next.js creators' own platform, [Vercel](https://vercel.com/), where you can deploy it for free with the following button.
|
Create a `.env.local` file:
|
||||||
|
|
||||||
[](https://vercel.com/new/git/external?repository-url=https%3A%2F%2Fgithub.com%2Fthedaviddelta%2Flingva-translate%2Ftree%2Fmain&env=NEXT_PUBLIC_SITE_DOMAIN&envDescription=Your%20domain&utm_source=lingva-team&utm_campaign=oss)
|
```env
|
||||||
|
# Admin panel
|
||||||
|
ADMIN_PASSWORD=your_secure_password # Default: admin
|
||||||
|
ADMIN_JWT_SECRET=random_secret_string # Used to sign admin session tokens
|
||||||
|
|
||||||
|
# Replicate AI (optional - can also be set via admin panel)
|
||||||
|
REPLICATE_API_TOKEN=r8_...
|
||||||
|
JIGSAWSTACK_API_KEY=sk_...
|
||||||
|
|
||||||
## Instances
|
# Optional: override default languages
|
||||||
|
NEXT_PUBLIC_DEFAULT_SOURCE_LANG=auto
|
||||||
These are the currently known *Lingva* instances. Feel free to make a Pull Request including yours (please remember to add `[skip ci]` to the last commit).
|
NEXT_PUBLIC_DEFAULT_TARGET_LANG=en
|
||||||
|
|
||||||
| Domain | Hosting | SSL Provider |
|
|
||||||
|:---------------------------------------------------------------------------:|:-----------------------------------------:|:-----------------------------------------------------------------------------------------------:|
|
|
||||||
| [lingva.ml](https://lingva.ml/) (Official) | [Vercel](https://vercel.com/) | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=lingva.ml) |
|
|
||||||
| [translate.igna.wtf](https://translate.igna.wtf/) | [Vercel](https://vercel.com/) | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.igna.wtf) |
|
|
||||||
| [translate.plausibility.cloud](https://translate.plausibility.cloud/) | [Hetzner](https://hetzner.com/) | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.plausibility.cloud) |
|
|
||||||
| [lingva.lunar.icu](https://lingva.lunar.icu/) | [Lansol](https://lansol.de/) | [Cloudflare](https://www.ssllabs.com/ssltest/analyze.html?d=lingva.lunar.icu) |
|
|
||||||
| [translate.projectsegfau.lt](https://translate.projectsegfau.lt/) | Self-hosted | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.projectsegfau.lt) |
|
|
||||||
| [translate.dr460nf1r3.org](https://translate.dr460nf1r3.org/) | [Netcup](https://netcup.eu/) | [Cloudflare](https://www.ssllabs.com/ssltest/analyze.html?d=translate.dr460nf1r3.org) |
|
|
||||||
| [lingva.garudalinux.org](https://lingva.garudalinux.org/) | [Hetzner](https://hetzner.com/) | [Cloudflare](https://www.ssllabs.com/ssltest/analyze.html?d=lingva.garudalinux.org) |
|
|
||||||
| [translate.jae.fi](https://translate.jae.fi/) | Self-hosted | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.jae.fi) |
|
|
||||||
|
|
||||||
|
|
||||||
## Public APIs
|
|
||||||
|
|
||||||
Nearly all the *Lingva* instances should supply a pair of public developer APIs: a RESTful one and a GraphQL one.
|
|
||||||
|
|
||||||
*Note: both APIs return the translation audio as a `Uint8Array` (served as `number[]` in JSON and `[Int]` in GraphQL) with the contents of the audio buffer.*
|
|
||||||
|
|
||||||
### REST API v1
|
|
||||||
|
|
||||||
+ GET `/api/v1/:source/:target/:query`
|
|
||||||
```typescript
|
|
||||||
{
|
|
||||||
translation: string
|
|
||||||
info?: TranslationInfo
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
+ GET `/api/v1/audio/:lang/:query`
|
### Running
|
||||||
```typescript
|
|
||||||
{
|
```bash
|
||||||
audio: number[]
|
# Development
|
||||||
}
|
npm run dev
|
||||||
|
|
||||||
|
# Production
|
||||||
|
npm run build
|
||||||
|
npm start
|
||||||
```
|
```
|
||||||
|
|
||||||
+ GET `/api/v1/languages/?:(source|target)`
|
The app runs on [http://localhost:3000](http://localhost:3000) by default.
|
||||||
```typescript
|
|
||||||
{
|
---
|
||||||
languages: [
|
|
||||||
{
|
## Configuration
|
||||||
code: string,
|
|
||||||
name: string
|
### Setting up Replicate AI Translation
|
||||||
}
|
|
||||||
]
|
1. Open the app and click the **gear icon** (⚙) in the top-right header
|
||||||
}
|
2. Log in with your admin password (default: `admin`)
|
||||||
|
3. Enter your **Replicate API token** — get one at [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens)
|
||||||
|
4. Enter your **JigsawStack API key** — get one at [jigsawstack.com](https://jigsawstack.com)
|
||||||
|
5. Optionally change the **model version** (default is the JigsawStack text-translate model)
|
||||||
|
6. Toggle **Enable Replicate Translation** on
|
||||||
|
7. Click **Save Settings**
|
||||||
|
8. Use **Test Translation** to verify it works
|
||||||
|
|
||||||
|
Default model:
|
||||||
|
```
|
||||||
|
jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89
|
||||||
```
|
```
|
||||||
|
|
||||||
In addition, every endpoint can return an error message with the following structure instead.
|
### Replicate API call format
|
||||||
```typescript
|
|
||||||
{
|
```bash
|
||||||
error: string
|
curl -X POST \
|
||||||
}
|
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Prefer: wait" \
|
||||||
|
-d '{
|
||||||
|
"version": "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89",
|
||||||
|
"input": {
|
||||||
|
"text": "Hello, world!",
|
||||||
|
"api_key": "<jigsawstack_api_key>",
|
||||||
|
"target_language": "es"
|
||||||
|
}
|
||||||
|
}' \
|
||||||
|
https://api.replicate.com/v1/predictions
|
||||||
```
|
```
|
||||||
|
|
||||||
### GraphQL API
|
---
|
||||||
|
|
||||||
+ `/api/graphql`
|
## API Endpoints
|
||||||
```graphql
|
|
||||||
query {
|
### Original REST API
|
||||||
translation(source: String target: String query: String!) {
|
```
|
||||||
source: {
|
GET /api/v1/:source/:target/:query → { translation, info }
|
||||||
lang: {
|
GET /api/v1/audio/:lang/:text → { audio: number[] }
|
||||||
code: String!
|
GET /api/v1/languages → { languages }
|
||||||
name: String!
|
|
||||||
}
|
|
||||||
text: String!
|
|
||||||
audio: [Int]!
|
|
||||||
detected: {
|
|
||||||
code: String
|
|
||||||
name: String
|
|
||||||
}
|
|
||||||
typo: String
|
|
||||||
pronunciation: String
|
|
||||||
definitions: {
|
|
||||||
type: String
|
|
||||||
list: {
|
|
||||||
definition: String
|
|
||||||
example: String
|
|
||||||
field: String
|
|
||||||
synonyms: [String]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
examples: [String]
|
|
||||||
similar: [String]
|
|
||||||
}
|
|
||||||
target: {
|
|
||||||
lang: {
|
|
||||||
code: String!
|
|
||||||
name: String!
|
|
||||||
}
|
|
||||||
text: String!
|
|
||||||
audio: [Int]!
|
|
||||||
pronunciation: String
|
|
||||||
extraTranslations: {
|
|
||||||
type: String
|
|
||||||
list: {
|
|
||||||
word: String
|
|
||||||
article: String
|
|
||||||
frequency: Int
|
|
||||||
meanings: [String]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
audio(lang: String! query: String!) {
|
|
||||||
lang: {
|
|
||||||
code: String!
|
|
||||||
name: String!
|
|
||||||
}
|
|
||||||
text: String!
|
|
||||||
audio: [Int]!
|
|
||||||
}
|
|
||||||
languages(type: SOURCE|TARGET) {
|
|
||||||
code: String!
|
|
||||||
name: String!
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### GraphQL
|
||||||
|
```
|
||||||
|
POST /api/graphql
|
||||||
|
```
|
||||||
|
|
||||||
## Related projects
|
### New Endpoints
|
||||||
|
|
||||||
+ [Lingva Scraper](https://github.com/thedaviddelta/lingva-scraper) - Google Translate scraper built and maintained specifically for this project
|
```
|
||||||
+ [SimplyTranslate](https://codeberg.org/SimpleWeb/SimplyTranslate-Web) - Very simple translation front-end with multi-engine support
|
POST /api/translate/replicate
|
||||||
+ [LibreTranslate](https://github.com/LibreTranslate/LibreTranslate) - FOSS translation service that uses the open [Argos](https://github.com/argosopentech/argos-translate) engine
|
Body: { text: string, targetLanguage: string }
|
||||||
+ [Lentil for Android](https://github.com/yaxarat/lingvaandroid) - Unofficial native client for Android that uses Lingva's public API
|
Returns: { translation: string }
|
||||||
+ [Arna Translate](https://github.com/MahanRahmati/translate) - Unofficial cross-platform native client that uses Lingva's public API
|
|
||||||
+ [Translate-UT](https://github.com/walking-octopus/translate-ut) - Unofficial native client for Ubuntu Touch that uses Lingva's public API
|
|
||||||
|
|
||||||
|
POST /api/translate/document
|
||||||
|
Body: multipart/form-data
|
||||||
|
file: <file>
|
||||||
|
targetLanguage: string
|
||||||
|
action: "translate" | "getColumns"
|
||||||
|
columnSelections?: JSON string (for Excel/CSV)
|
||||||
|
Returns: file download (translate) or { columns } (getColumns)
|
||||||
|
|
||||||
## Contributors
|
GET /api/admin/auth → { authenticated: boolean }
|
||||||
|
POST /api/admin/auth body: { password } → sets session cookie
|
||||||
|
DELETE /api/admin/auth → clears session cookie
|
||||||
|
|
||||||
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
|
GET /api/admin/settings → { replicateApiToken, jigsawApiKey, modelVersion, replicateEnabled }
|
||||||
|
POST /api/admin/settings body: { ...settings, newPassword? }
|
||||||
|
```
|
||||||
|
|
||||||
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
|
---
|
||||||
<!-- prettier-ignore-start -->
|
|
||||||
<!-- markdownlint-disable -->
|
|
||||||
<table>
|
|
||||||
<tr>
|
|
||||||
<td align="center"><a href="https://thedaviddelta.com/"><img src="https://avatars.githubusercontent.com/u/6679900?v=4?s=100" width="100px;" alt=""/><br /><sub><b>David</b></sub></a><br /><a href="#a11y-TheDavidDelta" title="Accessibility">️️️️♿️</a> <a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheDavidDelta" title="Code">💻</a> <a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheDavidDelta" title="Documentation">📖</a> <a href="#design-TheDavidDelta" title="Design">🎨</a> <a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheDavidDelta" title="Tests">⚠️</a></td>
|
|
||||||
<td align="center"><a href="https://github.com/mhmdanas"><img src="https://avatars.githubusercontent.com/u/32234660?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Mohammed Anas</b></sub></a><br /><a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=mhmdanas" title="Code">💻</a></td>
|
|
||||||
<td align="center"><a href="https://PussTheCat.org/"><img src="https://avatars.githubusercontent.com/u/47571719?v=4?s=100" width="100px;" alt=""/><br /><sub><b>TheFrenchGhosty</b></sub></a><br /><a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheFrenchGhosty" title="Documentation">📖</a></td>
|
|
||||||
</tr>
|
|
||||||
</table>
|
|
||||||
|
|
||||||
<!-- markdownlint-restore -->
|
## Architecture
|
||||||
<!-- prettier-ignore-end -->
|
|
||||||
|
|
||||||
<!-- ALL-CONTRIBUTORS-LIST:END -->
|
```
|
||||||
|
pages/
|
||||||
|
[[...slug]].tsx Main translation page (Text + Document tabs)
|
||||||
|
admin/index.tsx Admin settings panel
|
||||||
|
api/
|
||||||
|
admin/
|
||||||
|
auth.ts JWT-based admin authentication
|
||||||
|
settings.ts Settings read/write (requires admin auth)
|
||||||
|
translate/
|
||||||
|
replicate.ts Replicate AI translation endpoint
|
||||||
|
document.ts Document upload & translation endpoint
|
||||||
|
v1/[[...slug]].ts Original REST API
|
||||||
|
graphql.ts Original GraphQL API
|
||||||
|
|
||||||
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
|
components/
|
||||||
|
DocumentTranslator.tsx File upload UI, progress, download
|
||||||
|
ColumnSelector.tsx Per-sheet column selection for Excel/CSV
|
||||||
|
Header.tsx + admin gear icon link
|
||||||
|
|
||||||
|
utils/
|
||||||
|
settings-store.ts Read/write data/settings.json
|
||||||
|
admin-auth.ts JWT sign/verify helpers
|
||||||
|
replicate-translate.ts Replicate API calls + batch helper
|
||||||
|
document-processors/
|
||||||
|
excel.ts SheetJS Excel/CSV processor
|
||||||
|
docx.ts JSZip + XML DOCX processor
|
||||||
|
pdf.ts pdf-parse + pdf-lib PDF processor
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM node:18-alpine
|
||||||
|
WORKDIR /app
|
||||||
|
COPY . .
|
||||||
|
RUN npm install && npm run build
|
||||||
|
EXPOSE 3000
|
||||||
|
CMD ["npm", "start"]
|
||||||
|
```
|
||||||
|
|
||||||
|
Or using the included `Dockerfile`:
|
||||||
|
```bash
|
||||||
|
docker build -t lingvai .
|
||||||
|
docker run -p 3000:3000 \
|
||||||
|
-e ADMIN_PASSWORD=secret \
|
||||||
|
-e ADMIN_JWT_SECRET=random \
|
||||||
|
-v ./data:/app/data \
|
||||||
|
lingvai
|
||||||
|
```
|
||||||
|
|
||||||
|
> Mount `./data` as a volume to persist admin settings across container restarts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
|
||||||
|
| Layer | Technology |
|
||||||
|
|-------|-----------|
|
||||||
|
| Framework | Next.js 12, React 18, TypeScript |
|
||||||
|
| UI | Chakra UI 2, Framer Motion |
|
||||||
|
| Translation (default) | lingva-scraper (Google Translate) |
|
||||||
|
| Translation (AI) | Replicate + JigsawStack text-translate |
|
||||||
|
| Document processing | SheetJS (xlsx), JSZip, pdf-lib, pdf-parse |
|
||||||
|
| Admin auth | jose (JWT), HTTP-only cookie |
|
||||||
|
| File uploads | formidable v3 |
|
||||||
|
| API | REST + GraphQL (Apollo Server) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
[](https://www.gnu.org/licenses/agpl-3.0.html)
|
[AGPL-3.0](./LICENSE) — same as the upstream Lingva Translate project.
|
||||||
|
|
||||||
Copyright © 2021 [thedaviddelta](https://github.com/thedaviddelta) & contributors.
|
Original project by [thedaviddelta](https://github.com/thedaviddelta/lingva-translate).
|
||||||
This project is [GNU AGPLv3](./LICENSE) licensed.
|
LingvAI enhancements: admin panel, Replicate AI integration, document translation.
|
||||||
|
|||||||
107
components/ColumnSelector.tsx
Normal file
107
components/ColumnSelector.tsx
Normal file
@@ -0,0 +1,107 @@
|
|||||||
|
import { FC, useState, useEffect } from "react";
|
||||||
|
import {
|
||||||
|
Box, Checkbox, CheckboxGroup, VStack, HStack, Text, Button,
|
||||||
|
Accordion, AccordionItem, AccordionButton, AccordionPanel, AccordionIcon,
|
||||||
|
Badge
|
||||||
|
} from "@chakra-ui/react";
|
||||||
|
|
||||||
|
export type SheetColumnInfo = {
|
||||||
|
sheetName: string;
|
||||||
|
columns: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ColumnSelection = {
|
||||||
|
sheetName: string;
|
||||||
|
columnIndices: number[];
|
||||||
|
};
|
||||||
|
|
||||||
|
type Props = {
|
||||||
|
sheetColumns: SheetColumnInfo[];
|
||||||
|
onChange: (selections: ColumnSelection[]) => void;
|
||||||
|
};
|
||||||
|
|
||||||
|
const ColumnSelector: FC<Props> = ({ sheetColumns, onChange }) => {
|
||||||
|
const [selections, setSelections] = useState<Record<string, Set<number>>>(() => {
|
||||||
|
const init: Record<string, Set<number>> = {};
|
||||||
|
sheetColumns.forEach(s => {
|
||||||
|
init[s.sheetName] = new Set(s.columns.map((_, i) => i));
|
||||||
|
});
|
||||||
|
return init;
|
||||||
|
});
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const result: ColumnSelection[] = sheetColumns.map(s => ({
|
||||||
|
sheetName: s.sheetName,
|
||||||
|
columnIndices: Array.from(selections[s.sheetName] ?? [])
|
||||||
|
}));
|
||||||
|
onChange(result);
|
||||||
|
}, [selections, sheetColumns, onChange]);
|
||||||
|
|
||||||
|
const toggleColumn = (sheetName: string, colIdx: number) => {
|
||||||
|
setSelections(prev => {
|
||||||
|
const set = new Set(prev[sheetName] ?? []);
|
||||||
|
if (set.has(colIdx)) set.delete(colIdx);
|
||||||
|
else set.add(colIdx);
|
||||||
|
return { ...prev, [sheetName]: set };
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const selectAll = (sheetName: string, cols: string[]) => {
|
||||||
|
setSelections(prev => ({
|
||||||
|
...prev,
|
||||||
|
[sheetName]: new Set(cols.map((_, i) => i))
|
||||||
|
}));
|
||||||
|
};
|
||||||
|
|
||||||
|
const selectNone = (sheetName: string) => {
|
||||||
|
setSelections(prev => ({ ...prev, [sheetName]: new Set() }));
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Box w="full" borderWidth={1} borderRadius="md" p={3}>
|
||||||
|
<Text fontWeight="bold" mb={2} fontSize="sm">Select Columns to Translate</Text>
|
||||||
|
<Accordion allowMultiple defaultIndex={sheetColumns.map((_, i) => i)}>
|
||||||
|
{sheetColumns.map(sheet => (
|
||||||
|
<AccordionItem key={sheet.sheetName}>
|
||||||
|
<AccordionButton>
|
||||||
|
<Box flex="1" textAlign="left" fontSize="sm" fontWeight="semibold">
|
||||||
|
{sheet.sheetName}
|
||||||
|
<Badge ml={2} colorScheme="lingva">
|
||||||
|
{selections[sheet.sheetName]?.size ?? 0}/{sheet.columns.length}
|
||||||
|
</Badge>
|
||||||
|
</Box>
|
||||||
|
<AccordionIcon />
|
||||||
|
</AccordionButton>
|
||||||
|
<AccordionPanel pb={3}>
|
||||||
|
<HStack mb={2} spacing={2}>
|
||||||
|
<Button size="xs" variant="outline" colorScheme="lingva"
|
||||||
|
onClick={() => selectAll(sheet.sheetName, sheet.columns)}>
|
||||||
|
All
|
||||||
|
</Button>
|
||||||
|
<Button size="xs" variant="outline"
|
||||||
|
onClick={() => selectNone(sheet.sheetName)}>
|
||||||
|
None
|
||||||
|
</Button>
|
||||||
|
</HStack>
|
||||||
|
<VStack align="start" spacing={1} maxH="150px" overflowY="auto">
|
||||||
|
{sheet.columns.map((col, idx) => (
|
||||||
|
<Checkbox
|
||||||
|
key={idx}
|
||||||
|
size="sm"
|
||||||
|
isChecked={selections[sheet.sheetName]?.has(idx) ?? false}
|
||||||
|
onChange={() => toggleColumn(sheet.sheetName, idx)}
|
||||||
|
colorScheme="lingva"
|
||||||
|
>
|
||||||
|
<Text fontSize="xs">{col}</Text>
|
||||||
|
</Checkbox>
|
||||||
|
))}
|
||||||
|
</VStack>
|
||||||
|
</AccordionPanel>
|
||||||
|
</AccordionItem>
|
||||||
|
))}
|
||||||
|
</Accordion>
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
export default ColumnSelector;
|
||||||
261
components/DocumentTranslator.tsx
Normal file
261
components/DocumentTranslator.tsx
Normal file
@@ -0,0 +1,261 @@
|
|||||||
|
import { FC, useState, useCallback, useRef } from "react";
|
||||||
|
import {
|
||||||
|
Box, Button, VStack, HStack, Text, Input, Alert, AlertIcon,
|
||||||
|
Progress, Select, Badge, Icon, useColorModeValue
|
||||||
|
} from "@chakra-ui/react";
|
||||||
|
import { FiUpload, FiDownload, FiFile } from "react-icons/fi";
|
||||||
|
import { languageList, LangCode } from "lingva-scraper";
|
||||||
|
import dynamic from "next/dynamic";
|
||||||
|
import type { ColumnSelection, SheetColumnInfo } from "./ColumnSelector";
|
||||||
|
|
||||||
|
const ColumnSelector = dynamic(() => import("./ColumnSelector"), { ssr: false });
|
||||||
|
|
||||||
|
const SUPPORTED_TYPES = [".pdf", ".docx", ".xlsx", ".xls", ".csv"];
|
||||||
|
const TABULAR_TYPES = [".xlsx", ".xls", ".csv"];
|
||||||
|
|
||||||
|
type Stage = "idle" | "columns" | "translating" | "done" | "error";
|
||||||
|
|
||||||
|
const DocumentTranslator: FC = () => {
|
||||||
|
const [file, setFile] = useState<File | null>(null);
|
||||||
|
const [target, setTarget] = useState<string>("en");
|
||||||
|
const [stage, setStage] = useState<Stage>("idle");
|
||||||
|
const [progress, setProgress] = useState(0);
|
||||||
|
const [errorMsg, setErrorMsg] = useState("");
|
||||||
|
const [sheetColumns, setSheetColumns] = useState<SheetColumnInfo[]>([]);
|
||||||
|
const [columnSelections, setColumnSelections] = useState<ColumnSelection[]>([]);
|
||||||
|
const [downloadUrl, setDownloadUrl] = useState<string | null>(null);
|
||||||
|
const [downloadName, setDownloadName] = useState("");
|
||||||
|
const fileInputRef = useRef<HTMLInputElement>(null);
|
||||||
|
const borderColor = useColorModeValue("gray.200", "gray.600");
|
||||||
|
const dropBg = useColorModeValue("gray.50", "gray.700");
|
||||||
|
|
||||||
|
const isTabular = file
|
||||||
|
? TABULAR_TYPES.some(e => file.name.toLowerCase().endsWith(e))
|
||||||
|
: false;
|
||||||
|
|
||||||
|
const handleFile = useCallback(async (f: File) => {
|
||||||
|
setFile(f);
|
||||||
|
setStage("idle");
|
||||||
|
setErrorMsg("");
|
||||||
|
setDownloadUrl(null);
|
||||||
|
setSheetColumns([]);
|
||||||
|
|
||||||
|
const ext = "." + f.name.split(".").pop()?.toLowerCase();
|
||||||
|
if (!SUPPORTED_TYPES.includes(ext)) {
|
||||||
|
setErrorMsg(`Unsupported file type. Supported: ${SUPPORTED_TYPES.join(", ")}`);
|
||||||
|
setStage("error");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (TABULAR_TYPES.includes(ext)) {
|
||||||
|
// Fetch column info
|
||||||
|
const fd = new FormData();
|
||||||
|
fd.append("file", f);
|
||||||
|
fd.append("action", "getColumns");
|
||||||
|
try {
|
||||||
|
const res = await fetch("/api/translate/document", { method: "POST", body: fd });
|
||||||
|
if (res.ok) {
|
||||||
|
const data = await res.json();
|
||||||
|
setSheetColumns(data.columns ?? []);
|
||||||
|
setStage("columns");
|
||||||
|
} else {
|
||||||
|
const e = await res.json();
|
||||||
|
setErrorMsg(e.error ?? "Failed to read columns");
|
||||||
|
setStage("error");
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
setErrorMsg("Network error reading file columns");
|
||||||
|
setStage("error");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const onDrop = useCallback((e: React.DragEvent) => {
|
||||||
|
e.preventDefault();
|
||||||
|
const f = e.dataTransfer.files[0];
|
||||||
|
if (f) handleFile(f);
|
||||||
|
}, [handleFile]);
|
||||||
|
|
||||||
|
const translate = useCallback(async () => {
|
||||||
|
if (!file) return;
|
||||||
|
setStage("translating");
|
||||||
|
setProgress(10);
|
||||||
|
setErrorMsg("");
|
||||||
|
|
||||||
|
const fd = new FormData();
|
||||||
|
fd.append("file", file);
|
||||||
|
fd.append("targetLanguage", target);
|
||||||
|
fd.append("action", "translate");
|
||||||
|
if (columnSelections.length > 0) {
|
||||||
|
fd.append("columnSelections", JSON.stringify(columnSelections));
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
setProgress(30);
|
||||||
|
const res = await fetch("/api/translate/document", { method: "POST", body: fd });
|
||||||
|
setProgress(90);
|
||||||
|
|
||||||
|
if (!res.ok) {
|
||||||
|
const e = await res.json().catch(() => ({ error: "Translation failed" }));
|
||||||
|
setErrorMsg(e.error ?? "Translation failed");
|
||||||
|
setStage("error");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const blob = await res.blob();
|
||||||
|
const ext = "." + file.name.split(".").pop()!;
|
||||||
|
const outName = file.name.replace(ext, `_${target}${ext}`);
|
||||||
|
const url = URL.createObjectURL(blob);
|
||||||
|
setDownloadUrl(url);
|
||||||
|
setDownloadName(outName);
|
||||||
|
setProgress(100);
|
||||||
|
setStage("done");
|
||||||
|
} catch (err) {
|
||||||
|
setErrorMsg("Network error during translation");
|
||||||
|
setStage("error");
|
||||||
|
}
|
||||||
|
}, [file, target, columnSelections]);
|
||||||
|
|
||||||
|
const reset = () => {
|
||||||
|
setFile(null);
|
||||||
|
setStage("idle");
|
||||||
|
setErrorMsg("");
|
||||||
|
setDownloadUrl(null);
|
||||||
|
setSheetColumns([]);
|
||||||
|
if (fileInputRef.current) fileInputRef.current.value = "";
|
||||||
|
};
|
||||||
|
|
||||||
|
const targetLangs = Object.entries(languageList.target) as [LangCode<"target">, string][];
|
||||||
|
|
||||||
|
return (
|
||||||
|
<VStack w="full" spacing={4} px={[8, null, 24, 40]}>
|
||||||
|
{/* Drop Zone */}
|
||||||
|
<Box
|
||||||
|
w="full"
|
||||||
|
border="2px dashed"
|
||||||
|
borderColor={file ? "lingva.400" : borderColor}
|
||||||
|
borderRadius="xl"
|
||||||
|
p={8}
|
||||||
|
bg={dropBg}
|
||||||
|
textAlign="center"
|
||||||
|
cursor="pointer"
|
||||||
|
onDrop={onDrop}
|
||||||
|
onDragOver={e => e.preventDefault()}
|
||||||
|
onClick={() => fileInputRef.current?.click()}
|
||||||
|
_hover={{ borderColor: "lingva.400" }}
|
||||||
|
transition="border-color 0.2s"
|
||||||
|
>
|
||||||
|
<Input
|
||||||
|
ref={fileInputRef}
|
||||||
|
type="file"
|
||||||
|
display="none"
|
||||||
|
accept=".pdf,.docx,.xlsx,.xls,.csv"
|
||||||
|
onChange={e => { const f = e.target.files?.[0]; if (f) handleFile(f); }}
|
||||||
|
/>
|
||||||
|
{file ? (
|
||||||
|
<VStack spacing={1}>
|
||||||
|
<Icon as={FiFile} boxSize={8} color="lingva.400" />
|
||||||
|
<Text fontWeight="semibold">{file.name}</Text>
|
||||||
|
<Text fontSize="sm" color="gray.500">
|
||||||
|
{(file.size / 1024 / 1024).toFixed(2)} MB
|
||||||
|
</Text>
|
||||||
|
</VStack>
|
||||||
|
) : (
|
||||||
|
<VStack spacing={2}>
|
||||||
|
<Icon as={FiUpload} boxSize={10} color="gray.400" />
|
||||||
|
<Text fontWeight="semibold">Drop file here or click to upload</Text>
|
||||||
|
<Text fontSize="sm" color="gray.500">
|
||||||
|
Supported: PDF, Word (.docx), Excel (.xlsx, .xls), CSV
|
||||||
|
</Text>
|
||||||
|
</VStack>
|
||||||
|
)}
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
{/* Target language + controls */}
|
||||||
|
{file && stage !== "error" && (
|
||||||
|
<HStack w="full" spacing={3}>
|
||||||
|
<Box flex={1}>
|
||||||
|
<Select
|
||||||
|
value={target}
|
||||||
|
onChange={e => setTarget(e.target.value)}
|
||||||
|
size="md"
|
||||||
|
aria-label="Target language"
|
||||||
|
>
|
||||||
|
{targetLangs.map(([code, name]) => (
|
||||||
|
<option key={code} value={code}>{name}</option>
|
||||||
|
))}
|
||||||
|
</Select>
|
||||||
|
</Box>
|
||||||
|
<Button
|
||||||
|
colorScheme="lingva"
|
||||||
|
leftIcon={<Icon as={FiFile} />}
|
||||||
|
onClick={translate}
|
||||||
|
isLoading={stage === "translating"}
|
||||||
|
loadingText="Translating…"
|
||||||
|
isDisabled={stage === "translating" || (isTabular && stage !== "columns" && stage !== "idle")}
|
||||||
|
>
|
||||||
|
Translate
|
||||||
|
</Button>
|
||||||
|
<Button variant="ghost" size="sm" onClick={reset}>Reset</Button>
|
||||||
|
</HStack>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Column selector for Excel/CSV */}
|
||||||
|
{stage === "columns" && sheetColumns.length > 0 && (
|
||||||
|
<ColumnSelector
|
||||||
|
sheetColumns={sheetColumns}
|
||||||
|
onChange={setColumnSelections}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Progress */}
|
||||||
|
{stage === "translating" && (
|
||||||
|
<Box w="full">
|
||||||
|
<Progress value={progress} colorScheme="lingva" borderRadius="full" hasStripe isAnimated />
|
||||||
|
<Text fontSize="sm" textAlign="center" mt={1} color="gray.500">
|
||||||
|
Translating… this may take a moment for large documents.
|
||||||
|
</Text>
|
||||||
|
</Box>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Download */}
|
||||||
|
{stage === "done" && downloadUrl && (
|
||||||
|
<Alert status="success" borderRadius="md">
|
||||||
|
<AlertIcon />
|
||||||
|
<HStack justify="space-between" w="full">
|
||||||
|
<Text fontSize="sm">Translation complete!</Text>
|
||||||
|
<Button
|
||||||
|
as="a"
|
||||||
|
href={downloadUrl}
|
||||||
|
download={downloadName}
|
||||||
|
size="sm"
|
||||||
|
colorScheme="lingva"
|
||||||
|
leftIcon={<Icon as={FiDownload} />}
|
||||||
|
>
|
||||||
|
Download
|
||||||
|
</Button>
|
||||||
|
</HStack>
|
||||||
|
</Alert>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Error */}
|
||||||
|
{stage === "error" && errorMsg && (
|
||||||
|
<Alert status="error" borderRadius="md">
|
||||||
|
<AlertIcon />
|
||||||
|
<VStack align="start" spacing={1}>
|
||||||
|
<Text fontSize="sm">{errorMsg}</Text>
|
||||||
|
<Button size="xs" variant="link" colorScheme="red" onClick={reset}>Try again</Button>
|
||||||
|
</VStack>
|
||||||
|
</Alert>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<Text fontSize="xs" color="gray.400" textAlign="center">
|
||||||
|
Document translation requires Replicate AI to be enabled in the admin settings.
|
||||||
|
<br />
|
||||||
|
PDF formatting is best-effort; Excel and Word formatting is fully preserved.
|
||||||
|
</Text>
|
||||||
|
</VStack>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
export default DocumentTranslator;
|
||||||
@@ -3,6 +3,7 @@ import Head from "next/head";
|
|||||||
import NextLink from "next/link";
|
import NextLink from "next/link";
|
||||||
import { Flex, HStack, IconButton, Link, useColorModeValue } from "@chakra-ui/react";
|
import { Flex, HStack, IconButton, Link, useColorModeValue } from "@chakra-ui/react";
|
||||||
import { FaGithub } from "react-icons/fa";
|
import { FaGithub } from "react-icons/fa";
|
||||||
|
import { FiSettings } from "react-icons/fi";
|
||||||
import Image from "next/image";
|
import Image from "next/image";
|
||||||
import { ColorModeToggler } from ".";
|
import { ColorModeToggler } from ".";
|
||||||
|
|
||||||
@@ -40,6 +41,15 @@ const Header: FC<Props> = (props) => (
|
|||||||
<ColorModeToggler
|
<ColorModeToggler
|
||||||
variant={useColorModeValue("outline", "solid")}
|
variant={useColorModeValue("outline", "solid")}
|
||||||
/>
|
/>
|
||||||
|
<NextLink href="/admin" passHref={true}>
|
||||||
|
<IconButton
|
||||||
|
as={Link}
|
||||||
|
aria-label="Admin settings"
|
||||||
|
icon={<FiSettings />}
|
||||||
|
colorScheme="lingva"
|
||||||
|
variant={useColorModeValue("outline", "solid")}
|
||||||
|
/>
|
||||||
|
</NextLink>
|
||||||
<IconButton
|
<IconButton
|
||||||
as={Link}
|
as={Link}
|
||||||
href="https://github.com/thedaviddelta/lingva-translate"
|
href="https://github.com/thedaviddelta/lingva-translate"
|
||||||
|
|||||||
@@ -7,3 +7,5 @@ export { default as ColorModeToggler } from "./ColorModeToggler";
|
|||||||
export { default as LangSelect } from "./LangSelect";
|
export { default as LangSelect } from "./LangSelect";
|
||||||
export { default as TranslationArea } from "./TranslationArea";
|
export { default as TranslationArea } from "./TranslationArea";
|
||||||
export { default as AutoTranslateButton } from "./AutoTranslateButton";
|
export { default as AutoTranslateButton } from "./AutoTranslateButton";
|
||||||
|
export { default as DocumentTranslator } from "./DocumentTranslator";
|
||||||
|
export { default as ColumnSelector } from "./ColumnSelector";
|
||||||
|
|||||||
0
data/.gitkeep
Normal file
0
data/.gitkeep
Normal file
@@ -5,6 +5,20 @@ module.exports = withPWA({
|
|||||||
pwa: {
|
pwa: {
|
||||||
dest: "public"
|
dest: "public"
|
||||||
},
|
},
|
||||||
|
// Exclude server-only packages from client bundles
|
||||||
|
webpack: (config, { isServer }) => {
|
||||||
|
if (!isServer) {
|
||||||
|
config.resolve.fallback = {
|
||||||
|
...config.resolve.fallback,
|
||||||
|
fs: false,
|
||||||
|
path: false,
|
||||||
|
crypto: false,
|
||||||
|
stream: false,
|
||||||
|
zlib: false
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return config;
|
||||||
|
},
|
||||||
async headers() {
|
async headers() {
|
||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
|
|||||||
16714
package-lock.json
generated
Normal file
16714
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
@@ -18,18 +18,25 @@
|
|||||||
"@emotion/react": "^11",
|
"@emotion/react": "^11",
|
||||||
"@emotion/styled": "^11",
|
"@emotion/styled": "^11",
|
||||||
"apollo-server-micro": "^2.25.2",
|
"apollo-server-micro": "^2.25.2",
|
||||||
|
"formidable": "^3.5.1",
|
||||||
"framer-motion": "^6",
|
"framer-motion": "^6",
|
||||||
"graphql": "^15.8.0",
|
"graphql": "^15.8.0",
|
||||||
|
"jose": "^4.14.4",
|
||||||
|
"jszip": "^3.10.1",
|
||||||
"lingva-scraper": "1.1.0",
|
"lingva-scraper": "1.1.0",
|
||||||
"next": "12.1.6",
|
"next": "12.1.6",
|
||||||
"next-pwa": "^5.4.4",
|
"next-pwa": "^5.4.4",
|
||||||
"nextjs-cors": "^2.1.0",
|
"nextjs-cors": "^2.1.0",
|
||||||
|
"pdf-lib": "^1.17.1",
|
||||||
|
"pdf-parse": "^1.1.1",
|
||||||
"react": "18.2.0",
|
"react": "18.2.0",
|
||||||
"react-dom": "18.2.0",
|
"react-dom": "18.2.0",
|
||||||
"react-hotkeys-hook": "^3.4.6",
|
"react-hotkeys-hook": "^3.4.6",
|
||||||
"react-icons": "^4.4.0"
|
"react-icons": "^4.4.0",
|
||||||
|
"xlsx": "^0.18.5"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
|
"@types/formidable": "^3.4.4",
|
||||||
"@testing-library/cypress": "^8.0.3",
|
"@testing-library/cypress": "^8.0.3",
|
||||||
"@testing-library/jest-dom": "^5.16.4",
|
"@testing-library/jest-dom": "^5.16.4",
|
||||||
"@testing-library/react": "^13.3.0",
|
"@testing-library/react": "^13.3.0",
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { useCallback, useEffect, useReducer } from "react";
|
import { useCallback, useEffect, useReducer, useState } from "react";
|
||||||
import { GetStaticPaths, GetStaticProps, NextPage } from "next";
|
import { GetStaticPaths, GetStaticProps, NextPage } from "next";
|
||||||
import { useRouter } from "next/router";
|
import { useRouter } from "next/router";
|
||||||
import dynamic from "next/dynamic";
|
import dynamic from "next/dynamic";
|
||||||
@@ -13,9 +13,10 @@ import {
|
|||||||
TranslationInfo,
|
TranslationInfo,
|
||||||
LangCode
|
LangCode
|
||||||
} from "lingva-scraper";
|
} from "lingva-scraper";
|
||||||
import { HStack, IconButton, Stack, VStack } from "@chakra-ui/react";
|
import { HStack, IconButton, Stack, VStack, Tabs, TabList, Tab, TabPanels, TabPanel } from "@chakra-ui/react";
|
||||||
import { FaExchangeAlt } from "react-icons/fa";
|
import { FaExchangeAlt } from "react-icons/fa";
|
||||||
import { HiTranslate } from "react-icons/hi";
|
import { HiTranslate } from "react-icons/hi";
|
||||||
|
import { FiFileText } from "react-icons/fi";
|
||||||
import { useHotkeys } from "react-hotkeys-hook";
|
import { useHotkeys } from "react-hotkeys-hook";
|
||||||
import { CustomHead, LangSelect, TranslationArea } from "@components";
|
import { CustomHead, LangSelect, TranslationArea } from "@components";
|
||||||
import { useToastOnLoad } from "@hooks";
|
import { useToastOnLoad } from "@hooks";
|
||||||
@@ -24,6 +25,7 @@ import langReducer, { Actions, initialState, State } from "@utils/reducer";
|
|||||||
import { localGetItem, localSetItem } from "@utils/storage";
|
import { localGetItem, localSetItem } from "@utils/storage";
|
||||||
|
|
||||||
const AutoTranslateButton = dynamic(() => import("@components/AutoTranslateButton"), { ssr: false });
|
const AutoTranslateButton = dynamic(() => import("@components/AutoTranslateButton"), { ssr: false });
|
||||||
|
const DocumentTranslator = dynamic(() => import("@components/DocumentTranslator"), { ssr: false });
|
||||||
|
|
||||||
export enum ResponseType {
|
export enum ResponseType {
|
||||||
SUCCESS,
|
SUCCESS,
|
||||||
@@ -63,6 +65,17 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
] = useReducer(langReducer, initialState);
|
] = useReducer(langReducer, initialState);
|
||||||
|
|
||||||
const router = useRouter();
|
const router = useRouter();
|
||||||
|
const [replicateEnabled, setReplicateEnabled] = useState(false);
|
||||||
|
const [replicateTranslation, setReplicateTranslation] = useState("");
|
||||||
|
const [replicateLoading, setReplicateLoading] = useState(false);
|
||||||
|
|
||||||
|
// Check if Replicate is enabled
|
||||||
|
useEffect(() => {
|
||||||
|
fetch("/api/admin/settings")
|
||||||
|
.then(r => r.ok ? r.json() : null)
|
||||||
|
.then(d => { if (d?.replicateEnabled) setReplicateEnabled(true); })
|
||||||
|
.catch(() => {});
|
||||||
|
}, []);
|
||||||
|
|
||||||
const setField = useCallback(<T extends keyof State,>(key: T, value: State[T]) => (
|
const setField = useCallback(<T extends keyof State,>(key: T, value: State[T]) => (
|
||||||
dispatch({ type: Actions.SET_FIELD, payload: { key, value }})
|
dispatch({ type: Actions.SET_FIELD, payload: { key, value }})
|
||||||
@@ -101,6 +114,30 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
router.push(`/${source}/${target}/${encodeURIComponent(customQuery)}`);
|
router.push(`/${source}/${target}/${encodeURIComponent(customQuery)}`);
|
||||||
}, [isLoading, source, target, props, router, setField]);
|
}, [isLoading, source, target, props, router, setField]);
|
||||||
|
|
||||||
|
// Replicate-powered translation
|
||||||
|
const translateWithReplicate = useCallback(async (text: string) => {
|
||||||
|
if (!text.trim()) return;
|
||||||
|
setReplicateLoading(true);
|
||||||
|
setReplicateTranslation("");
|
||||||
|
try {
|
||||||
|
const res = await fetch("/api/translate/replicate", {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ text, targetLanguage: target })
|
||||||
|
});
|
||||||
|
const data = await res.json();
|
||||||
|
if (res.ok) {
|
||||||
|
setReplicateTranslation(data.translation ?? "");
|
||||||
|
} else {
|
||||||
|
setReplicateTranslation(`[Error: ${data.error}]`);
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
setReplicateTranslation("[Error: Network failure]");
|
||||||
|
} finally {
|
||||||
|
setReplicateLoading(false);
|
||||||
|
}
|
||||||
|
}, [target]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (router.isFallback)
|
if (router.isFallback)
|
||||||
return;
|
return;
|
||||||
@@ -183,11 +220,7 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
: replaceExceptedCode(LanguageType.TARGET, source);
|
: replaceExceptedCode(LanguageType.TARGET, source);
|
||||||
const transLang = replaceExceptedCode(LanguageType.SOURCE, target);
|
const transLang = replaceExceptedCode(LanguageType.SOURCE, target);
|
||||||
|
|
||||||
return (
|
const LangControls = (
|
||||||
<>
|
|
||||||
<CustomHead home={props.type === ResponseType.HOME} />
|
|
||||||
|
|
||||||
<VStack px={[8, null, 24, 40]} w="full">
|
|
||||||
<HStack px={[1, null, 3, 4]} w="full">
|
<HStack px={[1, null, 3, 4]} w="full">
|
||||||
<LangSelect
|
<LangSelect
|
||||||
id="source"
|
id="source"
|
||||||
@@ -213,6 +246,24 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
langs={languageList.target}
|
langs={languageList.target}
|
||||||
/>
|
/>
|
||||||
</HStack>
|
</HStack>
|
||||||
|
);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<CustomHead home={props.type === ResponseType.HOME} />
|
||||||
|
|
||||||
|
<VStack px={[8, null, 24, 40]} w="full">
|
||||||
|
<Tabs w="full" colorScheme="lingva" variant="soft-rounded" size="sm">
|
||||||
|
<TabList mb={4} justifyContent="center">
|
||||||
|
<Tab><HiTranslate style={{ marginRight: 6 }} />Text</Tab>
|
||||||
|
<Tab><FiFileText style={{ marginRight: 6 }} />Document</Tab>
|
||||||
|
</TabList>
|
||||||
|
|
||||||
|
<TabPanels>
|
||||||
|
{/* Text Translation Tab */}
|
||||||
|
<TabPanel p={0}>
|
||||||
|
<VStack w="full" spacing={4}>
|
||||||
|
{LangControls}
|
||||||
<Stack direction={["column", null, "row"]} w="full">
|
<Stack direction={["column", null, "row"]} w="full">
|
||||||
<TranslationArea
|
<TranslationArea
|
||||||
id="query"
|
id="query"
|
||||||
@@ -220,7 +271,7 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
placeholder="Text"
|
placeholder="Text"
|
||||||
value={query}
|
value={query}
|
||||||
onChange={e => isLoading || setField("query", e.target.value)}
|
onChange={e => isLoading || setField("query", e.target.value)}
|
||||||
onSubmit={() => changeRoute(query)}
|
onSubmit={() => replicateEnabled ? translateWithReplicate(query) : changeRoute(query)}
|
||||||
lang={queryLang}
|
lang={queryLang}
|
||||||
audio={audio.query}
|
audio={audio.query}
|
||||||
pronunciation={pronunciation.query}
|
pronunciation={pronunciation.query}
|
||||||
@@ -231,14 +282,20 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
icon={<HiTranslate />}
|
icon={<HiTranslate />}
|
||||||
colorScheme="lingva"
|
colorScheme="lingva"
|
||||||
variant="outline"
|
variant="outline"
|
||||||
onClick={() => changeRoute(query)}
|
onClick={() => replicateEnabled ? translateWithReplicate(query) : changeRoute(query)}
|
||||||
isDisabled={isLoading}
|
isDisabled={isLoading || replicateLoading}
|
||||||
|
isLoading={replicateLoading}
|
||||||
w={["full", null, "auto"]}
|
w={["full", null, "auto"]}
|
||||||
/>
|
/>
|
||||||
<AutoTranslateButton
|
<AutoTranslateButton
|
||||||
isDisabled={isLoading}
|
isDisabled={isLoading || replicateLoading}
|
||||||
// runs on effect update
|
onAuto={useCallback(() => {
|
||||||
onAuto={useCallback(() => changeRoute(delayedQuery), [delayedQuery, changeRoute])}
|
if (replicateEnabled) {
|
||||||
|
translateWithReplicate(delayedQuery);
|
||||||
|
} else {
|
||||||
|
changeRoute(delayedQuery);
|
||||||
|
}
|
||||||
|
}, [delayedQuery, replicateEnabled, translateWithReplicate, changeRoute])}
|
||||||
w={["full", null, "auto"]}
|
w={["full", null, "auto"]}
|
||||||
/>
|
/>
|
||||||
</Stack>
|
</Stack>
|
||||||
@@ -246,16 +303,25 @@ const Page: NextPage<Props> = (props) => {
|
|||||||
id="translation"
|
id="translation"
|
||||||
aria-label="Translation result"
|
aria-label="Translation result"
|
||||||
placeholder="Translation"
|
placeholder="Translation"
|
||||||
value={translation ?? ""}
|
value={replicateEnabled ? replicateTranslation : (translation ?? "")}
|
||||||
readOnly={true}
|
readOnly={true}
|
||||||
lang={transLang}
|
lang={transLang}
|
||||||
audio={audio.translation}
|
audio={audio.translation}
|
||||||
canCopy={true}
|
canCopy={true}
|
||||||
isLoading={isLoading}
|
isLoading={replicateEnabled ? replicateLoading : isLoading}
|
||||||
pronunciation={pronunciation.translation}
|
pronunciation={replicateEnabled ? undefined : pronunciation.translation}
|
||||||
/>
|
/>
|
||||||
</Stack>
|
</Stack>
|
||||||
</VStack>
|
</VStack>
|
||||||
|
</TabPanel>
|
||||||
|
|
||||||
|
{/* Document Translation Tab */}
|
||||||
|
<TabPanel p={0}>
|
||||||
|
<DocumentTranslator />
|
||||||
|
</TabPanel>
|
||||||
|
</TabPanels>
|
||||||
|
</Tabs>
|
||||||
|
</VStack>
|
||||||
</>
|
</>
|
||||||
);
|
);
|
||||||
};
|
};
|
||||||
|
|||||||
372
pages/admin/index.tsx
Normal file
372
pages/admin/index.tsx
Normal file
@@ -0,0 +1,372 @@
|
|||||||
|
import { useState, useEffect, FormEvent } from "react";
|
||||||
|
import { NextPage } from "next";
|
||||||
|
import NextLink from "next/link";
|
||||||
|
import {
|
||||||
|
Box, VStack, HStack, Heading, Text, Input, Button, Switch,
|
||||||
|
FormControl, FormLabel, FormHelperText, Divider, Alert, AlertIcon,
|
||||||
|
InputGroup, InputRightElement, IconButton, Badge, Link,
|
||||||
|
useColorModeValue, Spinner, Textarea
|
||||||
|
} from "@chakra-ui/react";
|
||||||
|
import { FiEye, FiEyeOff, FiSave, FiLogOut, FiArrowLeft } from "react-icons/fi";
|
||||||
|
import { CustomHead } from "@components";
|
||||||
|
|
||||||
|
type Settings = {
|
||||||
|
replicateApiToken: string;
|
||||||
|
jigsawApiKey: string;
|
||||||
|
modelVersion: string;
|
||||||
|
replicateEnabled: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
const DEFAULT_MODEL = "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89";
|
||||||
|
|
||||||
|
const AdminPage: NextPage = () => {
|
||||||
|
const [authed, setAuthed] = useState<boolean | null>(null);
|
||||||
|
const [password, setPassword] = useState("");
|
||||||
|
const [loginError, setLoginError] = useState("");
|
||||||
|
const [loginLoading, setLoginLoading] = useState(false);
|
||||||
|
|
||||||
|
const [settings, setSettings] = useState<Settings>({
|
||||||
|
replicateApiToken: "",
|
||||||
|
jigsawApiKey: "",
|
||||||
|
modelVersion: DEFAULT_MODEL,
|
||||||
|
replicateEnabled: false
|
||||||
|
});
|
||||||
|
const [newPassword, setNewPassword] = useState("");
|
||||||
|
const [saveMsg, setSaveMsg] = useState<{ type: "success" | "error"; text: string } | null>(null);
|
||||||
|
const [saving, setSaving] = useState(false);
|
||||||
|
const [showToken, setShowToken] = useState(false);
|
||||||
|
const [showJigsawKey, setShowJigsawKey] = useState(false);
|
||||||
|
const [testResult, setTestResult] = useState<string | null>(null);
|
||||||
|
const [testLoading, setTestLoading] = useState(false);
|
||||||
|
|
||||||
|
const cardBg = useColorModeValue("white", "gray.800");
|
||||||
|
const borderCol = useColorModeValue("gray.200", "gray.600");
|
||||||
|
|
||||||
|
// Check auth on load
|
||||||
|
useEffect(() => {
|
||||||
|
fetch("/api/admin/auth")
|
||||||
|
.then(r => r.json())
|
||||||
|
.then(d => setAuthed(d.authenticated))
|
||||||
|
.catch(() => setAuthed(false));
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
// Load settings when authed
|
||||||
|
useEffect(() => {
|
||||||
|
if (!authed) return;
|
||||||
|
fetch("/api/admin/settings")
|
||||||
|
.then(r => r.json())
|
||||||
|
.then(d => setSettings(s => ({ ...s, ...d })))
|
||||||
|
.catch(() => {});
|
||||||
|
}, [authed]);
|
||||||
|
|
||||||
|
const login = async (e: FormEvent) => {
|
||||||
|
e.preventDefault();
|
||||||
|
setLoginLoading(true);
|
||||||
|
setLoginError("");
|
||||||
|
try {
|
||||||
|
const res = await fetch("/api/admin/auth", {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ password })
|
||||||
|
});
|
||||||
|
if (res.ok) {
|
||||||
|
setAuthed(true);
|
||||||
|
} else {
|
||||||
|
const d = await res.json();
|
||||||
|
setLoginError(d.error ?? "Login failed");
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
setLoginError("Network error");
|
||||||
|
} finally {
|
||||||
|
setLoginLoading(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const logout = async () => {
|
||||||
|
await fetch("/api/admin/auth", { method: "DELETE" });
|
||||||
|
setAuthed(false);
|
||||||
|
setPassword("");
|
||||||
|
};
|
||||||
|
|
||||||
|
const save = async (e: FormEvent) => {
|
||||||
|
e.preventDefault();
|
||||||
|
setSaving(true);
|
||||||
|
setSaveMsg(null);
|
||||||
|
try {
|
||||||
|
const body: Record<string, unknown> = { ...settings };
|
||||||
|
if (newPassword.length >= 6) body.newPassword = newPassword;
|
||||||
|
const res = await fetch("/api/admin/settings", {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(body)
|
||||||
|
});
|
||||||
|
const d = await res.json();
|
||||||
|
if (res.ok) {
|
||||||
|
setSettings(s => ({ ...s, ...d }));
|
||||||
|
setSaveMsg({ type: "success", text: "Settings saved successfully" });
|
||||||
|
setNewPassword("");
|
||||||
|
} else {
|
||||||
|
setSaveMsg({ type: "error", text: d.error ?? "Save failed" });
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
setSaveMsg({ type: "error", text: "Network error" });
|
||||||
|
} finally {
|
||||||
|
setSaving(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const testTranslation = async () => {
|
||||||
|
setTestLoading(true);
|
||||||
|
setTestResult(null);
|
||||||
|
try {
|
||||||
|
const res = await fetch("/api/translate/replicate", {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ text: "Hello, world!", targetLanguage: "es" })
|
||||||
|
});
|
||||||
|
const d = await res.json();
|
||||||
|
if (res.ok) {
|
||||||
|
setTestResult(`✓ Success: "${d.translation}"`);
|
||||||
|
} else {
|
||||||
|
setTestResult(`✗ Error: ${d.error}`);
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
setTestResult("✗ Network error");
|
||||||
|
} finally {
|
||||||
|
setTestLoading(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Loading state
|
||||||
|
if (authed === null) {
|
||||||
|
return (
|
||||||
|
<Box w="full" display="flex" justifyContent="center" pt={20}>
|
||||||
|
<Spinner size="xl" color="lingva.400" />
|
||||||
|
</Box>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Login form
|
||||||
|
if (!authed) {
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<CustomHead home={false} />
|
||||||
|
<Box
|
||||||
|
w="full"
|
||||||
|
maxW="400px"
|
||||||
|
mx="auto"
|
||||||
|
mt={10}
|
||||||
|
p={8}
|
||||||
|
bg={cardBg}
|
||||||
|
borderWidth={1}
|
||||||
|
borderColor={borderCol}
|
||||||
|
borderRadius="xl"
|
||||||
|
boxShadow="md"
|
||||||
|
>
|
||||||
|
<VStack spacing={6} as="form" onSubmit={login}>
|
||||||
|
<Heading size="md">Admin Login</Heading>
|
||||||
|
{loginError && (
|
||||||
|
<Alert status="error" borderRadius="md">
|
||||||
|
<AlertIcon />{loginError}
|
||||||
|
</Alert>
|
||||||
|
)}
|
||||||
|
<FormControl isRequired>
|
||||||
|
<FormLabel>Password</FormLabel>
|
||||||
|
<Input
|
||||||
|
type="password"
|
||||||
|
value={password}
|
||||||
|
onChange={e => setPassword(e.target.value)}
|
||||||
|
autoFocus
|
||||||
|
placeholder="Admin password"
|
||||||
|
/>
|
||||||
|
<FormHelperText>Default: admin (set ADMIN_PASSWORD env var)</FormHelperText>
|
||||||
|
</FormControl>
|
||||||
|
<Button
|
||||||
|
type="submit"
|
||||||
|
colorScheme="lingva"
|
||||||
|
w="full"
|
||||||
|
isLoading={loginLoading}
|
||||||
|
>
|
||||||
|
Login
|
||||||
|
</Button>
|
||||||
|
<NextLink href="/" passHref>
|
||||||
|
<Link fontSize="sm">← Back to translator</Link>
|
||||||
|
</NextLink>
|
||||||
|
</VStack>
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Admin panel
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<CustomHead home={false} />
|
||||||
|
<Box w="full" maxW="700px" mx="auto" px={4} pb={10}>
|
||||||
|
<HStack justify="space-between" mb={6}>
|
||||||
|
<NextLink href="/" passHref>
|
||||||
|
<Button as={Link} leftIcon={<FiArrowLeft />} variant="ghost" size="sm">
|
||||||
|
Back
|
||||||
|
</Button>
|
||||||
|
</NextLink>
|
||||||
|
<Heading size="md">Admin Settings</Heading>
|
||||||
|
<Button leftIcon={<FiLogOut />} variant="ghost" size="sm" onClick={logout}>
|
||||||
|
Logout
|
||||||
|
</Button>
|
||||||
|
</HStack>
|
||||||
|
|
||||||
|
<VStack spacing={6} as="form" onSubmit={save}>
|
||||||
|
{/* Replicate Section */}
|
||||||
|
<Box w="full" p={6} bg={cardBg} borderWidth={1} borderColor={borderCol} borderRadius="xl">
|
||||||
|
<HStack mb={4}>
|
||||||
|
<Heading size="sm">Replicate AI Translation</Heading>
|
||||||
|
<Badge colorScheme={settings.replicateEnabled ? "green" : "gray"}>
|
||||||
|
{settings.replicateEnabled ? "Enabled" : "Disabled"}
|
||||||
|
</Badge>
|
||||||
|
</HStack>
|
||||||
|
|
||||||
|
<VStack spacing={4} align="stretch">
|
||||||
|
<FormControl display="flex" alignItems="center">
|
||||||
|
<FormLabel mb={0}>Enable Replicate Translation</FormLabel>
|
||||||
|
<Switch
|
||||||
|
colorScheme="lingva"
|
||||||
|
isChecked={settings.replicateEnabled}
|
||||||
|
onChange={e => setSettings(s => ({ ...s, replicateEnabled: e.target.checked }))}
|
||||||
|
/>
|
||||||
|
</FormControl>
|
||||||
|
|
||||||
|
<FormControl>
|
||||||
|
<FormLabel>Replicate API Token</FormLabel>
|
||||||
|
<InputGroup>
|
||||||
|
<Input
|
||||||
|
type={showToken ? "text" : "password"}
|
||||||
|
value={settings.replicateApiToken}
|
||||||
|
onChange={e => setSettings(s => ({ ...s, replicateApiToken: e.target.value }))}
|
||||||
|
placeholder="r8_..."
|
||||||
|
fontFamily="mono"
|
||||||
|
fontSize="sm"
|
||||||
|
/>
|
||||||
|
<InputRightElement>
|
||||||
|
<IconButton
|
||||||
|
size="xs"
|
||||||
|
variant="ghost"
|
||||||
|
aria-label="Toggle visibility"
|
||||||
|
icon={showToken ? <FiEyeOff /> : <FiEye />}
|
||||||
|
onClick={() => setShowToken(v => !v)}
|
||||||
|
/>
|
||||||
|
</InputRightElement>
|
||||||
|
</InputGroup>
|
||||||
|
<FormHelperText>
|
||||||
|
Get your token at{" "}
|
||||||
|
<Link href="https://replicate.com/account/api-tokens" isExternal color="lingva.400">
|
||||||
|
replicate.com/account/api-tokens
|
||||||
|
</Link>
|
||||||
|
</FormHelperText>
|
||||||
|
</FormControl>
|
||||||
|
|
||||||
|
<FormControl>
|
||||||
|
<FormLabel>JigsawStack API Key</FormLabel>
|
||||||
|
<InputGroup>
|
||||||
|
<Input
|
||||||
|
type={showJigsawKey ? "text" : "password"}
|
||||||
|
value={settings.jigsawApiKey}
|
||||||
|
onChange={e => setSettings(s => ({ ...s, jigsawApiKey: e.target.value }))}
|
||||||
|
placeholder="sk_..."
|
||||||
|
fontFamily="mono"
|
||||||
|
fontSize="sm"
|
||||||
|
/>
|
||||||
|
<InputRightElement>
|
||||||
|
<IconButton
|
||||||
|
size="xs"
|
||||||
|
variant="ghost"
|
||||||
|
aria-label="Toggle visibility"
|
||||||
|
icon={showJigsawKey ? <FiEyeOff /> : <FiEye />}
|
||||||
|
onClick={() => setShowJigsawKey(v => !v)}
|
||||||
|
/>
|
||||||
|
</InputRightElement>
|
||||||
|
</InputGroup>
|
||||||
|
<FormHelperText>
|
||||||
|
Required for the JigsawStack translation model. Get yours at{" "}
|
||||||
|
<Link href="https://jigsawstack.com" isExternal color="lingva.400">
|
||||||
|
jigsawstack.com
|
||||||
|
</Link>
|
||||||
|
</FormHelperText>
|
||||||
|
</FormControl>
|
||||||
|
|
||||||
|
<FormControl>
|
||||||
|
<FormLabel>Model Version</FormLabel>
|
||||||
|
<Textarea
|
||||||
|
value={settings.modelVersion}
|
||||||
|
onChange={e => setSettings(s => ({ ...s, modelVersion: e.target.value }))}
|
||||||
|
fontFamily="mono"
|
||||||
|
fontSize="xs"
|
||||||
|
rows={2}
|
||||||
|
placeholder={DEFAULT_MODEL}
|
||||||
|
/>
|
||||||
|
<FormHelperText>
|
||||||
|
Replicate model version string. Default uses the JigsawStack text-translate model.
|
||||||
|
</FormHelperText>
|
||||||
|
</FormControl>
|
||||||
|
|
||||||
|
<HStack>
|
||||||
|
<Button
|
||||||
|
size="sm"
|
||||||
|
variant="outline"
|
||||||
|
colorScheme="lingva"
|
||||||
|
onClick={testTranslation}
|
||||||
|
isLoading={testLoading}
|
||||||
|
isDisabled={!settings.replicateEnabled || !settings.replicateApiToken}
|
||||||
|
>
|
||||||
|
Test Translation
|
||||||
|
</Button>
|
||||||
|
{testResult && (
|
||||||
|
<Text
|
||||||
|
fontSize="sm"
|
||||||
|
color={testResult.startsWith("✓") ? "green.500" : "red.500"}
|
||||||
|
>
|
||||||
|
{testResult}
|
||||||
|
</Text>
|
||||||
|
)}
|
||||||
|
</HStack>
|
||||||
|
</VStack>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
{/* Security Section */}
|
||||||
|
<Box w="full" p={6} bg={cardBg} borderWidth={1} borderColor={borderCol} borderRadius="xl">
|
||||||
|
<Heading size="sm" mb={4}>Security</Heading>
|
||||||
|
<FormControl>
|
||||||
|
<FormLabel>New Admin Password</FormLabel>
|
||||||
|
<Input
|
||||||
|
type="password"
|
||||||
|
value={newPassword}
|
||||||
|
onChange={e => setNewPassword(e.target.value)}
|
||||||
|
placeholder="Leave blank to keep current"
|
||||||
|
minLength={6}
|
||||||
|
/>
|
||||||
|
<FormHelperText>Minimum 6 characters. Leave blank to keep current password.</FormHelperText>
|
||||||
|
</FormControl>
|
||||||
|
</Box>
|
||||||
|
|
||||||
|
{saveMsg && (
|
||||||
|
<Alert status={saveMsg.type} borderRadius="md">
|
||||||
|
<AlertIcon />{saveMsg.text}
|
||||||
|
</Alert>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<Button
|
||||||
|
type="submit"
|
||||||
|
colorScheme="lingva"
|
||||||
|
leftIcon={<FiSave />}
|
||||||
|
isLoading={saving}
|
||||||
|
size="lg"
|
||||||
|
w="full"
|
||||||
|
>
|
||||||
|
Save Settings
|
||||||
|
</Button>
|
||||||
|
</VStack>
|
||||||
|
</Box>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
export default AdminPage;
|
||||||
33
pages/api/admin/auth.ts
Normal file
33
pages/api/admin/auth.ts
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
import { NextApiHandler } from "next";
|
||||||
|
import { signAdminToken, checkPassword, COOKIE_NAME, getTokenFromRequest, verifyAdminToken } from "@utils/admin-auth";
|
||||||
|
|
||||||
|
const handler: NextApiHandler = async (req, res) => {
|
||||||
|
if (req.method === "POST") {
|
||||||
|
const { password } = req.body ?? {};
|
||||||
|
if (!password || typeof password !== "string") {
|
||||||
|
return res.status(400).json({ error: "Password required" });
|
||||||
|
}
|
||||||
|
if (!checkPassword(password)) {
|
||||||
|
return res.status(401).json({ error: "Invalid password" });
|
||||||
|
}
|
||||||
|
const token = await signAdminToken();
|
||||||
|
res.setHeader("Set-Cookie", `${COOKIE_NAME}=${token}; HttpOnly; Path=/; SameSite=Lax; Max-Age=28800`);
|
||||||
|
return res.status(200).json({ ok: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
if (req.method === "DELETE") {
|
||||||
|
res.setHeader("Set-Cookie", `${COOKIE_NAME}=; HttpOnly; Path=/; Max-Age=0`);
|
||||||
|
return res.status(200).json({ ok: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
if (req.method === "GET") {
|
||||||
|
const token = getTokenFromRequest(req);
|
||||||
|
const valid = token ? await verifyAdminToken(token) : false;
|
||||||
|
return res.status(200).json({ authenticated: valid });
|
||||||
|
}
|
||||||
|
|
||||||
|
res.setHeader("Allow", ["GET", "POST", "DELETE"]);
|
||||||
|
return res.status(405).json({ error: "Method Not Allowed" });
|
||||||
|
};
|
||||||
|
|
||||||
|
export default handler;
|
||||||
42
pages/api/admin/settings.ts
Normal file
42
pages/api/admin/settings.ts
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
import { NextApiHandler } from "next";
|
||||||
|
import { requireAdmin } from "@utils/admin-auth";
|
||||||
|
import { readSettings, writeSettings } from "@utils/settings-store";
|
||||||
|
|
||||||
|
const handler: NextApiHandler = async (req, res) => {
|
||||||
|
if (!(await requireAdmin(req, res))) return;
|
||||||
|
|
||||||
|
if (req.method === "GET") {
|
||||||
|
const settings = readSettings();
|
||||||
|
// Never expose the adminPasswordHash
|
||||||
|
const { adminPasswordHash: _omit, ...safe } = settings;
|
||||||
|
return res.status(200).json(safe);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (req.method === "POST") {
|
||||||
|
const {
|
||||||
|
replicateApiToken,
|
||||||
|
jigsawApiKey,
|
||||||
|
modelVersion,
|
||||||
|
replicateEnabled,
|
||||||
|
newPassword
|
||||||
|
} = req.body ?? {};
|
||||||
|
|
||||||
|
const updates: Parameters<typeof writeSettings>[0] = {};
|
||||||
|
if (replicateApiToken !== undefined) updates.replicateApiToken = replicateApiToken;
|
||||||
|
if (jigsawApiKey !== undefined) updates.jigsawApiKey = jigsawApiKey;
|
||||||
|
if (modelVersion !== undefined) updates.modelVersion = modelVersion;
|
||||||
|
if (replicateEnabled !== undefined) updates.replicateEnabled = Boolean(replicateEnabled);
|
||||||
|
if (newPassword && typeof newPassword === "string" && newPassword.length >= 6) {
|
||||||
|
updates.adminPasswordHash = newPassword;
|
||||||
|
}
|
||||||
|
|
||||||
|
const saved = writeSettings(updates);
|
||||||
|
const { adminPasswordHash: _omit, ...safe } = saved;
|
||||||
|
return res.status(200).json(safe);
|
||||||
|
}
|
||||||
|
|
||||||
|
res.setHeader("Allow", ["GET", "POST"]);
|
||||||
|
return res.status(405).json({ error: "Method Not Allowed" });
|
||||||
|
};
|
||||||
|
|
||||||
|
export default handler;
|
||||||
136
pages/api/translate/document.ts
Normal file
136
pages/api/translate/document.ts
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
import { NextApiHandler } from "next";
|
||||||
|
import formidable, { File, Fields, Files } from "formidable";
|
||||||
|
import fs from "fs";
|
||||||
|
import path from "path";
|
||||||
|
import { translateExcel, getExcelColumns, ColumnSelection } from "@utils/document-processors/excel";
|
||||||
|
import { translateDocx } from "@utils/document-processors/docx";
|
||||||
|
import { translatePdf } from "@utils/document-processors/pdf";
|
||||||
|
import { readSettings } from "@utils/settings-store";
|
||||||
|
|
||||||
|
export const config = {
|
||||||
|
api: {
|
||||||
|
bodyParser: false,
|
||||||
|
responseLimit: "50mb"
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
type ParsedForm = {
|
||||||
|
fields: Fields;
|
||||||
|
files: Files;
|
||||||
|
};
|
||||||
|
|
||||||
|
function parseForm(req: Parameters<NextApiHandler>[0]): Promise<ParsedForm> {
|
||||||
|
const form = formidable({ maxFileSize: 50 * 1024 * 1024 }); // 50 MB
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
form.parse(req, (err, fields, files) => {
|
||||||
|
if (err) reject(err);
|
||||||
|
else resolve({ fields, files });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function getFileBuffer(file: File): Buffer {
|
||||||
|
return fs.readFileSync(file.filepath);
|
||||||
|
}
|
||||||
|
|
||||||
|
function getField(fields: Fields, key: string): string | undefined {
|
||||||
|
const val = fields[key];
|
||||||
|
return Array.isArray(val) ? val[0] : (val as string | undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
const handler: NextApiHandler = async (req, res) => {
|
||||||
|
if (req.method !== "POST") {
|
||||||
|
res.setHeader("Allow", ["POST"]);
|
||||||
|
return res.status(405).json({ error: "Method Not Allowed" });
|
||||||
|
}
|
||||||
|
|
||||||
|
const settings = readSettings();
|
||||||
|
if (!settings.replicateEnabled) {
|
||||||
|
return res.status(503).json({ error: "Replicate translation is not enabled. Configure it in the admin panel." });
|
||||||
|
}
|
||||||
|
|
||||||
|
let parsed: ParsedForm;
|
||||||
|
try {
|
||||||
|
parsed = await parseForm(req);
|
||||||
|
} catch (err) {
|
||||||
|
return res.status(400).json({ error: "Failed to parse upload" });
|
||||||
|
}
|
||||||
|
|
||||||
|
const { fields, files } = parsed;
|
||||||
|
const fileEntry = files["file"];
|
||||||
|
const file = Array.isArray(fileEntry) ? fileEntry[0] : fileEntry;
|
||||||
|
|
||||||
|
if (!file) {
|
||||||
|
return res.status(400).json({ error: "No file uploaded" });
|
||||||
|
}
|
||||||
|
|
||||||
|
const targetLanguage = getField(fields, "targetLanguage") ?? "en";
|
||||||
|
const sourceLanguage = getField(fields, "sourceLanguage");
|
||||||
|
const action = getField(fields, "action") ?? "translate";
|
||||||
|
const columnSelectionsRaw = getField(fields, "columnSelections");
|
||||||
|
|
||||||
|
const filename = file.originalFilename ?? file.newFilename ?? "file";
|
||||||
|
const ext = path.extname(filename).toLowerCase();
|
||||||
|
const buffer = getFileBuffer(file);
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Action: getColumns - return column info for Excel files
|
||||||
|
if (action === "getColumns") {
|
||||||
|
if (![".xlsx", ".xls", ".csv"].includes(ext)) {
|
||||||
|
return res.status(400).json({ error: "Column selection only supported for Excel/CSV files" });
|
||||||
|
}
|
||||||
|
const columns = getExcelColumns(buffer, filename);
|
||||||
|
return res.status(200).json({ columns });
|
||||||
|
}
|
||||||
|
|
||||||
|
// Action: translate
|
||||||
|
let outBuffer: Buffer;
|
||||||
|
let outMime: string;
|
||||||
|
let outFilename: string;
|
||||||
|
|
||||||
|
if ([".xlsx", ".xls"].includes(ext)) {
|
||||||
|
let columnSelections: ColumnSelection[] = [];
|
||||||
|
if (columnSelectionsRaw) {
|
||||||
|
try {
|
||||||
|
columnSelections = JSON.parse(columnSelectionsRaw);
|
||||||
|
} catch { /* use empty = translate all */ }
|
||||||
|
}
|
||||||
|
outBuffer = await translateExcel(buffer, targetLanguage, columnSelections);
|
||||||
|
outMime = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
|
||||||
|
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||||
|
} else if (ext === ".csv") {
|
||||||
|
// Treat CSV as Excel
|
||||||
|
let columnSelections: ColumnSelection[] = [];
|
||||||
|
if (columnSelectionsRaw) {
|
||||||
|
try {
|
||||||
|
columnSelections = JSON.parse(columnSelectionsRaw);
|
||||||
|
} catch { /* use empty = translate all */ }
|
||||||
|
}
|
||||||
|
outBuffer = await translateExcel(buffer, targetLanguage, columnSelections);
|
||||||
|
outMime = "text/csv";
|
||||||
|
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||||
|
} else if (ext === ".docx") {
|
||||||
|
outBuffer = await translateDocx(buffer, targetLanguage);
|
||||||
|
outMime = "application/vnd.openxmlformats-officedocument.wordprocessingml.document";
|
||||||
|
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||||
|
} else if (ext === ".pdf") {
|
||||||
|
outBuffer = await translatePdf(buffer, targetLanguage, sourceLanguage);
|
||||||
|
outMime = "application/pdf";
|
||||||
|
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||||
|
} else {
|
||||||
|
return res.status(400).json({
|
||||||
|
error: `Unsupported file type: ${ext}. Supported: .pdf, .docx, .xlsx, .xls, .csv`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
res.setHeader("Content-Type", outMime);
|
||||||
|
res.setHeader("Content-Disposition", `attachment; filename="${outFilename}"`);
|
||||||
|
res.setHeader("Content-Length", outBuffer.length);
|
||||||
|
return res.status(200).send(outBuffer);
|
||||||
|
} catch (err) {
|
||||||
|
const msg = err instanceof Error ? err.message : "Translation failed";
|
||||||
|
return res.status(500).json({ error: msg });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
export default handler;
|
||||||
38
pages/api/translate/replicate.ts
Normal file
38
pages/api/translate/replicate.ts
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
import { NextApiHandler } from "next";
|
||||||
|
import NextCors from "nextjs-cors";
|
||||||
|
import { replicateTranslate } from "@utils/replicate-translate";
|
||||||
|
import { readSettings } from "@utils/settings-store";
|
||||||
|
|
||||||
|
type Data = { translation: string } | { error: string };
|
||||||
|
|
||||||
|
const handler: NextApiHandler<Data> = async (req, res) => {
|
||||||
|
await NextCors(req, res, { methods: ["POST"], origin: "*" });
|
||||||
|
|
||||||
|
if (req.method !== "POST") {
|
||||||
|
res.setHeader("Allow", ["POST"]);
|
||||||
|
return res.status(405).json({ error: "Method Not Allowed" });
|
||||||
|
}
|
||||||
|
|
||||||
|
const settings = readSettings();
|
||||||
|
if (!settings.replicateEnabled) {
|
||||||
|
return res.status(503).json({ error: "Replicate translation is not enabled" });
|
||||||
|
}
|
||||||
|
|
||||||
|
const { text, targetLanguage } = req.body ?? {};
|
||||||
|
if (!text || typeof text !== "string") {
|
||||||
|
return res.status(400).json({ error: "text is required" });
|
||||||
|
}
|
||||||
|
if (!targetLanguage || typeof targetLanguage !== "string") {
|
||||||
|
return res.status(400).json({ error: "targetLanguage is required" });
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const translation = await replicateTranslate(text, targetLanguage);
|
||||||
|
return res.status(200).json({ translation });
|
||||||
|
} catch (err) {
|
||||||
|
const msg = err instanceof Error ? err.message : "Translation failed";
|
||||||
|
return res.status(500).json({ error: msg });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
export default handler;
|
||||||
55
utils/admin-auth.ts
Normal file
55
utils/admin-auth.ts
Normal file
@@ -0,0 +1,55 @@
|
|||||||
|
import { SignJWT, jwtVerify } from "jose";
|
||||||
|
import { NextApiRequest, NextApiResponse } from "next";
|
||||||
|
import { readSettings } from "./settings-store";
|
||||||
|
|
||||||
|
const JWT_COOKIE = "lingva_admin";
|
||||||
|
const JWT_EXPIRY = "8h";
|
||||||
|
|
||||||
|
function getSecret(): Uint8Array {
|
||||||
|
const secret = process.env["ADMIN_JWT_SECRET"] ?? "lingva-admin-secret-change-me";
|
||||||
|
return new TextEncoder().encode(secret);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function signAdminToken(): Promise<string> {
|
||||||
|
return new SignJWT({ role: "admin" })
|
||||||
|
.setProtectedHeader({ alg: "HS256" })
|
||||||
|
.setIssuedAt()
|
||||||
|
.setExpirationTime(JWT_EXPIRY)
|
||||||
|
.sign(getSecret());
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function verifyAdminToken(token: string): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
await jwtVerify(token, getSecret());
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getTokenFromRequest(req: NextApiRequest): string | null {
|
||||||
|
const cookie = req.cookies[JWT_COOKIE];
|
||||||
|
if (cookie) return cookie;
|
||||||
|
const auth = req.headers.authorization;
|
||||||
|
if (auth?.startsWith("Bearer ")) return auth.slice(7);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function requireAdmin(
|
||||||
|
req: NextApiRequest,
|
||||||
|
res: NextApiResponse
|
||||||
|
): Promise<boolean> {
|
||||||
|
const token = getTokenFromRequest(req);
|
||||||
|
if (!token || !(await verifyAdminToken(token))) {
|
||||||
|
res.status(401).json({ error: "Unauthorized" });
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function checkPassword(password: string): boolean {
|
||||||
|
const settings = readSettings();
|
||||||
|
return password === settings.adminPasswordHash;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const COOKIE_NAME = JWT_COOKIE;
|
||||||
106
utils/document-processors/docx.ts
Normal file
106
utils/document-processors/docx.ts
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
import JSZip from "jszip";
|
||||||
|
import { replicateTranslateBatch } from "../replicate-translate";
|
||||||
|
|
||||||
|
function escapeXml(text: string): string {
|
||||||
|
return text
|
||||||
|
.replace(/&/g, "&")
|
||||||
|
.replace(/</g, "<")
|
||||||
|
.replace(/>/g, ">")
|
||||||
|
.replace(/"/g, """)
|
||||||
|
.replace(/'/g, "'");
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Extract paragraph texts from document.xml string.
|
||||||
|
* Returns array of {index, text} where index is the paragraph number.
|
||||||
|
*/
|
||||||
|
function extractParagraphs(xml: string): { index: number; text: string; start: number; end: number }[] {
|
||||||
|
const paragraphs: { index: number; text: string; start: number; end: number }[] = [];
|
||||||
|
const pRegex = /<w:p[ >]/g;
|
||||||
|
const pCloseTag = "</w:p>";
|
||||||
|
let idx = 0;
|
||||||
|
let match: RegExpExecArray | null;
|
||||||
|
|
||||||
|
while ((match = pRegex.exec(xml)) !== null) {
|
||||||
|
const start = match.index;
|
||||||
|
const end = xml.indexOf(pCloseTag, start) + pCloseTag.length;
|
||||||
|
if (end < pCloseTag.length) break;
|
||||||
|
|
||||||
|
const paraXml = xml.slice(start, end);
|
||||||
|
|
||||||
|
// Extract all text content within this paragraph
|
||||||
|
const textParts: string[] = [];
|
||||||
|
const tRegex = /<w:t[^>]*>([\s\S]*?)<\/w:t>/g;
|
||||||
|
let tMatch: RegExpExecArray | null;
|
||||||
|
while ((tMatch = tRegex.exec(paraXml)) !== null) {
|
||||||
|
textParts.push(tMatch[1]);
|
||||||
|
}
|
||||||
|
|
||||||
|
const text = textParts.join("").trim();
|
||||||
|
if (text) {
|
||||||
|
paragraphs.push({ index: idx, text, start, end });
|
||||||
|
}
|
||||||
|
idx++;
|
||||||
|
}
|
||||||
|
return paragraphs;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Replace text within a paragraph XML while preserving formatting of first run.
|
||||||
|
* Empties all other text runs.
|
||||||
|
*/
|
||||||
|
function replaceParagraphText(paraXml: string, translatedText: string): string {
|
||||||
|
let firstDone = false;
|
||||||
|
return paraXml.replace(/<w:t([^>]*)>([\s\S]*?)<\/w:t>/g, (_match, attrs, content) => {
|
||||||
|
if (!firstDone && content.trim()) {
|
||||||
|
firstDone = true;
|
||||||
|
return `<w:t xml:space="preserve">${escapeXml(translatedText)}</w:t>`;
|
||||||
|
}
|
||||||
|
if (firstDone) {
|
||||||
|
return `<w:t></w:t>`;
|
||||||
|
}
|
||||||
|
return _match; // preserve empty runs before the first text
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Translate a DOCX file buffer, preserving formatting.
|
||||||
|
*/
|
||||||
|
export async function translateDocx(
|
||||||
|
buffer: Buffer,
|
||||||
|
targetLanguage: string
|
||||||
|
): Promise<Buffer> {
|
||||||
|
const zip = await JSZip.loadAsync(buffer);
|
||||||
|
const docFile = zip.file("word/document.xml");
|
||||||
|
if (!docFile) throw new Error("Invalid DOCX: missing word/document.xml");
|
||||||
|
|
||||||
|
let xml = await docFile.async("string");
|
||||||
|
const paragraphs = extractParagraphs(xml);
|
||||||
|
|
||||||
|
if (paragraphs.length === 0) {
|
||||||
|
return Buffer.from(await zip.generateAsync({ type: "nodebuffer" }));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Translate all paragraphs
|
||||||
|
const translations = await replicateTranslateBatch(
|
||||||
|
paragraphs.map(p => p.text),
|
||||||
|
targetLanguage
|
||||||
|
);
|
||||||
|
|
||||||
|
// Replace paragraphs from end to start to preserve offsets
|
||||||
|
const sorted = [...paragraphs].sort((a, b) => b.start - a.start);
|
||||||
|
for (const para of sorted) {
|
||||||
|
const translationIdx = paragraphs.findIndex(p => p.start === para.start);
|
||||||
|
const translated = translations[translationIdx] ?? para.text;
|
||||||
|
const originalPara = xml.slice(para.start, para.end);
|
||||||
|
const translatedPara = replaceParagraphText(originalPara, translated);
|
||||||
|
xml = xml.slice(0, para.start) + translatedPara + xml.slice(para.end);
|
||||||
|
}
|
||||||
|
|
||||||
|
zip.file("word/document.xml", xml);
|
||||||
|
const outBuffer = await zip.generateAsync({
|
||||||
|
type: "nodebuffer",
|
||||||
|
compression: "DEFLATE"
|
||||||
|
});
|
||||||
|
return Buffer.from(outBuffer);
|
||||||
|
}
|
||||||
95
utils/document-processors/excel.ts
Normal file
95
utils/document-processors/excel.ts
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
import * as XLSX from "xlsx";
|
||||||
|
import { replicateTranslateBatch } from "../replicate-translate";
|
||||||
|
|
||||||
|
export type SheetColumnInfo = {
|
||||||
|
sheetName: string;
|
||||||
|
columns: string[]; // header names or A, B, C...
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ColumnSelection = {
|
||||||
|
sheetName: string;
|
||||||
|
columnIndices: number[]; // 0-based column indices to translate
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse an Excel/CSV buffer and return sheet/column metadata for column selection UI.
|
||||||
|
*/
|
||||||
|
export function getExcelColumns(buffer: Buffer, filename: string): SheetColumnInfo[] {
|
||||||
|
const wb = XLSX.read(buffer, { type: "buffer" });
|
||||||
|
return wb.SheetNames.map(sheetName => {
|
||||||
|
const ws = wb.Sheets[sheetName];
|
||||||
|
const range = XLSX.utils.decode_range(ws["!ref"] ?? "A1");
|
||||||
|
const columns: string[] = [];
|
||||||
|
for (let c = range.s.c; c <= range.e.c; c++) {
|
||||||
|
// Try to get header from first row
|
||||||
|
const cellAddr = XLSX.utils.encode_cell({ r: 0, c });
|
||||||
|
const cell = ws[cellAddr];
|
||||||
|
const header = cell && cell.v != null ? String(cell.v) : XLSX.utils.encode_col(c);
|
||||||
|
columns.push(header);
|
||||||
|
}
|
||||||
|
return { sheetName, columns };
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Translate selected columns in an Excel buffer. Returns translated buffer.
|
||||||
|
* columnSelections: array of {sheetName, columnIndices}
|
||||||
|
* If columnSelections is empty, all text columns are translated.
|
||||||
|
*/
|
||||||
|
export async function translateExcel(
|
||||||
|
buffer: Buffer,
|
||||||
|
targetLanguage: string,
|
||||||
|
columnSelections: ColumnSelection[]
|
||||||
|
): Promise<Buffer> {
|
||||||
|
const wb = XLSX.read(buffer, { type: "buffer", cellStyles: true, cellNF: true });
|
||||||
|
|
||||||
|
for (const sheet of wb.SheetNames) {
|
||||||
|
const ws = wb.Sheets[sheet];
|
||||||
|
if (!ws["!ref"]) continue;
|
||||||
|
|
||||||
|
const range = XLSX.utils.decode_range(ws["!ref"]);
|
||||||
|
const selection = columnSelections.find(s => s.sheetName === sheet);
|
||||||
|
const columnsToTranslate = selection
|
||||||
|
? selection.columnIndices
|
||||||
|
: Array.from({ length: range.e.c - range.s.c + 1 }, (_, i) => i + range.s.c);
|
||||||
|
|
||||||
|
// Collect all text cells for batch translation
|
||||||
|
type CellRef = { addr: string; text: string };
|
||||||
|
const cellRefs: CellRef[] = [];
|
||||||
|
|
||||||
|
for (const colIdx of columnsToTranslate) {
|
||||||
|
// Start from row 1 to skip headers (row 0)
|
||||||
|
for (let r = range.s.r + 1; r <= range.e.r; r++) {
|
||||||
|
const addr = XLSX.utils.encode_cell({ r, c: colIdx });
|
||||||
|
const cell = ws[addr];
|
||||||
|
if (cell && cell.t === "s" && typeof cell.v === "string" && cell.v.trim()) {
|
||||||
|
cellRefs.push({ addr, text: cell.v });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (cellRefs.length === 0) continue;
|
||||||
|
|
||||||
|
// Translate in batches of 50
|
||||||
|
const BATCH_SIZE = 50;
|
||||||
|
for (let i = 0; i < cellRefs.length; i += BATCH_SIZE) {
|
||||||
|
const batch = cellRefs.slice(i, i + BATCH_SIZE);
|
||||||
|
const translations = await replicateTranslateBatch(
|
||||||
|
batch.map(c => c.text),
|
||||||
|
targetLanguage
|
||||||
|
);
|
||||||
|
batch.forEach((cellRef, idx) => {
|
||||||
|
const cell = ws[cellRef.addr];
|
||||||
|
if (cell) {
|
||||||
|
cell.v = translations[idx];
|
||||||
|
if (cell.h) cell.h = translations[idx];
|
||||||
|
if (cell.r) cell.r = undefined;
|
||||||
|
if (cell.w) cell.w = translations[idx];
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const out = XLSX.write(wb, { type: "buffer", bookType: "xlsx" });
|
||||||
|
return Buffer.from(out);
|
||||||
|
}
|
||||||
12
utils/document-processors/pdf-types.d.ts
vendored
Normal file
12
utils/document-processors/pdf-types.d.ts
vendored
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
declare module "pdf-parse/lib/pdf-parse.js" {
|
||||||
|
interface PdfData {
|
||||||
|
numpages: number;
|
||||||
|
numrender: number;
|
||||||
|
info: Record<string, unknown>;
|
||||||
|
metadata: Record<string, unknown>;
|
||||||
|
text: string;
|
||||||
|
version: string;
|
||||||
|
}
|
||||||
|
function pdfParse(dataBuffer: Buffer, options?: Record<string, unknown>): Promise<PdfData>;
|
||||||
|
export = pdfParse;
|
||||||
|
}
|
||||||
115
utils/document-processors/pdf.ts
Normal file
115
utils/document-processors/pdf.ts
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
import { PDFDocument, rgb, StandardFonts } from "pdf-lib";
|
||||||
|
import { replicateTranslateBatch } from "../replicate-translate";
|
||||||
|
|
||||||
|
type PdfParseResult = {
|
||||||
|
numpages: number;
|
||||||
|
text: string;
|
||||||
|
info: Record<string, unknown>;
|
||||||
|
};
|
||||||
|
|
||||||
|
async function parsePdf(buffer: Buffer): Promise<PdfParseResult> {
|
||||||
|
// Avoid Next.js issues with pdf-parse test file imports
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-var-requires
|
||||||
|
const pdfParse = require("pdf-parse/lib/pdf-parse.js");
|
||||||
|
return pdfParse(buffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
function wrapText(text: string, maxCharsPerLine: number): string[] {
|
||||||
|
const words = text.split(/\s+/);
|
||||||
|
const lines: string[] = [];
|
||||||
|
let current = "";
|
||||||
|
|
||||||
|
for (const word of words) {
|
||||||
|
if ((current + " " + word).trim().length > maxCharsPerLine) {
|
||||||
|
if (current) lines.push(current);
|
||||||
|
current = word;
|
||||||
|
} else {
|
||||||
|
current = current ? current + " " + word : word;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (current) lines.push(current);
|
||||||
|
return lines;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Translate a PDF buffer. Since PDFs don't support in-place text editing,
|
||||||
|
* this extracts text, translates it, and creates a new formatted PDF.
|
||||||
|
*/
|
||||||
|
export async function translatePdf(
|
||||||
|
buffer: Buffer,
|
||||||
|
targetLanguage: string,
|
||||||
|
sourceLanguage?: string
|
||||||
|
): Promise<Buffer> {
|
||||||
|
const parsed = await parsePdf(buffer);
|
||||||
|
const rawText = parsed.text;
|
||||||
|
|
||||||
|
// Split into paragraphs (separated by double newlines or page breaks)
|
||||||
|
const paragraphs = rawText
|
||||||
|
.split(/\n{2,}|\f/)
|
||||||
|
.map(p => p.replace(/\n/g, " ").trim())
|
||||||
|
.filter(p => p.length > 0);
|
||||||
|
|
||||||
|
if (paragraphs.length === 0) {
|
||||||
|
throw new Error("No extractable text found in PDF");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Translate all paragraphs
|
||||||
|
const translations = await replicateTranslateBatch(paragraphs, targetLanguage);
|
||||||
|
|
||||||
|
// Build output PDF
|
||||||
|
const pdfDoc = await PDFDocument.create();
|
||||||
|
const font = await pdfDoc.embedFont(StandardFonts.Helvetica);
|
||||||
|
const boldFont = await pdfDoc.embedFont(StandardFonts.HelveticaBold);
|
||||||
|
|
||||||
|
const PAGE_WIDTH = 595;
|
||||||
|
const PAGE_HEIGHT = 842;
|
||||||
|
const MARGIN = 50;
|
||||||
|
const FONT_SIZE = 11;
|
||||||
|
const TITLE_SIZE = 13;
|
||||||
|
const LINE_HEIGHT = 16;
|
||||||
|
const MAX_LINE_CHARS = 80;
|
||||||
|
|
||||||
|
let page = pdfDoc.addPage([PAGE_WIDTH, PAGE_HEIGHT]);
|
||||||
|
let y = PAGE_HEIGHT - MARGIN;
|
||||||
|
|
||||||
|
function ensureSpace(needed: number) {
|
||||||
|
if (y - needed < MARGIN) {
|
||||||
|
page = pdfDoc.addPage([PAGE_WIDTH, PAGE_HEIGHT]);
|
||||||
|
y = PAGE_HEIGHT - MARGIN;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Title
|
||||||
|
const title = `Translation to: ${targetLanguage}${sourceLanguage ? ` (from: ${sourceLanguage})` : ""}`;
|
||||||
|
ensureSpace(TITLE_SIZE + LINE_HEIGHT);
|
||||||
|
page.drawText(title, {
|
||||||
|
x: MARGIN,
|
||||||
|
y,
|
||||||
|
size: TITLE_SIZE,
|
||||||
|
font: boldFont,
|
||||||
|
color: rgb(0.2, 0.2, 0.7)
|
||||||
|
});
|
||||||
|
y -= TITLE_SIZE + LINE_HEIGHT;
|
||||||
|
|
||||||
|
// Draw translated paragraphs
|
||||||
|
for (const para of translations) {
|
||||||
|
const lines = wrapText(para, MAX_LINE_CHARS);
|
||||||
|
ensureSpace(lines.length * LINE_HEIGHT + LINE_HEIGHT);
|
||||||
|
|
||||||
|
for (const line of lines) {
|
||||||
|
ensureSpace(LINE_HEIGHT);
|
||||||
|
page.drawText(line, {
|
||||||
|
x: MARGIN,
|
||||||
|
y,
|
||||||
|
size: FONT_SIZE,
|
||||||
|
font,
|
||||||
|
color: rgb(0, 0, 0)
|
||||||
|
});
|
||||||
|
y -= LINE_HEIGHT;
|
||||||
|
}
|
||||||
|
y -= LINE_HEIGHT * 0.5; // paragraph gap
|
||||||
|
}
|
||||||
|
|
||||||
|
const pdfBytes = await pdfDoc.save();
|
||||||
|
return Buffer.from(pdfBytes);
|
||||||
|
}
|
||||||
83
utils/replicate-translate.ts
Normal file
83
utils/replicate-translate.ts
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
import { readSettings } from "./settings-store";
|
||||||
|
|
||||||
|
type ReplicateOutput = string | string[] | { translation?: string; translated_text?: string; output?: string };
|
||||||
|
|
||||||
|
export async function replicateTranslate(
|
||||||
|
text: string,
|
||||||
|
targetLanguage: string
|
||||||
|
): Promise<string> {
|
||||||
|
const settings = readSettings();
|
||||||
|
|
||||||
|
if (!settings.replicateApiToken) {
|
||||||
|
throw new Error("Replicate API token not configured");
|
||||||
|
}
|
||||||
|
if (!settings.jigsawApiKey) {
|
||||||
|
throw new Error("JigsawStack API key not configured");
|
||||||
|
}
|
||||||
|
|
||||||
|
const body = {
|
||||||
|
version: settings.modelVersion,
|
||||||
|
input: {
|
||||||
|
text,
|
||||||
|
api_key: settings.jigsawApiKey,
|
||||||
|
target_language: targetLanguage
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const response = await fetch("https://api.replicate.com/v1/predictions", {
|
||||||
|
method: "POST",
|
||||||
|
headers: {
|
||||||
|
"Authorization": `Bearer ${settings.replicateApiToken}`,
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Prefer": "wait"
|
||||||
|
},
|
||||||
|
body: JSON.stringify(body)
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
const err = await response.text();
|
||||||
|
throw new Error(`Replicate API error: ${response.status} ${err}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const data = await response.json();
|
||||||
|
|
||||||
|
if (data.error) {
|
||||||
|
throw new Error(`Replicate model error: ${data.error}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract translated text from various output formats
|
||||||
|
const output: ReplicateOutput = data.output;
|
||||||
|
|
||||||
|
if (typeof output === "string") return output;
|
||||||
|
if (Array.isArray(output)) return output.join("");
|
||||||
|
if (output && typeof output === "object") {
|
||||||
|
return output.translation ?? output.translated_text ?? output.output ?? String(output);
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error("Unexpected output format from Replicate");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Batch translate using separator trick to minimize API calls
|
||||||
|
const SEPARATOR = "\n{{SEP}}\n";
|
||||||
|
|
||||||
|
export async function replicateTranslateBatch(
|
||||||
|
texts: string[],
|
||||||
|
targetLanguage: string
|
||||||
|
): Promise<string[]> {
|
||||||
|
if (texts.length === 0) return [];
|
||||||
|
if (texts.length === 1) {
|
||||||
|
return [await replicateTranslate(texts[0], targetLanguage)];
|
||||||
|
}
|
||||||
|
|
||||||
|
const joined = texts.join(SEPARATOR);
|
||||||
|
const translated = await replicateTranslate(joined, targetLanguage);
|
||||||
|
|
||||||
|
// Try to split on the separator; fall back to individual calls if it got translated
|
||||||
|
const parts = translated.split(SEPARATOR);
|
||||||
|
if (parts.length === texts.length) {
|
||||||
|
return parts;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fallback: translate individually
|
||||||
|
return Promise.all(texts.map(t => replicateTranslate(t, targetLanguage)));
|
||||||
|
}
|
||||||
48
utils/settings-store.ts
Normal file
48
utils/settings-store.ts
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
import fs from "fs";
|
||||||
|
import path from "path";
|
||||||
|
|
||||||
|
export type Settings = {
|
||||||
|
replicateApiToken: string;
|
||||||
|
jigsawApiKey: string;
|
||||||
|
modelVersion: string;
|
||||||
|
replicateEnabled: boolean;
|
||||||
|
adminPasswordHash: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
const DEFAULT_SETTINGS: Settings = {
|
||||||
|
replicateApiToken: process.env["REPLICATE_API_TOKEN"] ?? "",
|
||||||
|
jigsawApiKey: process.env["JIGSAWSTACK_API_KEY"] ?? "",
|
||||||
|
modelVersion: "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89",
|
||||||
|
replicateEnabled: false,
|
||||||
|
adminPasswordHash: process.env["ADMIN_PASSWORD"] ?? "admin"
|
||||||
|
};
|
||||||
|
|
||||||
|
const SETTINGS_PATH = path.join(process.cwd(), "data", "settings.json");
|
||||||
|
|
||||||
|
function ensureDataDir() {
|
||||||
|
const dir = path.dirname(SETTINGS_PATH);
|
||||||
|
if (!fs.existsSync(dir)) {
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function readSettings(): Settings {
|
||||||
|
try {
|
||||||
|
ensureDataDir();
|
||||||
|
if (!fs.existsSync(SETTINGS_PATH)) {
|
||||||
|
return { ...DEFAULT_SETTINGS };
|
||||||
|
}
|
||||||
|
const raw = fs.readFileSync(SETTINGS_PATH, "utf-8");
|
||||||
|
return { ...DEFAULT_SETTINGS, ...JSON.parse(raw) };
|
||||||
|
} catch {
|
||||||
|
return { ...DEFAULT_SETTINGS };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function writeSettings(updates: Partial<Settings>): Settings {
|
||||||
|
ensureDataDir();
|
||||||
|
const current = readSettings();
|
||||||
|
const next = { ...current, ...updates };
|
||||||
|
fs.writeFileSync(SETTINGS_PATH, JSON.stringify(next, null, 2), "utf-8");
|
||||||
|
return next;
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user