feat: add admin panel, Replicate AI translation, and document translation
- Admin panel (/admin) with JWT auth: configure Replicate API token, JigsawStack API key, model version, enable/disable AI translation, change admin password. Settings persisted in data/settings.json. - Replicate AI translation: POST /api/translate/replicate uses JigsawStack text-translate model via Replicate API. Main page switches to client-side AI translation when enabled. - Document translation tab: supports PDF, DOCX, XLSX, XLS, CSV. Excel/Word formatting fully preserved (SheetJS + JSZip XML manipulation). PDF uses pdf-parse extraction + pdf-lib reconstruction. Column selector UI for tabular data (per-sheet, All/None toggles). - Updated README with full implementation documentation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -45,3 +45,6 @@ cypress/screenshots
|
||||
**/public/workbox-*.js*
|
||||
**/public/sw.js*
|
||||
**/public/worker-*.js*
|
||||
|
||||
# admin settings (contains API keys)
|
||||
/data/settings.json
|
||||
|
||||
397
README.md
397
README.md
@@ -1,241 +1,256 @@
|
||||
# Lingva Translate
|
||||
# LingvAI
|
||||
|
||||
<img src="public/logo.svg" width="128" align="right">
|
||||
|
||||
[](https://travis-ci.com/thedaviddelta/lingva-translate)
|
||||
[](https://lingva.ml/)
|
||||
[](https://dashboard.cypress.io/projects/qgjdyd/runs)
|
||||
[](./LICENSE)
|
||||
[](https://github.com/humanetech-community/awesome-humane-tech)
|
||||
[<img src="https://www.datocms-assets.com/31049/1618983297-powered-by-vercel.svg" alt="Powered by Vercel" height="20">](https://vercel.com?utm_source=lingva-team&utm_campaign=oss)
|
||||
|
||||
Alternative front-end for Google Translate, serving as a Free and Open Source translator with over a hundred languages available
|
||||
**LingvAI** is an enhanced fork of [Lingva Translate](https://github.com/thedaviddelta/lingva-translate) — a privacy-respecting alternative front-end for Google Translate — extended with **Replicate AI-powered translation**, an **admin panel**, and **document translation** (PDF, Word, Excel, CSV) with full formatting preservation.
|
||||
|
||||
---
|
||||
|
||||
## How does it work?
|
||||
## Features
|
||||
|
||||
Inspired by projects like [NewPipe](https://github.com/TeamNewPipe/NewPipe), [Nitter](https://github.com/zedeus/nitter), [Invidious](https://github.com/iv-org/invidious) or [Bibliogram](https://git.sr.ht/~cadence/bibliogram), *Lingva* scrapes through Google Translate and retrieves the translation without directly accessing any Google-related service, preventing them from tracking.
|
||||
### Core (from Lingva Translate)
|
||||
- 100+ languages via Google Translate scraper (no tracking)
|
||||
- Audio playback for source and translated text
|
||||
- Auto-translate mode
|
||||
- GraphQL and REST API
|
||||
- PWA support (installable)
|
||||
- Dark/light mode
|
||||
|
||||
For this purpose, *Lingva* is built, among others, with the following Open Source resources:
|
||||
### New in LingvAI
|
||||
|
||||
+ [Lingva Scraper](https://github.com/thedaviddelta/lingva-scraper), a Google Translate scraper built and maintained specifically for this project, which obtains all kind of information from this platform.
|
||||
+ [TypeScript](https://www.typescriptlang.org/), the JavaScript superset, as the language.
|
||||
+ [React](https://reactjs.org/) as the main front-end framework.
|
||||
+ [Next.js](https://nextjs.org/) as the complementary React framework, that provides Server-Side Rendering, Static Site Generation or serverless API endpoints.
|
||||
+ [ChakraUI](https://chakra-ui.com/) for the in-component styling.
|
||||
+ [Jest](https://jestjs.io/), [Testing Library](https://testing-library.com/) & [Cypress](https://www.cypress.io/) for unit, integration & E2E testing.
|
||||
+ [Apollo Server](https://www.apollographql.com/docs/apollo-server/) for handling the GraphQL endpoint.
|
||||
+ [Inkscape](https://inkscape.org/) for designing both the logo and the banner.
|
||||
#### Admin Panel (`/admin`)
|
||||
- Password-protected settings dashboard (gear icon in header)
|
||||
- Configure Replicate API token and JigsawStack API key
|
||||
- Select/change the Replicate model version
|
||||
- Enable/disable AI translation per-instance
|
||||
- Test translation button
|
||||
- Change admin password
|
||||
- Settings stored server-side in `data/settings.json` (never committed)
|
||||
|
||||
#### Replicate AI Translation
|
||||
- When enabled in admin, uses [Replicate](https://replicate.com) + [JigsawStack](https://jigsawstack.com) text-translate model
|
||||
- Replaces Google Translate scraper with AI translation when active
|
||||
- Falls back to original lingva-scraper when Replicate is disabled
|
||||
- Batch translation with separator trick for efficiency
|
||||
|
||||
## Deployment
|
||||
#### Document Translation (new "Document" tab)
|
||||
Translate whole documents while preserving original formatting:
|
||||
|
||||
As *Lingva* is a [Next.js](https://nextjs.org/) project you can deploy your own instance anywhere Next is supported.
|
||||
| Format | Formatting | Notes |
|
||||
|--------|-----------|-------|
|
||||
| `.xlsx` / `.xls` | **Fully preserved** | Cell styles, formulas, column widths intact. Select which columns to translate. |
|
||||
| `.docx` | **Fully preserved** | Fonts, tables, images, paragraph styles preserved via XML manipulation |
|
||||
| `.csv` | Structure preserved | Column selection supported |
|
||||
| `.pdf` | Best-effort | Text extracted, translated, new formatted PDF generated |
|
||||
|
||||
The only requirement is to set an environment variable called `NEXT_PUBLIC_SITE_DOMAIN` with the domain you're deploying the instance under. This is used for the canonical URL and the meta tags.
|
||||
- Drag-and-drop or click-to-upload (up to 50 MB)
|
||||
- **Column selector** for Excel/CSV: choose individual columns, use All/None toggles per sheet
|
||||
- Download translated file (named `original_<lang>.ext`)
|
||||
|
||||
Optionally, there are other environment variables available:
|
||||
+ `NEXT_PUBLIC_FORCE_DEFAULT_THEME`: Force a certain theme over the system preference set by the user. The accepted values are `light` and `dark`.
|
||||
+ `NEXT_PUBLIC_DEFAULT_SOURCE_LANG`: Set an initial *source* language instead of the default `auto`.
|
||||
+ `NEXT_PUBLIC_DEFAULT_TARGET_LANG`: Set an initial *target* language instead of the default `en`.
|
||||
---
|
||||
|
||||
### Docker
|
||||
## Getting Started
|
||||
|
||||
An [official Docker image](https://hub.docker.com/r/thedaviddelta/lingva-translate) is available to ease the deployment using Compose, Kubernetes or similar technologies. Remember to also include the environment variables (simplified to `site_domain`, `force_default_theme`, `default_source_lang` and `default_target_lang`) when running the container.
|
||||
### Prerequisites
|
||||
- Node.js 16+
|
||||
- npm
|
||||
|
||||
#### Docker Compose:
|
||||
|
||||
```
|
||||
version: '3'
|
||||
|
||||
services:
|
||||
|
||||
lingva:
|
||||
container_name: lingva
|
||||
image: thedaviddelta/lingva-translate:latest
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- site_domain=lingva.ml
|
||||
- force_default_theme=light
|
||||
- default_source_lang=auto
|
||||
- default_target_lang=en
|
||||
ports:
|
||||
- 3000:3000
|
||||
```
|
||||
|
||||
#### Docker Run
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
docker run -p 3000:3000 -e site_domain=lingva.ml -e force_default_theme=light -e default_source_lang=auto -e default_target_lang=en thedaviddelta/lingva-translate:latest
|
||||
git clone https://devops.cloudhost.es/CloudHost/LingvAI.git
|
||||
cd LingvAI
|
||||
npm install
|
||||
```
|
||||
|
||||
### Vercel
|
||||
### Environment Variables
|
||||
|
||||
Another easy way is to use the Next.js creators' own platform, [Vercel](https://vercel.com/), where you can deploy it for free with the following button.
|
||||
Create a `.env.local` file:
|
||||
|
||||
[](https://vercel.com/new/git/external?repository-url=https%3A%2F%2Fgithub.com%2Fthedaviddelta%2Flingva-translate%2Ftree%2Fmain&env=NEXT_PUBLIC_SITE_DOMAIN&envDescription=Your%20domain&utm_source=lingva-team&utm_campaign=oss)
|
||||
```env
|
||||
# Admin panel
|
||||
ADMIN_PASSWORD=your_secure_password # Default: admin
|
||||
ADMIN_JWT_SECRET=random_secret_string # Used to sign admin session tokens
|
||||
|
||||
# Replicate AI (optional - can also be set via admin panel)
|
||||
REPLICATE_API_TOKEN=r8_...
|
||||
JIGSAWSTACK_API_KEY=sk_...
|
||||
|
||||
## Instances
|
||||
|
||||
These are the currently known *Lingva* instances. Feel free to make a Pull Request including yours (please remember to add `[skip ci]` to the last commit).
|
||||
|
||||
| Domain | Hosting | SSL Provider |
|
||||
|:---------------------------------------------------------------------------:|:-----------------------------------------:|:-----------------------------------------------------------------------------------------------:|
|
||||
| [lingva.ml](https://lingva.ml/) (Official) | [Vercel](https://vercel.com/) | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=lingva.ml) |
|
||||
| [translate.igna.wtf](https://translate.igna.wtf/) | [Vercel](https://vercel.com/) | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.igna.wtf) |
|
||||
| [translate.plausibility.cloud](https://translate.plausibility.cloud/) | [Hetzner](https://hetzner.com/) | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.plausibility.cloud) |
|
||||
| [lingva.lunar.icu](https://lingva.lunar.icu/) | [Lansol](https://lansol.de/) | [Cloudflare](https://www.ssllabs.com/ssltest/analyze.html?d=lingva.lunar.icu) |
|
||||
| [translate.projectsegfau.lt](https://translate.projectsegfau.lt/) | Self-hosted | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.projectsegfau.lt) |
|
||||
| [translate.dr460nf1r3.org](https://translate.dr460nf1r3.org/) | [Netcup](https://netcup.eu/) | [Cloudflare](https://www.ssllabs.com/ssltest/analyze.html?d=translate.dr460nf1r3.org) |
|
||||
| [lingva.garudalinux.org](https://lingva.garudalinux.org/) | [Hetzner](https://hetzner.com/) | [Cloudflare](https://www.ssllabs.com/ssltest/analyze.html?d=lingva.garudalinux.org) |
|
||||
| [translate.jae.fi](https://translate.jae.fi/) | Self-hosted | [Let's Encrypt](https://www.ssllabs.com/ssltest/analyze.html?d=translate.jae.fi) |
|
||||
|
||||
|
||||
## Public APIs
|
||||
|
||||
Nearly all the *Lingva* instances should supply a pair of public developer APIs: a RESTful one and a GraphQL one.
|
||||
|
||||
*Note: both APIs return the translation audio as a `Uint8Array` (served as `number[]` in JSON and `[Int]` in GraphQL) with the contents of the audio buffer.*
|
||||
|
||||
### REST API v1
|
||||
|
||||
+ GET `/api/v1/:source/:target/:query`
|
||||
```typescript
|
||||
{
|
||||
translation: string
|
||||
info?: TranslationInfo
|
||||
}
|
||||
# Optional: override default languages
|
||||
NEXT_PUBLIC_DEFAULT_SOURCE_LANG=auto
|
||||
NEXT_PUBLIC_DEFAULT_TARGET_LANG=en
|
||||
```
|
||||
|
||||
+ GET `/api/v1/audio/:lang/:query`
|
||||
```typescript
|
||||
{
|
||||
audio: number[]
|
||||
}
|
||||
### Running
|
||||
|
||||
```bash
|
||||
# Development
|
||||
npm run dev
|
||||
|
||||
# Production
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
+ GET `/api/v1/languages/?:(source|target)`
|
||||
```typescript
|
||||
{
|
||||
languages: [
|
||||
{
|
||||
code: string,
|
||||
name: string
|
||||
}
|
||||
]
|
||||
}
|
||||
The app runs on [http://localhost:3000](http://localhost:3000) by default.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Setting up Replicate AI Translation
|
||||
|
||||
1. Open the app and click the **gear icon** (⚙) in the top-right header
|
||||
2. Log in with your admin password (default: `admin`)
|
||||
3. Enter your **Replicate API token** — get one at [replicate.com/account/api-tokens](https://replicate.com/account/api-tokens)
|
||||
4. Enter your **JigsawStack API key** — get one at [jigsawstack.com](https://jigsawstack.com)
|
||||
5. Optionally change the **model version** (default is the JigsawStack text-translate model)
|
||||
6. Toggle **Enable Replicate Translation** on
|
||||
7. Click **Save Settings**
|
||||
8. Use **Test Translation** to verify it works
|
||||
|
||||
Default model:
|
||||
```
|
||||
jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89
|
||||
```
|
||||
|
||||
In addition, every endpoint can return an error message with the following structure instead.
|
||||
```typescript
|
||||
{
|
||||
error: string
|
||||
}
|
||||
```
|
||||
### Replicate API call format
|
||||
|
||||
### GraphQL API
|
||||
|
||||
+ `/api/graphql`
|
||||
```graphql
|
||||
query {
|
||||
translation(source: String target: String query: String!) {
|
||||
source: {
|
||||
lang: {
|
||||
code: String!
|
||||
name: String!
|
||||
}
|
||||
text: String!
|
||||
audio: [Int]!
|
||||
detected: {
|
||||
code: String
|
||||
name: String
|
||||
}
|
||||
typo: String
|
||||
pronunciation: String
|
||||
definitions: {
|
||||
type: String
|
||||
list: {
|
||||
definition: String
|
||||
example: String
|
||||
field: String
|
||||
synonyms: [String]
|
||||
}
|
||||
}
|
||||
examples: [String]
|
||||
similar: [String]
|
||||
}
|
||||
target: {
|
||||
lang: {
|
||||
code: String!
|
||||
name: String!
|
||||
}
|
||||
text: String!
|
||||
audio: [Int]!
|
||||
pronunciation: String
|
||||
extraTranslations: {
|
||||
type: String
|
||||
list: {
|
||||
word: String
|
||||
article: String
|
||||
frequency: Int
|
||||
meanings: [String]
|
||||
}
|
||||
}
|
||||
}
|
||||
```bash
|
||||
curl -X POST \
|
||||
-H "Authorization: Bearer $REPLICATE_API_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Prefer: wait" \
|
||||
-d '{
|
||||
"version": "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89",
|
||||
"input": {
|
||||
"text": "Hello, world!",
|
||||
"api_key": "<jigsawstack_api_key>",
|
||||
"target_language": "es"
|
||||
}
|
||||
audio(lang: String! query: String!) {
|
||||
lang: {
|
||||
code: String!
|
||||
name: String!
|
||||
}
|
||||
text: String!
|
||||
audio: [Int]!
|
||||
}
|
||||
languages(type: SOURCE|TARGET) {
|
||||
code: String!
|
||||
name: String!
|
||||
}
|
||||
}
|
||||
}' \
|
||||
https://api.replicate.com/v1/predictions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related projects
|
||||
## API Endpoints
|
||||
|
||||
+ [Lingva Scraper](https://github.com/thedaviddelta/lingva-scraper) - Google Translate scraper built and maintained specifically for this project
|
||||
+ [SimplyTranslate](https://codeberg.org/SimpleWeb/SimplyTranslate-Web) - Very simple translation front-end with multi-engine support
|
||||
+ [LibreTranslate](https://github.com/LibreTranslate/LibreTranslate) - FOSS translation service that uses the open [Argos](https://github.com/argosopentech/argos-translate) engine
|
||||
+ [Lentil for Android](https://github.com/yaxarat/lingvaandroid) - Unofficial native client for Android that uses Lingva's public API
|
||||
+ [Arna Translate](https://github.com/MahanRahmati/translate) - Unofficial cross-platform native client that uses Lingva's public API
|
||||
+ [Translate-UT](https://github.com/walking-octopus/translate-ut) - Unofficial native client for Ubuntu Touch that uses Lingva's public API
|
||||
### Original REST API
|
||||
```
|
||||
GET /api/v1/:source/:target/:query → { translation, info }
|
||||
GET /api/v1/audio/:lang/:text → { audio: number[] }
|
||||
GET /api/v1/languages → { languages }
|
||||
```
|
||||
|
||||
### GraphQL
|
||||
```
|
||||
POST /api/graphql
|
||||
```
|
||||
|
||||
## Contributors
|
||||
### New Endpoints
|
||||
|
||||
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
|
||||
```
|
||||
POST /api/translate/replicate
|
||||
Body: { text: string, targetLanguage: string }
|
||||
Returns: { translation: string }
|
||||
|
||||
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
|
||||
<!-- prettier-ignore-start -->
|
||||
<!-- markdownlint-disable -->
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center"><a href="https://thedaviddelta.com/"><img src="https://avatars.githubusercontent.com/u/6679900?v=4?s=100" width="100px;" alt=""/><br /><sub><b>David</b></sub></a><br /><a href="#a11y-TheDavidDelta" title="Accessibility">️️️️♿️</a> <a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheDavidDelta" title="Code">💻</a> <a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheDavidDelta" title="Documentation">📖</a> <a href="#design-TheDavidDelta" title="Design">🎨</a> <a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheDavidDelta" title="Tests">⚠️</a></td>
|
||||
<td align="center"><a href="https://github.com/mhmdanas"><img src="https://avatars.githubusercontent.com/u/32234660?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Mohammed Anas</b></sub></a><br /><a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=mhmdanas" title="Code">💻</a></td>
|
||||
<td align="center"><a href="https://PussTheCat.org/"><img src="https://avatars.githubusercontent.com/u/47571719?v=4?s=100" width="100px;" alt=""/><br /><sub><b>TheFrenchGhosty</b></sub></a><br /><a href="https://github.com/TheDavidDelta/lingva-translate/commits?author=TheFrenchGhosty" title="Documentation">📖</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
POST /api/translate/document
|
||||
Body: multipart/form-data
|
||||
file: <file>
|
||||
targetLanguage: string
|
||||
action: "translate" | "getColumns"
|
||||
columnSelections?: JSON string (for Excel/CSV)
|
||||
Returns: file download (translate) or { columns } (getColumns)
|
||||
|
||||
<!-- markdownlint-restore -->
|
||||
<!-- prettier-ignore-end -->
|
||||
GET /api/admin/auth → { authenticated: boolean }
|
||||
POST /api/admin/auth body: { password } → sets session cookie
|
||||
DELETE /api/admin/auth → clears session cookie
|
||||
|
||||
<!-- ALL-CONTRIBUTORS-LIST:END -->
|
||||
GET /api/admin/settings → { replicateApiToken, jigsawApiKey, modelVersion, replicateEnabled }
|
||||
POST /api/admin/settings body: { ...settings, newPassword? }
|
||||
```
|
||||
|
||||
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
pages/
|
||||
[[...slug]].tsx Main translation page (Text + Document tabs)
|
||||
admin/index.tsx Admin settings panel
|
||||
api/
|
||||
admin/
|
||||
auth.ts JWT-based admin authentication
|
||||
settings.ts Settings read/write (requires admin auth)
|
||||
translate/
|
||||
replicate.ts Replicate AI translation endpoint
|
||||
document.ts Document upload & translation endpoint
|
||||
v1/[[...slug]].ts Original REST API
|
||||
graphql.ts Original GraphQL API
|
||||
|
||||
components/
|
||||
DocumentTranslator.tsx File upload UI, progress, download
|
||||
ColumnSelector.tsx Per-sheet column selection for Excel/CSV
|
||||
Header.tsx + admin gear icon link
|
||||
|
||||
utils/
|
||||
settings-store.ts Read/write data/settings.json
|
||||
admin-auth.ts JWT sign/verify helpers
|
||||
replicate-translate.ts Replicate API calls + batch helper
|
||||
document-processors/
|
||||
excel.ts SheetJS Excel/CSV processor
|
||||
docx.ts JSZip + XML DOCX processor
|
||||
pdf.ts pdf-parse + pdf-lib PDF processor
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docker
|
||||
|
||||
```dockerfile
|
||||
FROM node:18-alpine
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN npm install && npm run build
|
||||
EXPOSE 3000
|
||||
CMD ["npm", "start"]
|
||||
```
|
||||
|
||||
Or using the included `Dockerfile`:
|
||||
```bash
|
||||
docker build -t lingvai .
|
||||
docker run -p 3000:3000 \
|
||||
-e ADMIN_PASSWORD=secret \
|
||||
-e ADMIN_JWT_SECRET=random \
|
||||
-v ./data:/app/data \
|
||||
lingvai
|
||||
```
|
||||
|
||||
> Mount `./data` as a volume to persist admin settings across container restarts.
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|-------|-----------|
|
||||
| Framework | Next.js 12, React 18, TypeScript |
|
||||
| UI | Chakra UI 2, Framer Motion |
|
||||
| Translation (default) | lingva-scraper (Google Translate) |
|
||||
| Translation (AI) | Replicate + JigsawStack text-translate |
|
||||
| Document processing | SheetJS (xlsx), JSZip, pdf-lib, pdf-parse |
|
||||
| Admin auth | jose (JWT), HTTP-only cookie |
|
||||
| File uploads | formidable v3 |
|
||||
| API | REST + GraphQL (Apollo Server) |
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
[](https://www.gnu.org/licenses/agpl-3.0.html)
|
||||
[AGPL-3.0](./LICENSE) — same as the upstream Lingva Translate project.
|
||||
|
||||
Copyright © 2021 [thedaviddelta](https://github.com/thedaviddelta) & contributors.
|
||||
This project is [GNU AGPLv3](./LICENSE) licensed.
|
||||
Original project by [thedaviddelta](https://github.com/thedaviddelta/lingva-translate).
|
||||
LingvAI enhancements: admin panel, Replicate AI integration, document translation.
|
||||
|
||||
107
components/ColumnSelector.tsx
Normal file
107
components/ColumnSelector.tsx
Normal file
@@ -0,0 +1,107 @@
|
||||
import { FC, useState, useEffect } from "react";
|
||||
import {
|
||||
Box, Checkbox, CheckboxGroup, VStack, HStack, Text, Button,
|
||||
Accordion, AccordionItem, AccordionButton, AccordionPanel, AccordionIcon,
|
||||
Badge
|
||||
} from "@chakra-ui/react";
|
||||
|
||||
export type SheetColumnInfo = {
|
||||
sheetName: string;
|
||||
columns: string[];
|
||||
};
|
||||
|
||||
export type ColumnSelection = {
|
||||
sheetName: string;
|
||||
columnIndices: number[];
|
||||
};
|
||||
|
||||
type Props = {
|
||||
sheetColumns: SheetColumnInfo[];
|
||||
onChange: (selections: ColumnSelection[]) => void;
|
||||
};
|
||||
|
||||
const ColumnSelector: FC<Props> = ({ sheetColumns, onChange }) => {
|
||||
const [selections, setSelections] = useState<Record<string, Set<number>>>(() => {
|
||||
const init: Record<string, Set<number>> = {};
|
||||
sheetColumns.forEach(s => {
|
||||
init[s.sheetName] = new Set(s.columns.map((_, i) => i));
|
||||
});
|
||||
return init;
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
const result: ColumnSelection[] = sheetColumns.map(s => ({
|
||||
sheetName: s.sheetName,
|
||||
columnIndices: Array.from(selections[s.sheetName] ?? [])
|
||||
}));
|
||||
onChange(result);
|
||||
}, [selections, sheetColumns, onChange]);
|
||||
|
||||
const toggleColumn = (sheetName: string, colIdx: number) => {
|
||||
setSelections(prev => {
|
||||
const set = new Set(prev[sheetName] ?? []);
|
||||
if (set.has(colIdx)) set.delete(colIdx);
|
||||
else set.add(colIdx);
|
||||
return { ...prev, [sheetName]: set };
|
||||
});
|
||||
};
|
||||
|
||||
const selectAll = (sheetName: string, cols: string[]) => {
|
||||
setSelections(prev => ({
|
||||
...prev,
|
||||
[sheetName]: new Set(cols.map((_, i) => i))
|
||||
}));
|
||||
};
|
||||
|
||||
const selectNone = (sheetName: string) => {
|
||||
setSelections(prev => ({ ...prev, [sheetName]: new Set() }));
|
||||
};
|
||||
|
||||
return (
|
||||
<Box w="full" borderWidth={1} borderRadius="md" p={3}>
|
||||
<Text fontWeight="bold" mb={2} fontSize="sm">Select Columns to Translate</Text>
|
||||
<Accordion allowMultiple defaultIndex={sheetColumns.map((_, i) => i)}>
|
||||
{sheetColumns.map(sheet => (
|
||||
<AccordionItem key={sheet.sheetName}>
|
||||
<AccordionButton>
|
||||
<Box flex="1" textAlign="left" fontSize="sm" fontWeight="semibold">
|
||||
{sheet.sheetName}
|
||||
<Badge ml={2} colorScheme="lingva">
|
||||
{selections[sheet.sheetName]?.size ?? 0}/{sheet.columns.length}
|
||||
</Badge>
|
||||
</Box>
|
||||
<AccordionIcon />
|
||||
</AccordionButton>
|
||||
<AccordionPanel pb={3}>
|
||||
<HStack mb={2} spacing={2}>
|
||||
<Button size="xs" variant="outline" colorScheme="lingva"
|
||||
onClick={() => selectAll(sheet.sheetName, sheet.columns)}>
|
||||
All
|
||||
</Button>
|
||||
<Button size="xs" variant="outline"
|
||||
onClick={() => selectNone(sheet.sheetName)}>
|
||||
None
|
||||
</Button>
|
||||
</HStack>
|
||||
<VStack align="start" spacing={1} maxH="150px" overflowY="auto">
|
||||
{sheet.columns.map((col, idx) => (
|
||||
<Checkbox
|
||||
key={idx}
|
||||
size="sm"
|
||||
isChecked={selections[sheet.sheetName]?.has(idx) ?? false}
|
||||
onChange={() => toggleColumn(sheet.sheetName, idx)}
|
||||
colorScheme="lingva"
|
||||
>
|
||||
<Text fontSize="xs">{col}</Text>
|
||||
</Checkbox>
|
||||
))}
|
||||
</VStack>
|
||||
</AccordionPanel>
|
||||
</AccordionItem>
|
||||
))}
|
||||
</Accordion>
|
||||
</Box>
|
||||
);
|
||||
};
|
||||
|
||||
export default ColumnSelector;
|
||||
261
components/DocumentTranslator.tsx
Normal file
261
components/DocumentTranslator.tsx
Normal file
@@ -0,0 +1,261 @@
|
||||
import { FC, useState, useCallback, useRef } from "react";
|
||||
import {
|
||||
Box, Button, VStack, HStack, Text, Input, Alert, AlertIcon,
|
||||
Progress, Select, Badge, Icon, useColorModeValue
|
||||
} from "@chakra-ui/react";
|
||||
import { FiUpload, FiDownload, FiFile } from "react-icons/fi";
|
||||
import { languageList, LangCode } from "lingva-scraper";
|
||||
import dynamic from "next/dynamic";
|
||||
import type { ColumnSelection, SheetColumnInfo } from "./ColumnSelector";
|
||||
|
||||
const ColumnSelector = dynamic(() => import("./ColumnSelector"), { ssr: false });
|
||||
|
||||
const SUPPORTED_TYPES = [".pdf", ".docx", ".xlsx", ".xls", ".csv"];
|
||||
const TABULAR_TYPES = [".xlsx", ".xls", ".csv"];
|
||||
|
||||
type Stage = "idle" | "columns" | "translating" | "done" | "error";
|
||||
|
||||
const DocumentTranslator: FC = () => {
|
||||
const [file, setFile] = useState<File | null>(null);
|
||||
const [target, setTarget] = useState<string>("en");
|
||||
const [stage, setStage] = useState<Stage>("idle");
|
||||
const [progress, setProgress] = useState(0);
|
||||
const [errorMsg, setErrorMsg] = useState("");
|
||||
const [sheetColumns, setSheetColumns] = useState<SheetColumnInfo[]>([]);
|
||||
const [columnSelections, setColumnSelections] = useState<ColumnSelection[]>([]);
|
||||
const [downloadUrl, setDownloadUrl] = useState<string | null>(null);
|
||||
const [downloadName, setDownloadName] = useState("");
|
||||
const fileInputRef = useRef<HTMLInputElement>(null);
|
||||
const borderColor = useColorModeValue("gray.200", "gray.600");
|
||||
const dropBg = useColorModeValue("gray.50", "gray.700");
|
||||
|
||||
const isTabular = file
|
||||
? TABULAR_TYPES.some(e => file.name.toLowerCase().endsWith(e))
|
||||
: false;
|
||||
|
||||
const handleFile = useCallback(async (f: File) => {
|
||||
setFile(f);
|
||||
setStage("idle");
|
||||
setErrorMsg("");
|
||||
setDownloadUrl(null);
|
||||
setSheetColumns([]);
|
||||
|
||||
const ext = "." + f.name.split(".").pop()?.toLowerCase();
|
||||
if (!SUPPORTED_TYPES.includes(ext)) {
|
||||
setErrorMsg(`Unsupported file type. Supported: ${SUPPORTED_TYPES.join(", ")}`);
|
||||
setStage("error");
|
||||
return;
|
||||
}
|
||||
|
||||
if (TABULAR_TYPES.includes(ext)) {
|
||||
// Fetch column info
|
||||
const fd = new FormData();
|
||||
fd.append("file", f);
|
||||
fd.append("action", "getColumns");
|
||||
try {
|
||||
const res = await fetch("/api/translate/document", { method: "POST", body: fd });
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
setSheetColumns(data.columns ?? []);
|
||||
setStage("columns");
|
||||
} else {
|
||||
const e = await res.json();
|
||||
setErrorMsg(e.error ?? "Failed to read columns");
|
||||
setStage("error");
|
||||
}
|
||||
} catch {
|
||||
setErrorMsg("Network error reading file columns");
|
||||
setStage("error");
|
||||
}
|
||||
}
|
||||
}, []);
|
||||
|
||||
const onDrop = useCallback((e: React.DragEvent) => {
|
||||
e.preventDefault();
|
||||
const f = e.dataTransfer.files[0];
|
||||
if (f) handleFile(f);
|
||||
}, [handleFile]);
|
||||
|
||||
const translate = useCallback(async () => {
|
||||
if (!file) return;
|
||||
setStage("translating");
|
||||
setProgress(10);
|
||||
setErrorMsg("");
|
||||
|
||||
const fd = new FormData();
|
||||
fd.append("file", file);
|
||||
fd.append("targetLanguage", target);
|
||||
fd.append("action", "translate");
|
||||
if (columnSelections.length > 0) {
|
||||
fd.append("columnSelections", JSON.stringify(columnSelections));
|
||||
}
|
||||
|
||||
try {
|
||||
setProgress(30);
|
||||
const res = await fetch("/api/translate/document", { method: "POST", body: fd });
|
||||
setProgress(90);
|
||||
|
||||
if (!res.ok) {
|
||||
const e = await res.json().catch(() => ({ error: "Translation failed" }));
|
||||
setErrorMsg(e.error ?? "Translation failed");
|
||||
setStage("error");
|
||||
return;
|
||||
}
|
||||
|
||||
const blob = await res.blob();
|
||||
const ext = "." + file.name.split(".").pop()!;
|
||||
const outName = file.name.replace(ext, `_${target}${ext}`);
|
||||
const url = URL.createObjectURL(blob);
|
||||
setDownloadUrl(url);
|
||||
setDownloadName(outName);
|
||||
setProgress(100);
|
||||
setStage("done");
|
||||
} catch (err) {
|
||||
setErrorMsg("Network error during translation");
|
||||
setStage("error");
|
||||
}
|
||||
}, [file, target, columnSelections]);
|
||||
|
||||
const reset = () => {
|
||||
setFile(null);
|
||||
setStage("idle");
|
||||
setErrorMsg("");
|
||||
setDownloadUrl(null);
|
||||
setSheetColumns([]);
|
||||
if (fileInputRef.current) fileInputRef.current.value = "";
|
||||
};
|
||||
|
||||
const targetLangs = Object.entries(languageList.target) as [LangCode<"target">, string][];
|
||||
|
||||
return (
|
||||
<VStack w="full" spacing={4} px={[8, null, 24, 40]}>
|
||||
{/* Drop Zone */}
|
||||
<Box
|
||||
w="full"
|
||||
border="2px dashed"
|
||||
borderColor={file ? "lingva.400" : borderColor}
|
||||
borderRadius="xl"
|
||||
p={8}
|
||||
bg={dropBg}
|
||||
textAlign="center"
|
||||
cursor="pointer"
|
||||
onDrop={onDrop}
|
||||
onDragOver={e => e.preventDefault()}
|
||||
onClick={() => fileInputRef.current?.click()}
|
||||
_hover={{ borderColor: "lingva.400" }}
|
||||
transition="border-color 0.2s"
|
||||
>
|
||||
<Input
|
||||
ref={fileInputRef}
|
||||
type="file"
|
||||
display="none"
|
||||
accept=".pdf,.docx,.xlsx,.xls,.csv"
|
||||
onChange={e => { const f = e.target.files?.[0]; if (f) handleFile(f); }}
|
||||
/>
|
||||
{file ? (
|
||||
<VStack spacing={1}>
|
||||
<Icon as={FiFile} boxSize={8} color="lingva.400" />
|
||||
<Text fontWeight="semibold">{file.name}</Text>
|
||||
<Text fontSize="sm" color="gray.500">
|
||||
{(file.size / 1024 / 1024).toFixed(2)} MB
|
||||
</Text>
|
||||
</VStack>
|
||||
) : (
|
||||
<VStack spacing={2}>
|
||||
<Icon as={FiUpload} boxSize={10} color="gray.400" />
|
||||
<Text fontWeight="semibold">Drop file here or click to upload</Text>
|
||||
<Text fontSize="sm" color="gray.500">
|
||||
Supported: PDF, Word (.docx), Excel (.xlsx, .xls), CSV
|
||||
</Text>
|
||||
</VStack>
|
||||
)}
|
||||
</Box>
|
||||
|
||||
{/* Target language + controls */}
|
||||
{file && stage !== "error" && (
|
||||
<HStack w="full" spacing={3}>
|
||||
<Box flex={1}>
|
||||
<Select
|
||||
value={target}
|
||||
onChange={e => setTarget(e.target.value)}
|
||||
size="md"
|
||||
aria-label="Target language"
|
||||
>
|
||||
{targetLangs.map(([code, name]) => (
|
||||
<option key={code} value={code}>{name}</option>
|
||||
))}
|
||||
</Select>
|
||||
</Box>
|
||||
<Button
|
||||
colorScheme="lingva"
|
||||
leftIcon={<Icon as={FiFile} />}
|
||||
onClick={translate}
|
||||
isLoading={stage === "translating"}
|
||||
loadingText="Translating…"
|
||||
isDisabled={stage === "translating" || (isTabular && stage !== "columns" && stage !== "idle")}
|
||||
>
|
||||
Translate
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" onClick={reset}>Reset</Button>
|
||||
</HStack>
|
||||
)}
|
||||
|
||||
{/* Column selector for Excel/CSV */}
|
||||
{stage === "columns" && sheetColumns.length > 0 && (
|
||||
<ColumnSelector
|
||||
sheetColumns={sheetColumns}
|
||||
onChange={setColumnSelections}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Progress */}
|
||||
{stage === "translating" && (
|
||||
<Box w="full">
|
||||
<Progress value={progress} colorScheme="lingva" borderRadius="full" hasStripe isAnimated />
|
||||
<Text fontSize="sm" textAlign="center" mt={1} color="gray.500">
|
||||
Translating… this may take a moment for large documents.
|
||||
</Text>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{/* Download */}
|
||||
{stage === "done" && downloadUrl && (
|
||||
<Alert status="success" borderRadius="md">
|
||||
<AlertIcon />
|
||||
<HStack justify="space-between" w="full">
|
||||
<Text fontSize="sm">Translation complete!</Text>
|
||||
<Button
|
||||
as="a"
|
||||
href={downloadUrl}
|
||||
download={downloadName}
|
||||
size="sm"
|
||||
colorScheme="lingva"
|
||||
leftIcon={<Icon as={FiDownload} />}
|
||||
>
|
||||
Download
|
||||
</Button>
|
||||
</HStack>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{/* Error */}
|
||||
{stage === "error" && errorMsg && (
|
||||
<Alert status="error" borderRadius="md">
|
||||
<AlertIcon />
|
||||
<VStack align="start" spacing={1}>
|
||||
<Text fontSize="sm">{errorMsg}</Text>
|
||||
<Button size="xs" variant="link" colorScheme="red" onClick={reset}>Try again</Button>
|
||||
</VStack>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
<Text fontSize="xs" color="gray.400" textAlign="center">
|
||||
Document translation requires Replicate AI to be enabled in the admin settings.
|
||||
<br />
|
||||
PDF formatting is best-effort; Excel and Word formatting is fully preserved.
|
||||
</Text>
|
||||
</VStack>
|
||||
);
|
||||
};
|
||||
|
||||
export default DocumentTranslator;
|
||||
@@ -3,6 +3,7 @@ import Head from "next/head";
|
||||
import NextLink from "next/link";
|
||||
import { Flex, HStack, IconButton, Link, useColorModeValue } from "@chakra-ui/react";
|
||||
import { FaGithub } from "react-icons/fa";
|
||||
import { FiSettings } from "react-icons/fi";
|
||||
import Image from "next/image";
|
||||
import { ColorModeToggler } from ".";
|
||||
|
||||
@@ -40,6 +41,15 @@ const Header: FC<Props> = (props) => (
|
||||
<ColorModeToggler
|
||||
variant={useColorModeValue("outline", "solid")}
|
||||
/>
|
||||
<NextLink href="/admin" passHref={true}>
|
||||
<IconButton
|
||||
as={Link}
|
||||
aria-label="Admin settings"
|
||||
icon={<FiSettings />}
|
||||
colorScheme="lingva"
|
||||
variant={useColorModeValue("outline", "solid")}
|
||||
/>
|
||||
</NextLink>
|
||||
<IconButton
|
||||
as={Link}
|
||||
href="https://github.com/thedaviddelta/lingva-translate"
|
||||
|
||||
@@ -7,3 +7,5 @@ export { default as ColorModeToggler } from "./ColorModeToggler";
|
||||
export { default as LangSelect } from "./LangSelect";
|
||||
export { default as TranslationArea } from "./TranslationArea";
|
||||
export { default as AutoTranslateButton } from "./AutoTranslateButton";
|
||||
export { default as DocumentTranslator } from "./DocumentTranslator";
|
||||
export { default as ColumnSelector } from "./ColumnSelector";
|
||||
|
||||
0
data/.gitkeep
Normal file
0
data/.gitkeep
Normal file
@@ -5,6 +5,20 @@ module.exports = withPWA({
|
||||
pwa: {
|
||||
dest: "public"
|
||||
},
|
||||
// Exclude server-only packages from client bundles
|
||||
webpack: (config, { isServer }) => {
|
||||
if (!isServer) {
|
||||
config.resolve.fallback = {
|
||||
...config.resolve.fallback,
|
||||
fs: false,
|
||||
path: false,
|
||||
crypto: false,
|
||||
stream: false,
|
||||
zlib: false
|
||||
};
|
||||
}
|
||||
return config;
|
||||
},
|
||||
async headers() {
|
||||
return [
|
||||
{
|
||||
|
||||
16714
package-lock.json
generated
Normal file
16714
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
@@ -18,18 +18,25 @@
|
||||
"@emotion/react": "^11",
|
||||
"@emotion/styled": "^11",
|
||||
"apollo-server-micro": "^2.25.2",
|
||||
"formidable": "^3.5.1",
|
||||
"framer-motion": "^6",
|
||||
"graphql": "^15.8.0",
|
||||
"jose": "^4.14.4",
|
||||
"jszip": "^3.10.1",
|
||||
"lingva-scraper": "1.1.0",
|
||||
"next": "12.1.6",
|
||||
"next-pwa": "^5.4.4",
|
||||
"nextjs-cors": "^2.1.0",
|
||||
"pdf-lib": "^1.17.1",
|
||||
"pdf-parse": "^1.1.1",
|
||||
"react": "18.2.0",
|
||||
"react-dom": "18.2.0",
|
||||
"react-hotkeys-hook": "^3.4.6",
|
||||
"react-icons": "^4.4.0"
|
||||
"react-icons": "^4.4.0",
|
||||
"xlsx": "^0.18.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/formidable": "^3.4.4",
|
||||
"@testing-library/cypress": "^8.0.3",
|
||||
"@testing-library/jest-dom": "^5.16.4",
|
||||
"@testing-library/react": "^13.3.0",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useCallback, useEffect, useReducer } from "react";
|
||||
import { useCallback, useEffect, useReducer, useState } from "react";
|
||||
import { GetStaticPaths, GetStaticProps, NextPage } from "next";
|
||||
import { useRouter } from "next/router";
|
||||
import dynamic from "next/dynamic";
|
||||
@@ -13,9 +13,10 @@ import {
|
||||
TranslationInfo,
|
||||
LangCode
|
||||
} from "lingva-scraper";
|
||||
import { HStack, IconButton, Stack, VStack } from "@chakra-ui/react";
|
||||
import { HStack, IconButton, Stack, VStack, Tabs, TabList, Tab, TabPanels, TabPanel } from "@chakra-ui/react";
|
||||
import { FaExchangeAlt } from "react-icons/fa";
|
||||
import { HiTranslate } from "react-icons/hi";
|
||||
import { FiFileText } from "react-icons/fi";
|
||||
import { useHotkeys } from "react-hotkeys-hook";
|
||||
import { CustomHead, LangSelect, TranslationArea } from "@components";
|
||||
import { useToastOnLoad } from "@hooks";
|
||||
@@ -24,6 +25,7 @@ import langReducer, { Actions, initialState, State } from "@utils/reducer";
|
||||
import { localGetItem, localSetItem } from "@utils/storage";
|
||||
|
||||
const AutoTranslateButton = dynamic(() => import("@components/AutoTranslateButton"), { ssr: false });
|
||||
const DocumentTranslator = dynamic(() => import("@components/DocumentTranslator"), { ssr: false });
|
||||
|
||||
export enum ResponseType {
|
||||
SUCCESS,
|
||||
@@ -63,6 +65,17 @@ const Page: NextPage<Props> = (props) => {
|
||||
] = useReducer(langReducer, initialState);
|
||||
|
||||
const router = useRouter();
|
||||
const [replicateEnabled, setReplicateEnabled] = useState(false);
|
||||
const [replicateTranslation, setReplicateTranslation] = useState("");
|
||||
const [replicateLoading, setReplicateLoading] = useState(false);
|
||||
|
||||
// Check if Replicate is enabled
|
||||
useEffect(() => {
|
||||
fetch("/api/admin/settings")
|
||||
.then(r => r.ok ? r.json() : null)
|
||||
.then(d => { if (d?.replicateEnabled) setReplicateEnabled(true); })
|
||||
.catch(() => {});
|
||||
}, []);
|
||||
|
||||
const setField = useCallback(<T extends keyof State,>(key: T, value: State[T]) => (
|
||||
dispatch({ type: Actions.SET_FIELD, payload: { key, value }})
|
||||
@@ -101,6 +114,30 @@ const Page: NextPage<Props> = (props) => {
|
||||
router.push(`/${source}/${target}/${encodeURIComponent(customQuery)}`);
|
||||
}, [isLoading, source, target, props, router, setField]);
|
||||
|
||||
// Replicate-powered translation
|
||||
const translateWithReplicate = useCallback(async (text: string) => {
|
||||
if (!text.trim()) return;
|
||||
setReplicateLoading(true);
|
||||
setReplicateTranslation("");
|
||||
try {
|
||||
const res = await fetch("/api/translate/replicate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ text, targetLanguage: target })
|
||||
});
|
||||
const data = await res.json();
|
||||
if (res.ok) {
|
||||
setReplicateTranslation(data.translation ?? "");
|
||||
} else {
|
||||
setReplicateTranslation(`[Error: ${data.error}]`);
|
||||
}
|
||||
} catch {
|
||||
setReplicateTranslation("[Error: Network failure]");
|
||||
} finally {
|
||||
setReplicateLoading(false);
|
||||
}
|
||||
}, [target]);
|
||||
|
||||
useEffect(() => {
|
||||
if (router.isFallback)
|
||||
return;
|
||||
@@ -183,78 +220,107 @@ const Page: NextPage<Props> = (props) => {
|
||||
: replaceExceptedCode(LanguageType.TARGET, source);
|
||||
const transLang = replaceExceptedCode(LanguageType.SOURCE, target);
|
||||
|
||||
const LangControls = (
|
||||
<HStack px={[1, null, 3, 4]} w="full">
|
||||
<LangSelect
|
||||
id="source"
|
||||
aria-label="Source language"
|
||||
value={source}
|
||||
detectedSource={detectedSource}
|
||||
onChange={e => setLanguage(LanguageType.SOURCE, e.target.value)}
|
||||
langs={languageList.source}
|
||||
/>
|
||||
<IconButton
|
||||
aria-label="Switch languages"
|
||||
icon={<FaExchangeAlt />}
|
||||
colorScheme="lingva"
|
||||
variant="ghost"
|
||||
onClick={() => switchLanguages(detectedSource)}
|
||||
isDisabled={!canSwitch}
|
||||
/>
|
||||
<LangSelect
|
||||
id="target"
|
||||
aria-label="Target language"
|
||||
value={target}
|
||||
onChange={e => setLanguage(LanguageType.TARGET, e.target.value)}
|
||||
langs={languageList.target}
|
||||
/>
|
||||
</HStack>
|
||||
);
|
||||
|
||||
return (
|
||||
<>
|
||||
<CustomHead home={props.type === ResponseType.HOME} />
|
||||
|
||||
<VStack px={[8, null, 24, 40]} w="full">
|
||||
<HStack px={[1, null, 3, 4]} w="full">
|
||||
<LangSelect
|
||||
id="source"
|
||||
aria-label="Source language"
|
||||
value={source}
|
||||
detectedSource={detectedSource}
|
||||
onChange={e => setLanguage(LanguageType.SOURCE, e.target.value)}
|
||||
langs={languageList.source}
|
||||
/>
|
||||
<IconButton
|
||||
aria-label="Switch languages"
|
||||
icon={<FaExchangeAlt />}
|
||||
colorScheme="lingva"
|
||||
variant="ghost"
|
||||
onClick={() => switchLanguages(detectedSource)}
|
||||
isDisabled={!canSwitch}
|
||||
/>
|
||||
<LangSelect
|
||||
id="target"
|
||||
aria-label="Target language"
|
||||
value={target}
|
||||
onChange={e => setLanguage(LanguageType.TARGET, e.target.value)}
|
||||
langs={languageList.target}
|
||||
/>
|
||||
</HStack>
|
||||
<Stack direction={["column", null, "row"]} w="full">
|
||||
<TranslationArea
|
||||
id="query"
|
||||
aria-label="Translation query"
|
||||
placeholder="Text"
|
||||
value={query}
|
||||
onChange={e => isLoading || setField("query", e.target.value)}
|
||||
onSubmit={() => changeRoute(query)}
|
||||
lang={queryLang}
|
||||
audio={audio.query}
|
||||
pronunciation={pronunciation.query}
|
||||
/>
|
||||
<Stack direction={["row", null, "column"]} justify="center" spacing={3} px={[2, null, "initial"]}>
|
||||
<IconButton
|
||||
aria-label="Translate"
|
||||
icon={<HiTranslate />}
|
||||
colorScheme="lingva"
|
||||
variant="outline"
|
||||
onClick={() => changeRoute(query)}
|
||||
isDisabled={isLoading}
|
||||
w={["full", null, "auto"]}
|
||||
/>
|
||||
<AutoTranslateButton
|
||||
isDisabled={isLoading}
|
||||
// runs on effect update
|
||||
onAuto={useCallback(() => changeRoute(delayedQuery), [delayedQuery, changeRoute])}
|
||||
w={["full", null, "auto"]}
|
||||
/>
|
||||
</Stack>
|
||||
<TranslationArea
|
||||
id="translation"
|
||||
aria-label="Translation result"
|
||||
placeholder="Translation"
|
||||
value={translation ?? ""}
|
||||
readOnly={true}
|
||||
lang={transLang}
|
||||
audio={audio.translation}
|
||||
canCopy={true}
|
||||
isLoading={isLoading}
|
||||
pronunciation={pronunciation.translation}
|
||||
/>
|
||||
</Stack>
|
||||
<Tabs w="full" colorScheme="lingva" variant="soft-rounded" size="sm">
|
||||
<TabList mb={4} justifyContent="center">
|
||||
<Tab><HiTranslate style={{ marginRight: 6 }} />Text</Tab>
|
||||
<Tab><FiFileText style={{ marginRight: 6 }} />Document</Tab>
|
||||
</TabList>
|
||||
|
||||
<TabPanels>
|
||||
{/* Text Translation Tab */}
|
||||
<TabPanel p={0}>
|
||||
<VStack w="full" spacing={4}>
|
||||
{LangControls}
|
||||
<Stack direction={["column", null, "row"]} w="full">
|
||||
<TranslationArea
|
||||
id="query"
|
||||
aria-label="Translation query"
|
||||
placeholder="Text"
|
||||
value={query}
|
||||
onChange={e => isLoading || setField("query", e.target.value)}
|
||||
onSubmit={() => replicateEnabled ? translateWithReplicate(query) : changeRoute(query)}
|
||||
lang={queryLang}
|
||||
audio={audio.query}
|
||||
pronunciation={pronunciation.query}
|
||||
/>
|
||||
<Stack direction={["row", null, "column"]} justify="center" spacing={3} px={[2, null, "initial"]}>
|
||||
<IconButton
|
||||
aria-label="Translate"
|
||||
icon={<HiTranslate />}
|
||||
colorScheme="lingva"
|
||||
variant="outline"
|
||||
onClick={() => replicateEnabled ? translateWithReplicate(query) : changeRoute(query)}
|
||||
isDisabled={isLoading || replicateLoading}
|
||||
isLoading={replicateLoading}
|
||||
w={["full", null, "auto"]}
|
||||
/>
|
||||
<AutoTranslateButton
|
||||
isDisabled={isLoading || replicateLoading}
|
||||
onAuto={useCallback(() => {
|
||||
if (replicateEnabled) {
|
||||
translateWithReplicate(delayedQuery);
|
||||
} else {
|
||||
changeRoute(delayedQuery);
|
||||
}
|
||||
}, [delayedQuery, replicateEnabled, translateWithReplicate, changeRoute])}
|
||||
w={["full", null, "auto"]}
|
||||
/>
|
||||
</Stack>
|
||||
<TranslationArea
|
||||
id="translation"
|
||||
aria-label="Translation result"
|
||||
placeholder="Translation"
|
||||
value={replicateEnabled ? replicateTranslation : (translation ?? "")}
|
||||
readOnly={true}
|
||||
lang={transLang}
|
||||
audio={audio.translation}
|
||||
canCopy={true}
|
||||
isLoading={replicateEnabled ? replicateLoading : isLoading}
|
||||
pronunciation={replicateEnabled ? undefined : pronunciation.translation}
|
||||
/>
|
||||
</Stack>
|
||||
</VStack>
|
||||
</TabPanel>
|
||||
|
||||
{/* Document Translation Tab */}
|
||||
<TabPanel p={0}>
|
||||
<DocumentTranslator />
|
||||
</TabPanel>
|
||||
</TabPanels>
|
||||
</Tabs>
|
||||
</VStack>
|
||||
</>
|
||||
);
|
||||
|
||||
372
pages/admin/index.tsx
Normal file
372
pages/admin/index.tsx
Normal file
@@ -0,0 +1,372 @@
|
||||
import { useState, useEffect, FormEvent } from "react";
|
||||
import { NextPage } from "next";
|
||||
import NextLink from "next/link";
|
||||
import {
|
||||
Box, VStack, HStack, Heading, Text, Input, Button, Switch,
|
||||
FormControl, FormLabel, FormHelperText, Divider, Alert, AlertIcon,
|
||||
InputGroup, InputRightElement, IconButton, Badge, Link,
|
||||
useColorModeValue, Spinner, Textarea
|
||||
} from "@chakra-ui/react";
|
||||
import { FiEye, FiEyeOff, FiSave, FiLogOut, FiArrowLeft } from "react-icons/fi";
|
||||
import { CustomHead } from "@components";
|
||||
|
||||
type Settings = {
|
||||
replicateApiToken: string;
|
||||
jigsawApiKey: string;
|
||||
modelVersion: string;
|
||||
replicateEnabled: boolean;
|
||||
};
|
||||
|
||||
const DEFAULT_MODEL = "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89";
|
||||
|
||||
const AdminPage: NextPage = () => {
|
||||
const [authed, setAuthed] = useState<boolean | null>(null);
|
||||
const [password, setPassword] = useState("");
|
||||
const [loginError, setLoginError] = useState("");
|
||||
const [loginLoading, setLoginLoading] = useState(false);
|
||||
|
||||
const [settings, setSettings] = useState<Settings>({
|
||||
replicateApiToken: "",
|
||||
jigsawApiKey: "",
|
||||
modelVersion: DEFAULT_MODEL,
|
||||
replicateEnabled: false
|
||||
});
|
||||
const [newPassword, setNewPassword] = useState("");
|
||||
const [saveMsg, setSaveMsg] = useState<{ type: "success" | "error"; text: string } | null>(null);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [showToken, setShowToken] = useState(false);
|
||||
const [showJigsawKey, setShowJigsawKey] = useState(false);
|
||||
const [testResult, setTestResult] = useState<string | null>(null);
|
||||
const [testLoading, setTestLoading] = useState(false);
|
||||
|
||||
const cardBg = useColorModeValue("white", "gray.800");
|
||||
const borderCol = useColorModeValue("gray.200", "gray.600");
|
||||
|
||||
// Check auth on load
|
||||
useEffect(() => {
|
||||
fetch("/api/admin/auth")
|
||||
.then(r => r.json())
|
||||
.then(d => setAuthed(d.authenticated))
|
||||
.catch(() => setAuthed(false));
|
||||
}, []);
|
||||
|
||||
// Load settings when authed
|
||||
useEffect(() => {
|
||||
if (!authed) return;
|
||||
fetch("/api/admin/settings")
|
||||
.then(r => r.json())
|
||||
.then(d => setSettings(s => ({ ...s, ...d })))
|
||||
.catch(() => {});
|
||||
}, [authed]);
|
||||
|
||||
const login = async (e: FormEvent) => {
|
||||
e.preventDefault();
|
||||
setLoginLoading(true);
|
||||
setLoginError("");
|
||||
try {
|
||||
const res = await fetch("/api/admin/auth", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ password })
|
||||
});
|
||||
if (res.ok) {
|
||||
setAuthed(true);
|
||||
} else {
|
||||
const d = await res.json();
|
||||
setLoginError(d.error ?? "Login failed");
|
||||
}
|
||||
} catch {
|
||||
setLoginError("Network error");
|
||||
} finally {
|
||||
setLoginLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
const logout = async () => {
|
||||
await fetch("/api/admin/auth", { method: "DELETE" });
|
||||
setAuthed(false);
|
||||
setPassword("");
|
||||
};
|
||||
|
||||
const save = async (e: FormEvent) => {
|
||||
e.preventDefault();
|
||||
setSaving(true);
|
||||
setSaveMsg(null);
|
||||
try {
|
||||
const body: Record<string, unknown> = { ...settings };
|
||||
if (newPassword.length >= 6) body.newPassword = newPassword;
|
||||
const res = await fetch("/api/admin/settings", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body)
|
||||
});
|
||||
const d = await res.json();
|
||||
if (res.ok) {
|
||||
setSettings(s => ({ ...s, ...d }));
|
||||
setSaveMsg({ type: "success", text: "Settings saved successfully" });
|
||||
setNewPassword("");
|
||||
} else {
|
||||
setSaveMsg({ type: "error", text: d.error ?? "Save failed" });
|
||||
}
|
||||
} catch {
|
||||
setSaveMsg({ type: "error", text: "Network error" });
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const testTranslation = async () => {
|
||||
setTestLoading(true);
|
||||
setTestResult(null);
|
||||
try {
|
||||
const res = await fetch("/api/translate/replicate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ text: "Hello, world!", targetLanguage: "es" })
|
||||
});
|
||||
const d = await res.json();
|
||||
if (res.ok) {
|
||||
setTestResult(`✓ Success: "${d.translation}"`);
|
||||
} else {
|
||||
setTestResult(`✗ Error: ${d.error}`);
|
||||
}
|
||||
} catch {
|
||||
setTestResult("✗ Network error");
|
||||
} finally {
|
||||
setTestLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
// Loading state
|
||||
if (authed === null) {
|
||||
return (
|
||||
<Box w="full" display="flex" justifyContent="center" pt={20}>
|
||||
<Spinner size="xl" color="lingva.400" />
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
// Login form
|
||||
if (!authed) {
|
||||
return (
|
||||
<>
|
||||
<CustomHead home={false} />
|
||||
<Box
|
||||
w="full"
|
||||
maxW="400px"
|
||||
mx="auto"
|
||||
mt={10}
|
||||
p={8}
|
||||
bg={cardBg}
|
||||
borderWidth={1}
|
||||
borderColor={borderCol}
|
||||
borderRadius="xl"
|
||||
boxShadow="md"
|
||||
>
|
||||
<VStack spacing={6} as="form" onSubmit={login}>
|
||||
<Heading size="md">Admin Login</Heading>
|
||||
{loginError && (
|
||||
<Alert status="error" borderRadius="md">
|
||||
<AlertIcon />{loginError}
|
||||
</Alert>
|
||||
)}
|
||||
<FormControl isRequired>
|
||||
<FormLabel>Password</FormLabel>
|
||||
<Input
|
||||
type="password"
|
||||
value={password}
|
||||
onChange={e => setPassword(e.target.value)}
|
||||
autoFocus
|
||||
placeholder="Admin password"
|
||||
/>
|
||||
<FormHelperText>Default: admin (set ADMIN_PASSWORD env var)</FormHelperText>
|
||||
</FormControl>
|
||||
<Button
|
||||
type="submit"
|
||||
colorScheme="lingva"
|
||||
w="full"
|
||||
isLoading={loginLoading}
|
||||
>
|
||||
Login
|
||||
</Button>
|
||||
<NextLink href="/" passHref>
|
||||
<Link fontSize="sm">← Back to translator</Link>
|
||||
</NextLink>
|
||||
</VStack>
|
||||
</Box>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
// Admin panel
|
||||
return (
|
||||
<>
|
||||
<CustomHead home={false} />
|
||||
<Box w="full" maxW="700px" mx="auto" px={4} pb={10}>
|
||||
<HStack justify="space-between" mb={6}>
|
||||
<NextLink href="/" passHref>
|
||||
<Button as={Link} leftIcon={<FiArrowLeft />} variant="ghost" size="sm">
|
||||
Back
|
||||
</Button>
|
||||
</NextLink>
|
||||
<Heading size="md">Admin Settings</Heading>
|
||||
<Button leftIcon={<FiLogOut />} variant="ghost" size="sm" onClick={logout}>
|
||||
Logout
|
||||
</Button>
|
||||
</HStack>
|
||||
|
||||
<VStack spacing={6} as="form" onSubmit={save}>
|
||||
{/* Replicate Section */}
|
||||
<Box w="full" p={6} bg={cardBg} borderWidth={1} borderColor={borderCol} borderRadius="xl">
|
||||
<HStack mb={4}>
|
||||
<Heading size="sm">Replicate AI Translation</Heading>
|
||||
<Badge colorScheme={settings.replicateEnabled ? "green" : "gray"}>
|
||||
{settings.replicateEnabled ? "Enabled" : "Disabled"}
|
||||
</Badge>
|
||||
</HStack>
|
||||
|
||||
<VStack spacing={4} align="stretch">
|
||||
<FormControl display="flex" alignItems="center">
|
||||
<FormLabel mb={0}>Enable Replicate Translation</FormLabel>
|
||||
<Switch
|
||||
colorScheme="lingva"
|
||||
isChecked={settings.replicateEnabled}
|
||||
onChange={e => setSettings(s => ({ ...s, replicateEnabled: e.target.checked }))}
|
||||
/>
|
||||
</FormControl>
|
||||
|
||||
<FormControl>
|
||||
<FormLabel>Replicate API Token</FormLabel>
|
||||
<InputGroup>
|
||||
<Input
|
||||
type={showToken ? "text" : "password"}
|
||||
value={settings.replicateApiToken}
|
||||
onChange={e => setSettings(s => ({ ...s, replicateApiToken: e.target.value }))}
|
||||
placeholder="r8_..."
|
||||
fontFamily="mono"
|
||||
fontSize="sm"
|
||||
/>
|
||||
<InputRightElement>
|
||||
<IconButton
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
aria-label="Toggle visibility"
|
||||
icon={showToken ? <FiEyeOff /> : <FiEye />}
|
||||
onClick={() => setShowToken(v => !v)}
|
||||
/>
|
||||
</InputRightElement>
|
||||
</InputGroup>
|
||||
<FormHelperText>
|
||||
Get your token at{" "}
|
||||
<Link href="https://replicate.com/account/api-tokens" isExternal color="lingva.400">
|
||||
replicate.com/account/api-tokens
|
||||
</Link>
|
||||
</FormHelperText>
|
||||
</FormControl>
|
||||
|
||||
<FormControl>
|
||||
<FormLabel>JigsawStack API Key</FormLabel>
|
||||
<InputGroup>
|
||||
<Input
|
||||
type={showJigsawKey ? "text" : "password"}
|
||||
value={settings.jigsawApiKey}
|
||||
onChange={e => setSettings(s => ({ ...s, jigsawApiKey: e.target.value }))}
|
||||
placeholder="sk_..."
|
||||
fontFamily="mono"
|
||||
fontSize="sm"
|
||||
/>
|
||||
<InputRightElement>
|
||||
<IconButton
|
||||
size="xs"
|
||||
variant="ghost"
|
||||
aria-label="Toggle visibility"
|
||||
icon={showJigsawKey ? <FiEyeOff /> : <FiEye />}
|
||||
onClick={() => setShowJigsawKey(v => !v)}
|
||||
/>
|
||||
</InputRightElement>
|
||||
</InputGroup>
|
||||
<FormHelperText>
|
||||
Required for the JigsawStack translation model. Get yours at{" "}
|
||||
<Link href="https://jigsawstack.com" isExternal color="lingva.400">
|
||||
jigsawstack.com
|
||||
</Link>
|
||||
</FormHelperText>
|
||||
</FormControl>
|
||||
|
||||
<FormControl>
|
||||
<FormLabel>Model Version</FormLabel>
|
||||
<Textarea
|
||||
value={settings.modelVersion}
|
||||
onChange={e => setSettings(s => ({ ...s, modelVersion: e.target.value }))}
|
||||
fontFamily="mono"
|
||||
fontSize="xs"
|
||||
rows={2}
|
||||
placeholder={DEFAULT_MODEL}
|
||||
/>
|
||||
<FormHelperText>
|
||||
Replicate model version string. Default uses the JigsawStack text-translate model.
|
||||
</FormHelperText>
|
||||
</FormControl>
|
||||
|
||||
<HStack>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
colorScheme="lingva"
|
||||
onClick={testTranslation}
|
||||
isLoading={testLoading}
|
||||
isDisabled={!settings.replicateEnabled || !settings.replicateApiToken}
|
||||
>
|
||||
Test Translation
|
||||
</Button>
|
||||
{testResult && (
|
||||
<Text
|
||||
fontSize="sm"
|
||||
color={testResult.startsWith("✓") ? "green.500" : "red.500"}
|
||||
>
|
||||
{testResult}
|
||||
</Text>
|
||||
)}
|
||||
</HStack>
|
||||
</VStack>
|
||||
</Box>
|
||||
|
||||
{/* Security Section */}
|
||||
<Box w="full" p={6} bg={cardBg} borderWidth={1} borderColor={borderCol} borderRadius="xl">
|
||||
<Heading size="sm" mb={4}>Security</Heading>
|
||||
<FormControl>
|
||||
<FormLabel>New Admin Password</FormLabel>
|
||||
<Input
|
||||
type="password"
|
||||
value={newPassword}
|
||||
onChange={e => setNewPassword(e.target.value)}
|
||||
placeholder="Leave blank to keep current"
|
||||
minLength={6}
|
||||
/>
|
||||
<FormHelperText>Minimum 6 characters. Leave blank to keep current password.</FormHelperText>
|
||||
</FormControl>
|
||||
</Box>
|
||||
|
||||
{saveMsg && (
|
||||
<Alert status={saveMsg.type} borderRadius="md">
|
||||
<AlertIcon />{saveMsg.text}
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
<Button
|
||||
type="submit"
|
||||
colorScheme="lingva"
|
||||
leftIcon={<FiSave />}
|
||||
isLoading={saving}
|
||||
size="lg"
|
||||
w="full"
|
||||
>
|
||||
Save Settings
|
||||
</Button>
|
||||
</VStack>
|
||||
</Box>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default AdminPage;
|
||||
33
pages/api/admin/auth.ts
Normal file
33
pages/api/admin/auth.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { NextApiHandler } from "next";
|
||||
import { signAdminToken, checkPassword, COOKIE_NAME, getTokenFromRequest, verifyAdminToken } from "@utils/admin-auth";
|
||||
|
||||
const handler: NextApiHandler = async (req, res) => {
|
||||
if (req.method === "POST") {
|
||||
const { password } = req.body ?? {};
|
||||
if (!password || typeof password !== "string") {
|
||||
return res.status(400).json({ error: "Password required" });
|
||||
}
|
||||
if (!checkPassword(password)) {
|
||||
return res.status(401).json({ error: "Invalid password" });
|
||||
}
|
||||
const token = await signAdminToken();
|
||||
res.setHeader("Set-Cookie", `${COOKIE_NAME}=${token}; HttpOnly; Path=/; SameSite=Lax; Max-Age=28800`);
|
||||
return res.status(200).json({ ok: true });
|
||||
}
|
||||
|
||||
if (req.method === "DELETE") {
|
||||
res.setHeader("Set-Cookie", `${COOKIE_NAME}=; HttpOnly; Path=/; Max-Age=0`);
|
||||
return res.status(200).json({ ok: true });
|
||||
}
|
||||
|
||||
if (req.method === "GET") {
|
||||
const token = getTokenFromRequest(req);
|
||||
const valid = token ? await verifyAdminToken(token) : false;
|
||||
return res.status(200).json({ authenticated: valid });
|
||||
}
|
||||
|
||||
res.setHeader("Allow", ["GET", "POST", "DELETE"]);
|
||||
return res.status(405).json({ error: "Method Not Allowed" });
|
||||
};
|
||||
|
||||
export default handler;
|
||||
42
pages/api/admin/settings.ts
Normal file
42
pages/api/admin/settings.ts
Normal file
@@ -0,0 +1,42 @@
|
||||
import { NextApiHandler } from "next";
|
||||
import { requireAdmin } from "@utils/admin-auth";
|
||||
import { readSettings, writeSettings } from "@utils/settings-store";
|
||||
|
||||
const handler: NextApiHandler = async (req, res) => {
|
||||
if (!(await requireAdmin(req, res))) return;
|
||||
|
||||
if (req.method === "GET") {
|
||||
const settings = readSettings();
|
||||
// Never expose the adminPasswordHash
|
||||
const { adminPasswordHash: _omit, ...safe } = settings;
|
||||
return res.status(200).json(safe);
|
||||
}
|
||||
|
||||
if (req.method === "POST") {
|
||||
const {
|
||||
replicateApiToken,
|
||||
jigsawApiKey,
|
||||
modelVersion,
|
||||
replicateEnabled,
|
||||
newPassword
|
||||
} = req.body ?? {};
|
||||
|
||||
const updates: Parameters<typeof writeSettings>[0] = {};
|
||||
if (replicateApiToken !== undefined) updates.replicateApiToken = replicateApiToken;
|
||||
if (jigsawApiKey !== undefined) updates.jigsawApiKey = jigsawApiKey;
|
||||
if (modelVersion !== undefined) updates.modelVersion = modelVersion;
|
||||
if (replicateEnabled !== undefined) updates.replicateEnabled = Boolean(replicateEnabled);
|
||||
if (newPassword && typeof newPassword === "string" && newPassword.length >= 6) {
|
||||
updates.adminPasswordHash = newPassword;
|
||||
}
|
||||
|
||||
const saved = writeSettings(updates);
|
||||
const { adminPasswordHash: _omit, ...safe } = saved;
|
||||
return res.status(200).json(safe);
|
||||
}
|
||||
|
||||
res.setHeader("Allow", ["GET", "POST"]);
|
||||
return res.status(405).json({ error: "Method Not Allowed" });
|
||||
};
|
||||
|
||||
export default handler;
|
||||
136
pages/api/translate/document.ts
Normal file
136
pages/api/translate/document.ts
Normal file
@@ -0,0 +1,136 @@
|
||||
import { NextApiHandler } from "next";
|
||||
import formidable, { File, Fields, Files } from "formidable";
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { translateExcel, getExcelColumns, ColumnSelection } from "@utils/document-processors/excel";
|
||||
import { translateDocx } from "@utils/document-processors/docx";
|
||||
import { translatePdf } from "@utils/document-processors/pdf";
|
||||
import { readSettings } from "@utils/settings-store";
|
||||
|
||||
export const config = {
|
||||
api: {
|
||||
bodyParser: false,
|
||||
responseLimit: "50mb"
|
||||
}
|
||||
};
|
||||
|
||||
type ParsedForm = {
|
||||
fields: Fields;
|
||||
files: Files;
|
||||
};
|
||||
|
||||
function parseForm(req: Parameters<NextApiHandler>[0]): Promise<ParsedForm> {
|
||||
const form = formidable({ maxFileSize: 50 * 1024 * 1024 }); // 50 MB
|
||||
return new Promise((resolve, reject) => {
|
||||
form.parse(req, (err, fields, files) => {
|
||||
if (err) reject(err);
|
||||
else resolve({ fields, files });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function getFileBuffer(file: File): Buffer {
|
||||
return fs.readFileSync(file.filepath);
|
||||
}
|
||||
|
||||
function getField(fields: Fields, key: string): string | undefined {
|
||||
const val = fields[key];
|
||||
return Array.isArray(val) ? val[0] : (val as string | undefined);
|
||||
}
|
||||
|
||||
const handler: NextApiHandler = async (req, res) => {
|
||||
if (req.method !== "POST") {
|
||||
res.setHeader("Allow", ["POST"]);
|
||||
return res.status(405).json({ error: "Method Not Allowed" });
|
||||
}
|
||||
|
||||
const settings = readSettings();
|
||||
if (!settings.replicateEnabled) {
|
||||
return res.status(503).json({ error: "Replicate translation is not enabled. Configure it in the admin panel." });
|
||||
}
|
||||
|
||||
let parsed: ParsedForm;
|
||||
try {
|
||||
parsed = await parseForm(req);
|
||||
} catch (err) {
|
||||
return res.status(400).json({ error: "Failed to parse upload" });
|
||||
}
|
||||
|
||||
const { fields, files } = parsed;
|
||||
const fileEntry = files["file"];
|
||||
const file = Array.isArray(fileEntry) ? fileEntry[0] : fileEntry;
|
||||
|
||||
if (!file) {
|
||||
return res.status(400).json({ error: "No file uploaded" });
|
||||
}
|
||||
|
||||
const targetLanguage = getField(fields, "targetLanguage") ?? "en";
|
||||
const sourceLanguage = getField(fields, "sourceLanguage");
|
||||
const action = getField(fields, "action") ?? "translate";
|
||||
const columnSelectionsRaw = getField(fields, "columnSelections");
|
||||
|
||||
const filename = file.originalFilename ?? file.newFilename ?? "file";
|
||||
const ext = path.extname(filename).toLowerCase();
|
||||
const buffer = getFileBuffer(file);
|
||||
|
||||
try {
|
||||
// Action: getColumns - return column info for Excel files
|
||||
if (action === "getColumns") {
|
||||
if (![".xlsx", ".xls", ".csv"].includes(ext)) {
|
||||
return res.status(400).json({ error: "Column selection only supported for Excel/CSV files" });
|
||||
}
|
||||
const columns = getExcelColumns(buffer, filename);
|
||||
return res.status(200).json({ columns });
|
||||
}
|
||||
|
||||
// Action: translate
|
||||
let outBuffer: Buffer;
|
||||
let outMime: string;
|
||||
let outFilename: string;
|
||||
|
||||
if ([".xlsx", ".xls"].includes(ext)) {
|
||||
let columnSelections: ColumnSelection[] = [];
|
||||
if (columnSelectionsRaw) {
|
||||
try {
|
||||
columnSelections = JSON.parse(columnSelectionsRaw);
|
||||
} catch { /* use empty = translate all */ }
|
||||
}
|
||||
outBuffer = await translateExcel(buffer, targetLanguage, columnSelections);
|
||||
outMime = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
|
||||
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||
} else if (ext === ".csv") {
|
||||
// Treat CSV as Excel
|
||||
let columnSelections: ColumnSelection[] = [];
|
||||
if (columnSelectionsRaw) {
|
||||
try {
|
||||
columnSelections = JSON.parse(columnSelectionsRaw);
|
||||
} catch { /* use empty = translate all */ }
|
||||
}
|
||||
outBuffer = await translateExcel(buffer, targetLanguage, columnSelections);
|
||||
outMime = "text/csv";
|
||||
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||
} else if (ext === ".docx") {
|
||||
outBuffer = await translateDocx(buffer, targetLanguage);
|
||||
outMime = "application/vnd.openxmlformats-officedocument.wordprocessingml.document";
|
||||
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||
} else if (ext === ".pdf") {
|
||||
outBuffer = await translatePdf(buffer, targetLanguage, sourceLanguage);
|
||||
outMime = "application/pdf";
|
||||
outFilename = filename.replace(ext, `_${targetLanguage}${ext}`);
|
||||
} else {
|
||||
return res.status(400).json({
|
||||
error: `Unsupported file type: ${ext}. Supported: .pdf, .docx, .xlsx, .xls, .csv`
|
||||
});
|
||||
}
|
||||
|
||||
res.setHeader("Content-Type", outMime);
|
||||
res.setHeader("Content-Disposition", `attachment; filename="${outFilename}"`);
|
||||
res.setHeader("Content-Length", outBuffer.length);
|
||||
return res.status(200).send(outBuffer);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : "Translation failed";
|
||||
return res.status(500).json({ error: msg });
|
||||
}
|
||||
};
|
||||
|
||||
export default handler;
|
||||
38
pages/api/translate/replicate.ts
Normal file
38
pages/api/translate/replicate.ts
Normal file
@@ -0,0 +1,38 @@
|
||||
import { NextApiHandler } from "next";
|
||||
import NextCors from "nextjs-cors";
|
||||
import { replicateTranslate } from "@utils/replicate-translate";
|
||||
import { readSettings } from "@utils/settings-store";
|
||||
|
||||
type Data = { translation: string } | { error: string };
|
||||
|
||||
const handler: NextApiHandler<Data> = async (req, res) => {
|
||||
await NextCors(req, res, { methods: ["POST"], origin: "*" });
|
||||
|
||||
if (req.method !== "POST") {
|
||||
res.setHeader("Allow", ["POST"]);
|
||||
return res.status(405).json({ error: "Method Not Allowed" });
|
||||
}
|
||||
|
||||
const settings = readSettings();
|
||||
if (!settings.replicateEnabled) {
|
||||
return res.status(503).json({ error: "Replicate translation is not enabled" });
|
||||
}
|
||||
|
||||
const { text, targetLanguage } = req.body ?? {};
|
||||
if (!text || typeof text !== "string") {
|
||||
return res.status(400).json({ error: "text is required" });
|
||||
}
|
||||
if (!targetLanguage || typeof targetLanguage !== "string") {
|
||||
return res.status(400).json({ error: "targetLanguage is required" });
|
||||
}
|
||||
|
||||
try {
|
||||
const translation = await replicateTranslate(text, targetLanguage);
|
||||
return res.status(200).json({ translation });
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : "Translation failed";
|
||||
return res.status(500).json({ error: msg });
|
||||
}
|
||||
};
|
||||
|
||||
export default handler;
|
||||
55
utils/admin-auth.ts
Normal file
55
utils/admin-auth.ts
Normal file
@@ -0,0 +1,55 @@
|
||||
import { SignJWT, jwtVerify } from "jose";
|
||||
import { NextApiRequest, NextApiResponse } from "next";
|
||||
import { readSettings } from "./settings-store";
|
||||
|
||||
const JWT_COOKIE = "lingva_admin";
|
||||
const JWT_EXPIRY = "8h";
|
||||
|
||||
function getSecret(): Uint8Array {
|
||||
const secret = process.env["ADMIN_JWT_SECRET"] ?? "lingva-admin-secret-change-me";
|
||||
return new TextEncoder().encode(secret);
|
||||
}
|
||||
|
||||
export async function signAdminToken(): Promise<string> {
|
||||
return new SignJWT({ role: "admin" })
|
||||
.setProtectedHeader({ alg: "HS256" })
|
||||
.setIssuedAt()
|
||||
.setExpirationTime(JWT_EXPIRY)
|
||||
.sign(getSecret());
|
||||
}
|
||||
|
||||
export async function verifyAdminToken(token: string): Promise<boolean> {
|
||||
try {
|
||||
await jwtVerify(token, getSecret());
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function getTokenFromRequest(req: NextApiRequest): string | null {
|
||||
const cookie = req.cookies[JWT_COOKIE];
|
||||
if (cookie) return cookie;
|
||||
const auth = req.headers.authorization;
|
||||
if (auth?.startsWith("Bearer ")) return auth.slice(7);
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function requireAdmin(
|
||||
req: NextApiRequest,
|
||||
res: NextApiResponse
|
||||
): Promise<boolean> {
|
||||
const token = getTokenFromRequest(req);
|
||||
if (!token || !(await verifyAdminToken(token))) {
|
||||
res.status(401).json({ error: "Unauthorized" });
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
export function checkPassword(password: string): boolean {
|
||||
const settings = readSettings();
|
||||
return password === settings.adminPasswordHash;
|
||||
}
|
||||
|
||||
export const COOKIE_NAME = JWT_COOKIE;
|
||||
106
utils/document-processors/docx.ts
Normal file
106
utils/document-processors/docx.ts
Normal file
@@ -0,0 +1,106 @@
|
||||
import JSZip from "jszip";
|
||||
import { replicateTranslateBatch } from "../replicate-translate";
|
||||
|
||||
function escapeXml(text: string): string {
|
||||
return text
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, """)
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract paragraph texts from document.xml string.
|
||||
* Returns array of {index, text} where index is the paragraph number.
|
||||
*/
|
||||
function extractParagraphs(xml: string): { index: number; text: string; start: number; end: number }[] {
|
||||
const paragraphs: { index: number; text: string; start: number; end: number }[] = [];
|
||||
const pRegex = /<w:p[ >]/g;
|
||||
const pCloseTag = "</w:p>";
|
||||
let idx = 0;
|
||||
let match: RegExpExecArray | null;
|
||||
|
||||
while ((match = pRegex.exec(xml)) !== null) {
|
||||
const start = match.index;
|
||||
const end = xml.indexOf(pCloseTag, start) + pCloseTag.length;
|
||||
if (end < pCloseTag.length) break;
|
||||
|
||||
const paraXml = xml.slice(start, end);
|
||||
|
||||
// Extract all text content within this paragraph
|
||||
const textParts: string[] = [];
|
||||
const tRegex = /<w:t[^>]*>([\s\S]*?)<\/w:t>/g;
|
||||
let tMatch: RegExpExecArray | null;
|
||||
while ((tMatch = tRegex.exec(paraXml)) !== null) {
|
||||
textParts.push(tMatch[1]);
|
||||
}
|
||||
|
||||
const text = textParts.join("").trim();
|
||||
if (text) {
|
||||
paragraphs.push({ index: idx, text, start, end });
|
||||
}
|
||||
idx++;
|
||||
}
|
||||
return paragraphs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace text within a paragraph XML while preserving formatting of first run.
|
||||
* Empties all other text runs.
|
||||
*/
|
||||
function replaceParagraphText(paraXml: string, translatedText: string): string {
|
||||
let firstDone = false;
|
||||
return paraXml.replace(/<w:t([^>]*)>([\s\S]*?)<\/w:t>/g, (_match, attrs, content) => {
|
||||
if (!firstDone && content.trim()) {
|
||||
firstDone = true;
|
||||
return `<w:t xml:space="preserve">${escapeXml(translatedText)}</w:t>`;
|
||||
}
|
||||
if (firstDone) {
|
||||
return `<w:t></w:t>`;
|
||||
}
|
||||
return _match; // preserve empty runs before the first text
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate a DOCX file buffer, preserving formatting.
|
||||
*/
|
||||
export async function translateDocx(
|
||||
buffer: Buffer,
|
||||
targetLanguage: string
|
||||
): Promise<Buffer> {
|
||||
const zip = await JSZip.loadAsync(buffer);
|
||||
const docFile = zip.file("word/document.xml");
|
||||
if (!docFile) throw new Error("Invalid DOCX: missing word/document.xml");
|
||||
|
||||
let xml = await docFile.async("string");
|
||||
const paragraphs = extractParagraphs(xml);
|
||||
|
||||
if (paragraphs.length === 0) {
|
||||
return Buffer.from(await zip.generateAsync({ type: "nodebuffer" }));
|
||||
}
|
||||
|
||||
// Translate all paragraphs
|
||||
const translations = await replicateTranslateBatch(
|
||||
paragraphs.map(p => p.text),
|
||||
targetLanguage
|
||||
);
|
||||
|
||||
// Replace paragraphs from end to start to preserve offsets
|
||||
const sorted = [...paragraphs].sort((a, b) => b.start - a.start);
|
||||
for (const para of sorted) {
|
||||
const translationIdx = paragraphs.findIndex(p => p.start === para.start);
|
||||
const translated = translations[translationIdx] ?? para.text;
|
||||
const originalPara = xml.slice(para.start, para.end);
|
||||
const translatedPara = replaceParagraphText(originalPara, translated);
|
||||
xml = xml.slice(0, para.start) + translatedPara + xml.slice(para.end);
|
||||
}
|
||||
|
||||
zip.file("word/document.xml", xml);
|
||||
const outBuffer = await zip.generateAsync({
|
||||
type: "nodebuffer",
|
||||
compression: "DEFLATE"
|
||||
});
|
||||
return Buffer.from(outBuffer);
|
||||
}
|
||||
95
utils/document-processors/excel.ts
Normal file
95
utils/document-processors/excel.ts
Normal file
@@ -0,0 +1,95 @@
|
||||
import * as XLSX from "xlsx";
|
||||
import { replicateTranslateBatch } from "../replicate-translate";
|
||||
|
||||
export type SheetColumnInfo = {
|
||||
sheetName: string;
|
||||
columns: string[]; // header names or A, B, C...
|
||||
};
|
||||
|
||||
export type ColumnSelection = {
|
||||
sheetName: string;
|
||||
columnIndices: number[]; // 0-based column indices to translate
|
||||
};
|
||||
|
||||
/**
|
||||
* Parse an Excel/CSV buffer and return sheet/column metadata for column selection UI.
|
||||
*/
|
||||
export function getExcelColumns(buffer: Buffer, filename: string): SheetColumnInfo[] {
|
||||
const wb = XLSX.read(buffer, { type: "buffer" });
|
||||
return wb.SheetNames.map(sheetName => {
|
||||
const ws = wb.Sheets[sheetName];
|
||||
const range = XLSX.utils.decode_range(ws["!ref"] ?? "A1");
|
||||
const columns: string[] = [];
|
||||
for (let c = range.s.c; c <= range.e.c; c++) {
|
||||
// Try to get header from first row
|
||||
const cellAddr = XLSX.utils.encode_cell({ r: 0, c });
|
||||
const cell = ws[cellAddr];
|
||||
const header = cell && cell.v != null ? String(cell.v) : XLSX.utils.encode_col(c);
|
||||
columns.push(header);
|
||||
}
|
||||
return { sheetName, columns };
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate selected columns in an Excel buffer. Returns translated buffer.
|
||||
* columnSelections: array of {sheetName, columnIndices}
|
||||
* If columnSelections is empty, all text columns are translated.
|
||||
*/
|
||||
export async function translateExcel(
|
||||
buffer: Buffer,
|
||||
targetLanguage: string,
|
||||
columnSelections: ColumnSelection[]
|
||||
): Promise<Buffer> {
|
||||
const wb = XLSX.read(buffer, { type: "buffer", cellStyles: true, cellNF: true });
|
||||
|
||||
for (const sheet of wb.SheetNames) {
|
||||
const ws = wb.Sheets[sheet];
|
||||
if (!ws["!ref"]) continue;
|
||||
|
||||
const range = XLSX.utils.decode_range(ws["!ref"]);
|
||||
const selection = columnSelections.find(s => s.sheetName === sheet);
|
||||
const columnsToTranslate = selection
|
||||
? selection.columnIndices
|
||||
: Array.from({ length: range.e.c - range.s.c + 1 }, (_, i) => i + range.s.c);
|
||||
|
||||
// Collect all text cells for batch translation
|
||||
type CellRef = { addr: string; text: string };
|
||||
const cellRefs: CellRef[] = [];
|
||||
|
||||
for (const colIdx of columnsToTranslate) {
|
||||
// Start from row 1 to skip headers (row 0)
|
||||
for (let r = range.s.r + 1; r <= range.e.r; r++) {
|
||||
const addr = XLSX.utils.encode_cell({ r, c: colIdx });
|
||||
const cell = ws[addr];
|
||||
if (cell && cell.t === "s" && typeof cell.v === "string" && cell.v.trim()) {
|
||||
cellRefs.push({ addr, text: cell.v });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (cellRefs.length === 0) continue;
|
||||
|
||||
// Translate in batches of 50
|
||||
const BATCH_SIZE = 50;
|
||||
for (let i = 0; i < cellRefs.length; i += BATCH_SIZE) {
|
||||
const batch = cellRefs.slice(i, i + BATCH_SIZE);
|
||||
const translations = await replicateTranslateBatch(
|
||||
batch.map(c => c.text),
|
||||
targetLanguage
|
||||
);
|
||||
batch.forEach((cellRef, idx) => {
|
||||
const cell = ws[cellRef.addr];
|
||||
if (cell) {
|
||||
cell.v = translations[idx];
|
||||
if (cell.h) cell.h = translations[idx];
|
||||
if (cell.r) cell.r = undefined;
|
||||
if (cell.w) cell.w = translations[idx];
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const out = XLSX.write(wb, { type: "buffer", bookType: "xlsx" });
|
||||
return Buffer.from(out);
|
||||
}
|
||||
12
utils/document-processors/pdf-types.d.ts
vendored
Normal file
12
utils/document-processors/pdf-types.d.ts
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
declare module "pdf-parse/lib/pdf-parse.js" {
|
||||
interface PdfData {
|
||||
numpages: number;
|
||||
numrender: number;
|
||||
info: Record<string, unknown>;
|
||||
metadata: Record<string, unknown>;
|
||||
text: string;
|
||||
version: string;
|
||||
}
|
||||
function pdfParse(dataBuffer: Buffer, options?: Record<string, unknown>): Promise<PdfData>;
|
||||
export = pdfParse;
|
||||
}
|
||||
115
utils/document-processors/pdf.ts
Normal file
115
utils/document-processors/pdf.ts
Normal file
@@ -0,0 +1,115 @@
|
||||
import { PDFDocument, rgb, StandardFonts } from "pdf-lib";
|
||||
import { replicateTranslateBatch } from "../replicate-translate";
|
||||
|
||||
type PdfParseResult = {
|
||||
numpages: number;
|
||||
text: string;
|
||||
info: Record<string, unknown>;
|
||||
};
|
||||
|
||||
async function parsePdf(buffer: Buffer): Promise<PdfParseResult> {
|
||||
// Avoid Next.js issues with pdf-parse test file imports
|
||||
// eslint-disable-next-line @typescript-eslint/no-var-requires
|
||||
const pdfParse = require("pdf-parse/lib/pdf-parse.js");
|
||||
return pdfParse(buffer);
|
||||
}
|
||||
|
||||
function wrapText(text: string, maxCharsPerLine: number): string[] {
|
||||
const words = text.split(/\s+/);
|
||||
const lines: string[] = [];
|
||||
let current = "";
|
||||
|
||||
for (const word of words) {
|
||||
if ((current + " " + word).trim().length > maxCharsPerLine) {
|
||||
if (current) lines.push(current);
|
||||
current = word;
|
||||
} else {
|
||||
current = current ? current + " " + word : word;
|
||||
}
|
||||
}
|
||||
if (current) lines.push(current);
|
||||
return lines;
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate a PDF buffer. Since PDFs don't support in-place text editing,
|
||||
* this extracts text, translates it, and creates a new formatted PDF.
|
||||
*/
|
||||
export async function translatePdf(
|
||||
buffer: Buffer,
|
||||
targetLanguage: string,
|
||||
sourceLanguage?: string
|
||||
): Promise<Buffer> {
|
||||
const parsed = await parsePdf(buffer);
|
||||
const rawText = parsed.text;
|
||||
|
||||
// Split into paragraphs (separated by double newlines or page breaks)
|
||||
const paragraphs = rawText
|
||||
.split(/\n{2,}|\f/)
|
||||
.map(p => p.replace(/\n/g, " ").trim())
|
||||
.filter(p => p.length > 0);
|
||||
|
||||
if (paragraphs.length === 0) {
|
||||
throw new Error("No extractable text found in PDF");
|
||||
}
|
||||
|
||||
// Translate all paragraphs
|
||||
const translations = await replicateTranslateBatch(paragraphs, targetLanguage);
|
||||
|
||||
// Build output PDF
|
||||
const pdfDoc = await PDFDocument.create();
|
||||
const font = await pdfDoc.embedFont(StandardFonts.Helvetica);
|
||||
const boldFont = await pdfDoc.embedFont(StandardFonts.HelveticaBold);
|
||||
|
||||
const PAGE_WIDTH = 595;
|
||||
const PAGE_HEIGHT = 842;
|
||||
const MARGIN = 50;
|
||||
const FONT_SIZE = 11;
|
||||
const TITLE_SIZE = 13;
|
||||
const LINE_HEIGHT = 16;
|
||||
const MAX_LINE_CHARS = 80;
|
||||
|
||||
let page = pdfDoc.addPage([PAGE_WIDTH, PAGE_HEIGHT]);
|
||||
let y = PAGE_HEIGHT - MARGIN;
|
||||
|
||||
function ensureSpace(needed: number) {
|
||||
if (y - needed < MARGIN) {
|
||||
page = pdfDoc.addPage([PAGE_WIDTH, PAGE_HEIGHT]);
|
||||
y = PAGE_HEIGHT - MARGIN;
|
||||
}
|
||||
}
|
||||
|
||||
// Title
|
||||
const title = `Translation to: ${targetLanguage}${sourceLanguage ? ` (from: ${sourceLanguage})` : ""}`;
|
||||
ensureSpace(TITLE_SIZE + LINE_HEIGHT);
|
||||
page.drawText(title, {
|
||||
x: MARGIN,
|
||||
y,
|
||||
size: TITLE_SIZE,
|
||||
font: boldFont,
|
||||
color: rgb(0.2, 0.2, 0.7)
|
||||
});
|
||||
y -= TITLE_SIZE + LINE_HEIGHT;
|
||||
|
||||
// Draw translated paragraphs
|
||||
for (const para of translations) {
|
||||
const lines = wrapText(para, MAX_LINE_CHARS);
|
||||
ensureSpace(lines.length * LINE_HEIGHT + LINE_HEIGHT);
|
||||
|
||||
for (const line of lines) {
|
||||
ensureSpace(LINE_HEIGHT);
|
||||
page.drawText(line, {
|
||||
x: MARGIN,
|
||||
y,
|
||||
size: FONT_SIZE,
|
||||
font,
|
||||
color: rgb(0, 0, 0)
|
||||
});
|
||||
y -= LINE_HEIGHT;
|
||||
}
|
||||
y -= LINE_HEIGHT * 0.5; // paragraph gap
|
||||
}
|
||||
|
||||
const pdfBytes = await pdfDoc.save();
|
||||
return Buffer.from(pdfBytes);
|
||||
}
|
||||
83
utils/replicate-translate.ts
Normal file
83
utils/replicate-translate.ts
Normal file
@@ -0,0 +1,83 @@
|
||||
import { readSettings } from "./settings-store";
|
||||
|
||||
type ReplicateOutput = string | string[] | { translation?: string; translated_text?: string; output?: string };
|
||||
|
||||
export async function replicateTranslate(
|
||||
text: string,
|
||||
targetLanguage: string
|
||||
): Promise<string> {
|
||||
const settings = readSettings();
|
||||
|
||||
if (!settings.replicateApiToken) {
|
||||
throw new Error("Replicate API token not configured");
|
||||
}
|
||||
if (!settings.jigsawApiKey) {
|
||||
throw new Error("JigsawStack API key not configured");
|
||||
}
|
||||
|
||||
const body = {
|
||||
version: settings.modelVersion,
|
||||
input: {
|
||||
text,
|
||||
api_key: settings.jigsawApiKey,
|
||||
target_language: targetLanguage
|
||||
}
|
||||
};
|
||||
|
||||
const response = await fetch("https://api.replicate.com/v1/predictions", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Authorization": `Bearer ${settings.replicateApiToken}`,
|
||||
"Content-Type": "application/json",
|
||||
"Prefer": "wait"
|
||||
},
|
||||
body: JSON.stringify(body)
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const err = await response.text();
|
||||
throw new Error(`Replicate API error: ${response.status} ${err}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (data.error) {
|
||||
throw new Error(`Replicate model error: ${data.error}`);
|
||||
}
|
||||
|
||||
// Extract translated text from various output formats
|
||||
const output: ReplicateOutput = data.output;
|
||||
|
||||
if (typeof output === "string") return output;
|
||||
if (Array.isArray(output)) return output.join("");
|
||||
if (output && typeof output === "object") {
|
||||
return output.translation ?? output.translated_text ?? output.output ?? String(output);
|
||||
}
|
||||
|
||||
throw new Error("Unexpected output format from Replicate");
|
||||
}
|
||||
|
||||
// Batch translate using separator trick to minimize API calls
|
||||
const SEPARATOR = "\n{{SEP}}\n";
|
||||
|
||||
export async function replicateTranslateBatch(
|
||||
texts: string[],
|
||||
targetLanguage: string
|
||||
): Promise<string[]> {
|
||||
if (texts.length === 0) return [];
|
||||
if (texts.length === 1) {
|
||||
return [await replicateTranslate(texts[0], targetLanguage)];
|
||||
}
|
||||
|
||||
const joined = texts.join(SEPARATOR);
|
||||
const translated = await replicateTranslate(joined, targetLanguage);
|
||||
|
||||
// Try to split on the separator; fall back to individual calls if it got translated
|
||||
const parts = translated.split(SEPARATOR);
|
||||
if (parts.length === texts.length) {
|
||||
return parts;
|
||||
}
|
||||
|
||||
// Fallback: translate individually
|
||||
return Promise.all(texts.map(t => replicateTranslate(t, targetLanguage)));
|
||||
}
|
||||
48
utils/settings-store.ts
Normal file
48
utils/settings-store.ts
Normal file
@@ -0,0 +1,48 @@
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
|
||||
export type Settings = {
|
||||
replicateApiToken: string;
|
||||
jigsawApiKey: string;
|
||||
modelVersion: string;
|
||||
replicateEnabled: boolean;
|
||||
adminPasswordHash: string;
|
||||
};
|
||||
|
||||
const DEFAULT_SETTINGS: Settings = {
|
||||
replicateApiToken: process.env["REPLICATE_API_TOKEN"] ?? "",
|
||||
jigsawApiKey: process.env["JIGSAWSTACK_API_KEY"] ?? "",
|
||||
modelVersion: "jigsawstack/text-translate:454df4c49941c05dea05175bd37686d0872c73c1f9366d1c2505db32ade52a89",
|
||||
replicateEnabled: false,
|
||||
adminPasswordHash: process.env["ADMIN_PASSWORD"] ?? "admin"
|
||||
};
|
||||
|
||||
const SETTINGS_PATH = path.join(process.cwd(), "data", "settings.json");
|
||||
|
||||
function ensureDataDir() {
|
||||
const dir = path.dirname(SETTINGS_PATH);
|
||||
if (!fs.existsSync(dir)) {
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
}
|
||||
}
|
||||
|
||||
export function readSettings(): Settings {
|
||||
try {
|
||||
ensureDataDir();
|
||||
if (!fs.existsSync(SETTINGS_PATH)) {
|
||||
return { ...DEFAULT_SETTINGS };
|
||||
}
|
||||
const raw = fs.readFileSync(SETTINGS_PATH, "utf-8");
|
||||
return { ...DEFAULT_SETTINGS, ...JSON.parse(raw) };
|
||||
} catch {
|
||||
return { ...DEFAULT_SETTINGS };
|
||||
}
|
||||
}
|
||||
|
||||
export function writeSettings(updates: Partial<Settings>): Settings {
|
||||
ensureDataDir();
|
||||
const current = readSettings();
|
||||
const next = { ...current, ...updates };
|
||||
fs.writeFileSync(SETTINGS_PATH, JSON.stringify(next, null, 2), "utf-8");
|
||||
return next;
|
||||
}
|
||||
Reference in New Issue
Block a user