Webtoon storyboard translator
Jake Tae-moon Jo · Engineering lead · https://jake-jo-portfolio.vercel.app/en/work/webtoon-translation/
Built a job-queue app that finds the text in speech balloons with OCR from uploaded JPG pages, translates it, and returns, per language, a PDF with the translation overlaid and a table of source and translated text.
- target languages (from KO or JA)
- 5
- balloon blocks found in one sample
- 63 blocks
- job states, queued to done
- 5
Problem and context
The translation operations team took pages that customers sent as JPG bundles, merged them into a PDF by hand, translated the balloon text with GPT and Google Translate, and overlaid it on the PDF. The only step that truly needed a person's judgment was finding the text areas inside the balloons, so I judged that step could go to a vision model.
The plan also recorded a risk: the text positions OCR finds can be wrong. So the core design question was what people would use to check and fix the output when OCR got it wrong.
One translation job, end to end
A person only uploads the pages and downloads the result; because OCR can be wrong, every overlaid PDF comes with a source-and-translation table.
- Create a jobPerson
Several JPG pages, any of 5 target languages, choice of OCR provider
- QueueSystem
An in-app worker claims jobs with row locks (SKIP LOCKED)
- Build the PDFSystem
JPG pages into one PDF
- OCR and translateExternal
Page slices go to a vision model in parallel; 5xx is retried; 8% margin on each balloon box
- ComposeSystem
Overlaid PDF plus a block-by-block XLSX of source and translation; reruns overwrite
- DownloadPerson
PDF and XLSX per language; progress polled every 2 seconds
What I built
- A job-queue app. After a JWT login you create a job with several JPG pages, any of 5 target languages and an OCR provider, poll its progress every 2 seconds, and download a PDF and an XLSX per language.
- A 5-state job pipeline: queued, build PDF, OCR and translate, compose, done. An in-app worker claims jobs with database row locks (SKIP LOCKED), so there is no separate queue server.
- A swappable OCR and translation adapter. GPT-4o vision and Gemini can be chosen, a page is cut into slices and called in parallel, and 5xx errors are retried. Text positions use box_2d coordinates plus an 8% margin.
- Because OCR can be wrong, I chose a correspondence table as the safety net instead of a correction screen. The compose stage writes the overlaid PDF together with an XLSX of block ID, source and translated text, and a rerun overwrites it. The screen for fixing positions by hand stayed out of v1, and the risk table says "OCR position accuracy → XLSX as the safety net, correction screen in v2".
Project facts
- Period
- Apr 2026
- Role
- Only human on the build team; designed and ran the AI agent team
- Team
- Just me (all 18 commits are mine, all AI co-authored)
- Platforms
- Web app (job queue · internal tool)
- Stack
- Next.js 14 · TypeScript · Prisma · PostgreSQL · JWT (jose) · OpenAI · Gemini · pdf-lib · exceljs · Vitest · Playwright
- Context
- A client's internal tool supporting storyboard translation operations for webtoons
Results
One sample ran from upload to PDF and XLSX on a local machine; there is no record of production use.
- One storyboard sample ran from upload to PDF and XLSX on a local machine. The smoke run in the work log shows 13 slices, 63 blocks and 40–90 seconds, for one sample on the developer's machine.
- Kickoff was on 2026-04-22, and the next day the commits covered job creation and upload, the PDF stage, OCR and compose. All 18 commits carry an AI co-author trailer.
1 more result
- There is no record of production use.
Only what one sample on the developer's machine showed is stated. Production use and test pass counts are not given.
From kickoff to upload-to-compose
By the second day of commits the work ran from upload to compose, and there is no record of production use.
- Translator kickoff
Plan, spec, authentication
- Upload connected to compose
Job creation and upload, PDF stage, OCR (Gemini), compose commits. One local sample: 13 slices, 63 blocks, 40–90 s
- Last work-log entry
E2E, deployment and observability appear only in the log; commits stop at the compose stage