Nimani tekshirishni aytadigan dublyaj
Avtomatik dublyajning har bir qadami — model, xatolar esa toʻplanib boradi: notoʻgʻri eshitilgan soʻz ishonchli notoʻgʻri tarjimaga, soʻng notoʻgʻri narsani aytadigan ravon sunʼiy nutqqa aylanadi. DUBBE javobi — dublyaj yonidagi ikkinchi natija: inson tekshirishi kerak boʻlgan gaplarning qisqa roʻyxati, video vaqt shkalasida.
Capstone loyihasi
Applied AI & Business Capstone Design, Inha University × HumbleBeeAI, 2026-yil kuzi: 1-jamoa, C2b loyihasi, videoni tarjima qilish va dublyaj.
Jamoa
Modellar va litsenziyalar
Har bir qadam — ijaraga olingan GPUʼlarda oʻzimiz ishga tushiradigan ochiq model. Jonli koreys va oʻzbek ovozlari (MMS) — hali notijorat boʻlgan yagona qism.
Nutqni tanish
Whisper large-v3 / large-v3-turbo (faster-whisper); Uzbek: rubaistt v2
MIT · Apache 2.0Ovoz va fon
Demucs (htdemucs)
MITKim qachon gapiradi
NVIDIA Nemotron 3 Diarization, with TitaNet-large voice fingerprints
OpenMDW 1.1 · CC BY 4.0Tarjima
MADLAD-400 3B (int8, CTranslate2), fine-tuned by the team for Uzbek; Korean ↔ English second opinion: Qwen3-8B
Apache-2.0 · data ODC-BYIngliz va koreys ovozlari
Ming-3.0-tts 0.5B and CosyVoice 3, from Kokoro-82M preset recordings
MIT · Apache 2.0Oʻzbek va qozoq ovozlari
Navoiy TTS (CosyVoice 2); Piper with the ISSAI Kazakh voice
Apache 2.0 · MIT, voice CC BY 4.0Boshqa tillar
Chatterbox Multilingual; MMS as a fallback
MIT · MMS CC BY-NC 4.0His-tuygʻular va tovushlar
emotion2vec+ large (FunASR); AST on AudioSet
FunASR model licence · BSD 3-ClauseHar bir model, uning litsenziyasi va biz ishlatadigan aniq versiya (29)
| Model | Qadam | Litsenziya | Tijorat maqsadida foydalanish | Versiya |
|---|---|---|---|---|
| Systran/faster-whisper-large-v3 | Speech recognition | MIT (conversion of openai/whisper-large-v3, Apache-2.0) | yes | edaa852ec7 |
| mobiuslabsgmbh/faster-whisper-large-v3-turbo | Speech recognition, live dubbing, heard-back check | MIT (conversion of openai/whisper-large-v3-turbo, MIT) | yes | 0a363e9161 |
| islomov/rubaistt_v2_medium | Speech recognition (Uzbek) | Apache-2.0 | yes | af46011761 |
| htdemucs | Voice and background separation | MIT (Demucs v4 package and weights, dl.fbaipublicfiles.com) | yes | 955717e8-8 |
| nvidia/Nemotron-3-Diarization | Who speaks when | OpenMDW-1.1 | yes | f667ed73ae |
| nvidia/speakerverification_en_titanet_large | Voice fingerprints (speakers) | CC BY 4.0 TitaNet-large by NVIDIA, CC BY 4.0 |
yes | 0dc382f401 |
| microsoft/wavlm-base-plus-sv | Voice fingerprints (fallback without the NeMo environment) | none on the card (upstream microsoft/unilm repository is MIT); team rule 2026-11-11, no licence = MIT | yes | feb593a6c2 |
| Nextcloud-AI/madlad400-3b-mt-ct2-int8 | Translation (MADLAD-400 3B, int8 CTranslate2 conversion of google/madlad400-3b-mt) | Apache-2.0 | yes | aa32bbdeba |
| dubbe-ai/madlad400-3b-uz-ct2-int8 | Translation, MADLAD-400 3B fine-tuned by the team for Uzbek (C81; LoRA merged into google/madlad400-3b-mt, int8 CTranslate2). Local folder, loaded when DUBBE_MADLAD points at it; not uploaded | Apache-2.0 (derivative of google/madlad400-3b-mt); training data OPUS NLLB v1, ODC-BY 1.0 Fine-tuned on OPUS NLLB v1 (data mined by Meta AI, released by AllenAI; ODC-BY 1.0; opus.nlpl.eu) |
yes | sha256-427 |
| Qwen/Qwen3-ASR-1.7B-hf | Second listen (a second recogniser next to Whisper; flags sentences the two hear differently) | Apache-2.0 | yes | bcd2b5b7f3 |
| google/metricx-24-hybrid-large-v2p6-bfloat16 | Translation quality without a reference (MetricX-24, QE mode) | Apache-2.0 | yes | febb720e29 |
| google/mt5-xl | Tokenizer of MetricX-24 (only spiece.model and its configs are downloaded) | Apache-2.0 | yes | 63fc6450d8 |
| qwen3:8b | Second opinion and Natural mode, Korean<->English (Qwen3-8B, 4-bit, through a local Ollama) | Apache-2.0 | yes | 500a1f067a |
| facebook/nllb-200-3.3B | Translation baseline in scripts/mt_bench.py only (tokenizer); replaced by MADLAD in C75 | CC BY-NC 4.0 | no — being replaced | 1a07f7d195 |
| KomorebiAI/nllb-200-3.3B-int8-ct2 | Translation baseline in scripts/mt_bench.py only (int8 weights) | CC BY-NC 4.0 (conversion of facebook/nllb-200-3.3B) | no — being replaced | 85ad380f5e |
| inclusionAI/Ming-3.0-tts-0.5B | Voices (English, Korean, acted emotions) | MIT | yes | 7247f4c94a |
| FunAudioLLM/Fun-CosyVoice3-0.5B-2512 | Voices (English, Korean; retry voice) | Apache-2.0 | yes | 29e01c4e8d |
| FunAudioLLM/CosyVoice2-0.5B | Voices (the base Navoiy builds on) | Apache-2.0 | yes | eec1ae6c79 |
| aisha-org/navoiy-tts | Voices (Uzbek) | Apache-2.0 | yes | 9445f41ee5 |
| hexgrad/Kokoro-82M | Preset voices; live English voice | Apache-2.0 | yes | f3ff357179 |
| ResembleAI/chatterbox | Voices (21 other languages, Chatterbox Multilingual) | MIT | yes | 5bb1f6ee58 |
| ResembleAI/chatterbox-turbo | Voices (Chatterbox Turbo) | MIT | yes | 749d1c1a46 |
| Supertone/supertonic-3 | Live Korean voice; Korean fallback voice (ONNX on the CPU) | OpenRAIL-M (commercial use allowed with use-based restrictions, Attachment A; sample code MIT) | yes | 3cadd1ee63 |
| facebook/mms-tts-kor | Not used since C78 (Korean is Supertonic-3); kept for comparisons | CC BY-NC 4.0 | no — being replaced | 1b6491366d |
| facebook/mms-tts-uzb-script_cyrillic | Live Uzbek voice; fallback voice | CC BY-NC 4.0 | no — being replaced | a78336f13a |
| rhasspy/piper-voices | Voices (Kazakh, kk_KZ-issai-high) | MIT repository; ISSAI voice CC BY 4.0 (piper-tts package GPL-3.0, see licensing.md) Kazakh voice by ISSAI (Institute of Smart Systems and AI, Nazarbayev University), CC BY 4.0 |
yes | e7229f7d33 |
| emotion2vec/emotion2vec_plus_large | Emotion tags | FunASR Model Open Source License 1.1 (use, copy, modify and share with attribution) emotion2vec+ large, FunASR (Alibaba Group) |
to confirm | 6c303ba987 |
| facebook/audioseal | Watermark on the dubbed voice (AI-speech mark) | MIT (code and weights; generator_base.pth, detector_base.pth) | yes | 3c19eba533 |
| MIT/ast-finetuned-audioset-10-10-0.4593 | Sounds heard (laughs, sighs) | BSD-3-Clause | yes | f826b80d28 |
Kod va hujjatlarni yozishda SI vositalari (Claude) ishlatilgan; jamoa hammasini koʻrib chiqqan va sinab koʻrgan.